Implementing a trait for all lifetimes of the struct

help
1

I have this trait:

    pub trait Device<'a> {
        type RxToken: RxToken + 'a;
        type TxToken: TxToken + 'a;

And my implementation for it does the followiing:

    impl<'a: 'd, 'd> Device<'d> for VirtualTunInterface<'a> {
        type RxToken = RxToken;
        type TxToken = TxToken<'a>;

for the struct

#[derive(Clone)]
pub struct VirtualTunInterface<'a> {
    mtu: usize,
    packets_from_inside: Arc<Mutex<VecDeque<Vec<u8>>>>,
    packets_from_outside: Arc<Mutex<VecDeque<Blob<'a>>>>,
}

I have this struct that uses DeviceT which is implemented for all lifetimes:

pub struct SmolStack<'a, 'b: 'a, 'c: 'a + 'b, DeviceT>
where
    DeviceT: for<'d> Device<'d>,
{

And the error happens on this enum that uses that struct:

pub enum SmolStackType<'a, 'b: 'a, 'c: 'a + 'b> {
    VirtualTun(SmolStack<'a, 'b, 'c, VirtualTunDevice<'a>>),
    Tun(SmolStack<'a, 'b, 'c, TunDevice>),
}

I'm getting this error right at VirtualTun(SmolStack...:

error: implementation of `virtual_tun::smoltcp::phy::Device` is not general enough
   --> src/virtual_tun/interface.rs:39:16
    |
39  |       VirtualTun(SmolStack<'a, 'b, 'c, VirtualTunDevice<'a>>),
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `virtual_tun::smoltcp::phy::Device` is not general enough
    | 
   ::: /root/.cargo/git/checkouts/smoltcp-6b7a435d708e03f1/2e784fc/src/phy/mod.rs:260:1
    |
260 | / pub trait Device<'a> {
261 | |     type RxToken: RxToken + 'a;
262 | |     type TxToken: TxToken + 'a;
263 | |
...   |
279 | |     fn medium(&self) -> Medium;
280 | | }
    | |_- trait `virtual_tun::smoltcp::phy::Device` defined here
    |
    = note: `virtual_tun::smoltcp::phy::Device<'0>` would have to be implemented for the type `virtual_tun::virtual_tun::VirtualTunInterface<'a>`, for any lifetime `'0`...
    = note: ...but `virtual_tun::smoltcp::phy::Device<'_>` is actually implemented for the type `virtual_tun::virtual_tun::VirtualTunInterface<'1>`, for some specific lifetime `'1`

(VirtualTunInterface is the same as VirtualTunDevice)

If I understood correctly, the error happens because I'm implementing Device like this: impl<'a: 'd, 'd> Device<'d> for VirtualTunInterface<'a> {, for some lifetime 'a for VirtualTunInterface, but I should implement for all lifetimes of VirtualTunInterface. I don't get it, because for VirtualTunInterface<'a> already means for type VirtualTunInterface with some generic lifetime. What would implementing for all lifetimes look like?

Entire source if needed: smoltcp_cpp_interface/virtual_tun.rs at 1e55dd6e4454404dcdefb0fef31fd81e1dc06907 · lattice0/smoltcp_cpp_interface · GitHub

2

If I understand correctly, it's complaining because for some type VirtualTunInterface<'a>, Device<'d> will only be implemented for 'ds that are shorter than are shorter than (or as long as) 'a.

As in, if 'a were some lifetime shorter than 'static, Device<'static> wouldn't be implemented for VirtualTunInterface<'a>, for example, which is why the for<'d> Device<'d> constraint isn't satisfied.

An impl like:

impl<'a, 'd> Device<'d> for VirtualTunInterface<'a> { ... }

would be valid for any lifetime 'd, but this isn't really useful since any attempt to access/manipulate the data that VirtualTunInterface borrows for 'a will require you to clarify the relationship between 'a and 'd.

Something like:

impl<'a> Device<'a> for VirtualTunInterface<'a> { ... }

also doesn't work since even though VirtualTunInterface<'a>, when parameterized with any lifetime 'a, will implement Device<'a>, it won't implement Device for other lifetimes.

Unfortunately, I don't see any way to have lifetime parameters in VirtualTunInterface corresponding to data you need to access in its impl of Device since Interface requires for<'d> Device<'d>. :frowning:

3

I was going to say that you could maybe make Blob own it's slice instead of borrowing it but I'm not familiar enough with your codebase to know if that's a viable option.

FWIW, when I tried to make that change I got a bunch of (unrelated, I think) borrowck errors that I think have to do with lifetime variance (couldn't seem to shorten a few mutable borrows).

4

I actually need Blob to own a slice because this slice is from a C++ buffer, and it delivers it back to C++ on Drop so C++ can destruct it.

I understood your explanation. I really need VirtualTunInterface to hold a reference so I don't know what to do :frowning:

5

by the way, could you also explain why the Device might need to be implemented for all lifetimes? What's the difference that implement for a generic lifetime?

6

So actually, since you're handling destruction of the slice, I think you can effectively treat it as owned data. I think the lifetime that ends up being put on the slice is made up any ways, right now.

As in you can use Vec::from_raw_parts (this means that the type system won't enforce that you don't modify the slice, but you're taking a *mut u8 for the slice anyways) instead of slice::from_raw_parts in smol_stack_smol_socket_send.

7

I'm not familiar with smoltcp but after poking through src/iface/ethernet.rs a bit, I think it's because of the associated type on the Device trait; anytime receive or transmit are called the tokens that are returned are also bounded by the lifetime on Device which make the bound actually needed tricky.

I thought this was problematic since this meant the compiler can't shorten the lifetime of the tokens since it can't assume the trait is implemented for shorter lifetimes, but I actually don't think is what's happening. I went through and replaced the for<'d> DeviceT: Device<'d> bounds in smoltcp with where DeviceT: Device<'a> bounds on the functions that require it and it doesn't work for reasons I'm not very clear on.

I think what's needed is something like where for<'d> where 'a: 'd DeviceT: Device<'d> but, afaik, that's not something that we can express in Rust today.

8

Indeed I went into the Vec fix and wrapped it into a vec. However, there are various unsafe things that need to be considered: Vec in std::vec - Rust

For example it is not safe to build a Vec<u8> from a pointer to a C char array with length size_t

what about uint8_t with size size_t?

Also,

The ownership of ptr is effectively transferred to the Vec<T> which may then deallocate, reallocate or change the contents of memory pointed to by the pointer at will. Ensure that nothing else uses the pointer after calling this function.

Does that mean I shouldn't destruct the pointer on Drop? I feel unconfortable about Rust destructing C++ memory.

9

Rust can't deallocate C/C++ memory, any more than C or C++ can deallocate Rust memory. Memory has to be deallocated and reclaimed by the same memory allocator that allocated it.

From the earlier analysis you can see that, if Rust's code for Vec attempts to shrink or grow a Vec<T> and that size change results in a move, then Rust will attempt to deallocate the initial Vec memory that it did not allocate, potentially resulting in a SEGFAULT. Therein lies one part of the problem.

Of course the relocated Vec will have been allocated by Rust, so when C/C++ attempts to deallocate it another SEGFAULT may ensue.

Note that there is another issue that impacts LLVM, provenance, that also could be problematic

10

ok so I guess I'll switch back to slices! Just have to figure out how to do it in this case

11

Apologies; you are absolutely correct.

I was saying that you can treat the slice as owned since you're handling it's destruction (by calling the destructor function that's passed to you which, iiuc, then goes and has the C/C++ allocator actually deallocate the memory by calling free/delete). But, as you and @TomP note, using a Vec is the wrong way to go about this and is problematic even if you don't modify the slice (which, iiuc, you don't) since the Vec will attempt to free the raw pointer using the Rust allocator when it is dropped.

I don't know of a type that has the properties that you need (owned but not modifiable, destruction handled by Drop but not on the type itself). You can make one and have it impl Index or AsRef. Or, since the lifetime of the slice that slice::from_raw_parts gives you back doesn't really any meaning in this context, you could just have it give you back slices that are valid for 'static (i.e. std::slice::from_raw_parts::<'static, u8>(...)); this would let you remove the lifetime parameter. Though, if the slice isn't private within Blob (as in, if other things can take references to the slice and hold on to them) then others will be able to access the slice's data after it's freed (when Drop on that Blob instance runs). Making a wrapper type that can AsRef into a &[u8] might be a better idea.

12

these static slices are going to be put inside a queue. I'm not familiar enough with Rust but won't the static variables occupy space forever even if they reference memory from C++? For example, every static variable still holds a pointer to the pointer and the boundary. These 2 data would be kept forever even if the slice goes out of scope

13

I made a minimum reproducible example, can you take a look and see if nothing comes up?

14

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.