Understanding an unconventional lifetime error (stuck on this for days)

First of all, I know that a minimim reproducible example is recommended. I really tried making one but I couldn't. Since I dont have idea of where this error comes from, I couldn't make a minimum reproducible example that has the same error. They all compile ok.

So I ask of you guys a little patience with someone that just arrived in Rust. I've truly read everything from the rust lang tutorials about lifetime but this question makes no sense at all for me. You'll also be helping someone in an open source problem.

Take a look:

    pub struct Item<'a, 'b: 'a> {
        socket: Socket<'a, 'b>,
        refs:   usize
    }

    #[derive(Debug)]
    pub struct Set<'a, 'b: 'a, 'c: 'a + 'b> {
        sockets: ManagedSlice<'a, Option<Item<'b, 'c>>>
    }

    pub struct TunSmolStack<'a, 'b, 'c> {
        sockets: SocketSet<'a, 'b, 'c >,
    }

    impl<'a, 'b, 'c> TunSmolStack<'a, 'b, 'c> {
        pub fn new(interface_name: String) -> Result<TunSmolStack<'a, 'b, 'c>, u32> {
            let device = TunDevice::new("tun").unwrap();
            let neighbor_cache = NeighborCache::new(BTreeMap::new());
            let socket_set = SocketSet::new(vec![]);
            let mut interface = InterfaceBuilder::new(device)
                .neighbor_cache(neighbor_cache)
                .finalize();
            Ok(TunSmolStack {
                sockets: socket_set,
            })
        }

        pub fn add_socket(&mut self, socket_type: SocketType) -> usize {
            match socket_type {
                SocketType::TCP => {
                    let rx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
                    let tx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
                    let socket = TcpSocket::new(rx_buffer, tx_buffer);
                    self.sockets.add(socket);        
                }

My problem is in the self.sockets.add(socket);, but let's see the two structures that are missing:

TcpSocketBuffer is just another name for RingBuffer:

  #[derive(Debug)]
    pub struct RingBuffer<'a, T: 'a> {
        storage: ManagedSlice<'a, T>,
        read_at: usize,
        length:  usize,
    }
    pub fn new<S>(storage: S) -> RingBuffer<'a, T>
            where S: Into<ManagedSlice<'a, T>>,
        {
            RingBuffer {
                storage: storage.into(),
                read_at: 0,
                length:  0,
            }
        }

And here's the TcpSocket's new method:

impl<'a> TcpSocket<'a> {
    #[allow(unused_comparisons)] // small usize platforms always pass rx_capacity check
    /// Create a socket using the given buffers.
    pub fn new<T>(rx_buffer: T, tx_buffer: T) -> TcpSocket<'a>
            where T: Into<SocketBuffer<'a>> {
        let (rx_buffer, tx_buffer) = (rx_buffer.into(), tx_buffer.into());
        let rx_capacity = rx_buffer.capacity();

        // From RFC 1323:
        // [...] the above constraints imply that 2 * the max window size must be less
        // than 2**31 [...] Thus, the shift count must be limited to 14 (which allows
        // windows of 2**30 = 1 Gbyte).
        if rx_capacity > (1 << 30) {
            panic!("receiving buffer too large, cannot exceed 1 GiB")
        }
        let rx_cap_log2 = mem::size_of::<usize>() * 8 -
            rx_capacity.leading_zeros() as usize;

        TcpSocket {
            meta:            SocketMeta::default(),
            state:           State::Closed,
            timer:           Timer::default(),
            assembler:       Assembler::new(rx_buffer.capacity()),
            tx_buffer:       tx_buffer,
            rx_buffer:       rx_buffer,
            timeout:         None,
            keep_alive:      None,
            hop_limit:       None,
            listen_address:  IpAddress::default(),
            local_endpoint:  IpEndpoint::default(),
            remote_endpoint: IpEndpoint::default(),
            local_seq_no:    TcpSeqNumber::default(),
            remote_seq_no:   TcpSeqNumber::default(),
            remote_last_seq: TcpSeqNumber::default(),
            remote_last_ack: None,
            remote_last_win: 0,
            remote_win_len:  0,
            remote_win_shift: rx_cap_log2.saturating_sub(16) as u8,
            remote_win_scale: None,
            remote_has_sack: false,
            remote_mss:      DEFAULT_MSS,
            remote_last_ts:  None,
            local_rx_last_ack: None,
            local_rx_last_seq: None,
            local_rx_dup_acks: 0,
        }
    }

The error is on this line:

self.sockets.add(socket);

Error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'b` due to conflicting requirements
  --> src/virtual_tun/smol_stack.rs:51:30
   |
51 |                 self.sockets.add(socket);        
   |                              ^^^
   |
note: first, the lifetime cannot outlive the lifetime `'b` as defined on the impl at 23:10...
  --> src/virtual_tun/smol_stack.rs:23:10
   |
23 | impl<'a, 'b, 'c> TunSmolStack<'a, 'b, 'c> {
   |          ^^
note: ...but the lifetime must also be valid for the lifetime `'c` as defined on the impl at 23:14...
  --> src/virtual_tun/smol_stack.rs:23:14
   |
23 | impl<'a, 'b, 'c> TunSmolStack<'a, 'b, 'c> {
   |              ^^
note: ...so that the types are compatible
  --> src/virtual_tun/smol_stack.rs:51:30
   |
51 |                 self.sockets.add(socket);        
   |                              ^^^
   = note: expected  `&mut virtual_tun::smoltcp::socket::SocketSet<'_, '_, '_>`
              found  `&mut virtual_tun::smoltcp::socket::SocketSet<'a, 'b, 'c>`

At first, it looks like a simple ownership problem. Let's see it with the lines before, that have the same scope as the new method:

    let rx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
    let tx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
    let socket = TcpSocket::new(rx_buffer, tx_buffer);
    self.sockets.add(socket);    

since sockets lives in self, and I'm trying to add socket from a scope that is gonna be out soon (scope of the new method), this looks like a problem of adding something that is going to be invalid soon.

However, both TcpSocketBuffer::new and TcpSocket::new take its unique parameter by moving it (owning it), not by borrowing it. More than that: both new methods transform its inputs into a ManagedSlice. If we look on the From method on ManagedSlice for a vector std::vec, it owns the vector: rust-managed/slice.rs at bb952078405f436ffd53925a19f50ab4abd9d397 Ā· smoltcp-rs/rust-managed Ā· GitHub so again, no references are being used! So there should be no lifetime problem at all. References aren't being used! Both rx_buffer, tx_buffer go inside socket being moved, and socket goes into self.sockets by being moved. And these 3 objects transform their inputs into ManagedSlices that also move the vector, not borrow a reference to it.

Here's the full code if someone want to give a better read: https://github.com/lucaszanella/smoltcp_openvpn_bridge/blob/ea2ab0670436aab3de3b4742cc9cb089ef7c06ff/src/virtual_tun/smol_stack.rs

I think you need to restrict the lifetimes on the impl line the same way as Set. As written, the compiler can’t assume anything about the relationship between ā€˜a, ā€˜b, and ā€˜c, but Set requires a particular relationship to exist.

Can you explain what the lifetimes denote? Needing three lifetimes on a struct is very very rare, and it indicates that you are just adding lifetimes because the compiler told you to do so, and that the actual solution probably is changing types to be owned.

Please show the definition of SocketSet and the signature of SocketSet::add.

Edit: I see that it's from an external crate. The answer from @2e71828 appears correct, namely that you need to specify the relationships that SocketSet require.

2 Likes

This is interesting..

I have never used the smoltcp libraries before.. but here’s what I found:

Apparently there’s an instance impl<'a> Into<Socket<'a, 'a>> for TcpSocket<'a>.

And .add() on SocketSet<'a,'b,'c> takes T: Into<Socket<'b, 'c>>.

This means that unfortunately by passing a TcpSocket to .add() directly, you’ll get the constraint that 'b and 'c must be the same. This is a bit stupid for two reasons:

First, apparently Socket<'b,'c> is covariant in the 'b, so that you can use a Socket<'c,'c> whereever Socket<'b,'c> is needed provided that 'c: 'b. This means that manually calling the Into instance fixes the problem

self.sockets.add(<TcpSocket as Into<Socket<'c,'c>>>::into(socket));

The second reason is that Socket has a public constructor Socket::Tcp that doesn’t care about the second lifetime parameter of Socket at all, so the following works, too:

self.sockets.add(Socket::Tcp(socket));

(link to a working modified smol_stack.rs)

To make self.sockets.add(socket); work, one would currently need to restrict everything to TunSmolStack<'a, 'b, 'b> and remove the parameter 'c from the impl.

1 Like

I would’ve thought so, too. But for some reason this code (same link as in my previous post) does compile without any problems and I don’t reproduce the <'a, 'b: 'a, 'c: 'a+'b> constraint from SocketSet.

changed to

pub struct TunSmolStack<'a, 'b: 'a, 'c: 'a + 'b>

and

impl<'a, 'b: 'a, 'c: 'a + 'b> TunSmolStack<'a, 'b, 'c> {

but the error persists

the lifetimes are from the library, not mine, so I dont have control over it.

SocketSet is the name for Set. Here you see it: https://github.com/smoltcp-rs/smoltcp/blob/master/src/socket/set.rs#L36

or:

/// An extensible set of sockets.
///
/// The lifetimes `'b` and `'c` are used when storing a `Socket<'b, 'c>`.
#[derive(Debug)]
pub struct Set<'a, 'b: 'a, 'c: 'a + 'b> {
    sockets: ManagedSlice<'a, Option<Item<'b, 'c>>>
}

impl<'a, 'b: 'a, 'c: 'a + 'b> Set<'a, 'b, 'c> {
    /// Create a socket set using the provided storage.
    pub fn new<SocketsT>(sockets: SocketsT) -> Set<'a, 'b, 'c>
            where SocketsT: Into<ManagedSlice<'a, Option<Item<'b, 'c>>>> {
        let sockets = sockets.into();
        Set {
            sockets: sockets
        }
    }

    /// Add a socket to the set with the reference count 1, and return its handle.
    ///
    /// # Panics
    /// This function panics if the storage is fixed-size (not a `Vec`) and is full.
    pub fn add<T>(&mut self, socket: T) -> Handle
        where T: Into<Socket<'b, 'c>>
    {
        fn put<'b, 'c>(index: usize, slot: &mut Option<Item<'b, 'c>>,
                       mut socket: Socket<'b, 'c>) -> Handle {
            net_trace!("[{}]: adding", index);
            let handle = Handle(index);
            socket.meta_mut().handle = handle;
            *slot = Some(Item { socket: socket, refs: 1 });
            handle
        }

When I tried out smoltcp I faced similar problems of having too many lifetimes here but at least that means I can shed some light on their use.

The three different lifetimes come from diffferent levels of borrowed buffers. ManagedSlice<'a, T> (more or less a wrapper around &'a mut [T]) is used for all persistent state, to avoid any dynamic allocation. In this particular instance we have T = Socket which requires Socket: 'a. This type contains in some of its variants a PacketBuffer which has two different slice references of its own: One of metadata/header data and one for the packet contents; with lifetimes 'b, 'c respectively. Note how one of the lifetimes bounds the type of the header data while the other can be arbitrarily longer. This requires 'b: 'a and 'c: 'a so that the Socket lives long enough. This leaves the bound 'c: 'b but honestly I can't find any underlying reason why it is necessary.

I'm not quite sure how it is useful to preserve this longer lifetime 'c in the first place, instead of only borrowing the data buffer for the shorter of the two but this way you could later take the mutable borrow on one of the buffers with its original lifetime. Anyways, this would look nicer if there was some amount of generics or trait-based erasure involved to cleanup entirely dependent lifetimes but this would be some level of overhead. It's the libraries choice, simply.

I'm trying to understand what you wrote.

Ok, so because of impl<'a> Into<Socket<'a, 'a>> for TcpSocket<'a> we get 'b and c the same. But as I said, socket: TcpSocket receives things that contain ManagedSlices, but these ManagedSlices contains the moved vectors, not slices from it (references). Since there are no references being used, why lifetimes are still being calculated?

Now, looking at

impl<'a> Into<Socket<'a, 'a>> for TcpSocket<'a> {
    fn into(self) -> Socket<'a, 'a> {
        Socket::Tcp(self)
    }
}

it simply expects a TcpSocket<'a>, that is, a TcpSocket with lifetime 'a. I don't get which lifetime is it, since no references are being used.

Can you also comment on why the .add requiring 'b and 'c being the same is a problem? I don't see why it would violate anything: 'c must live more or the same as 'b, which indeed happens if 'b='c (dont know how they can be equal since, again, there's no references being used)

Sure. That’s what my last sentence was about:

By which I mean that the following works:

impl<'a, 'b> TunSmolStack<'a, 'b, 'b> {
    pub fn add_socket(&mut self, socket_type: SocketType) -> usize {
        match socket_type {
            SocketType::TCP => {
                let rx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
                let tx_buffer = TcpSocketBuffer::new(vec![0; 1024]);
                let socket = TcpSocket::new(rx_buffer, tx_buffer);
                self.sockets.add(socket);
            }
            _ => unimplemented!()
        }
        0
    }

The problem in the original version, with

impl<'a, 'b, 'c> TunSmolStack<'a, 'b, 'c>

you’re effectively defining a function

fn add_socket<'a,'b,'c>(self: &mut TunSmolStack<'a, 'b, 'c>, socket_type: SocketType) -> usize

Which doesn’t constrain 'b and 'c to be the same in its signature.
By the way, I’ve done some testing around this and came to the conclusion that the constraints 'b: 'a and 'c: 'a + 'b that @2e71828 and @alice thought might be missing are actually added implicitly automatically by the compiler, just by the fact that TunSmolStack<'a, 'b, 'c> contains a SocketSet<'a, 'b, 'c>, so in fact what your declaration would look like, fully spelled out, is:

fn add_socket<'a,'b: 'a,'c: 'a + 'b>(self: &mut TunSmolStack<'a, 'b, 'c>, socket_type: SocketType) -> usize

Now we need to discuss how to read this generic function signature. You need to get your quantifiers right in the interpretation. The type reads as: ā€œThis function (add_socket) can be applied to (a mutable reference to) any TunSmolStack<'a,'b,'c> for any choice of 'a, 'b, and 'c that the caller can make, as long as he makes sure that 'b: 'c and 'c: 'a + 'b hold.ā€

However if the compiler then figures out that the implementation of add_socket actually needs that 'b and 'c need to be the same (needing an additional 'c: 'b constraint over what’s already there), so as a result it gives you an error because your implementation cannot provide what the function signature promises.




And as I discussed in my previous comment, this restriction that 'b and 'c must be the same lifetime is generated rather artificially by what I would consider a flaw in the smoltcp library (in particular in the Into instance for TcpSocket that I mentioned) and I provided two ways to actually lift this limitation without changing the signature. I’m not sure if the link’s click-counter is just not working or if you actually didn’t look at it yet but in case you didn’t notice, I actually pulled downloaded your code, fixed it like I described, tested that it compiles and liked a github gist containing the result.

2 Likes

It's possible to avoid incrementing the link counter by right-clicking on the URL, copying it, and pasting it in the address bar in the same or a new tab.

I think I understood everything, except for this:

why <'a, 'b, 'b> violates the requirement <'a, 'b: 'a, 'c: 'a + 'b>?

It's difficult to me to think of lifetimes in this context because, as I said, no references are being used. Everything is passed by moving. So I don't have idea on the lifetime of rx_buffer, tx_buffer and socket, because the latest one moves the other 2 inside it, and these other 2 moves a vector inside it. So I don't know if in <'a, 'b, 'b>, the 'b lives more than a. For me, if I do this:

fn main() {
let a = 
    TunSmolStack{
        //...
    }
    a.add_socket(SocketType::TCP);//I guess `'b` lifetime, however it gets generated, has the lifetime less than of the main function because its called inside it
}

then 'a would be the lifetime of the scope of main, and thus how can any 'b have a greater lifetime?

And thanks for the clarification. Indeed I was thinking wrong about the quantifiers. Now I understand it better. And yes, I've seen the example. I'll modify my code with it but I'm more intersted in learning than in fixing this problem, as I'll likely arrive at it in the future again.

Another thing I just noted:

let rx_buffer = UdpSocketBuffer::new(Vec::new(), vec![0; 1024]);
let tx_buffer = UdpSocketBuffer::new(Vec::new(), vec![0; 1024]);
let socket = UdpSocket::new(rx_buffer, tx_buffer);
self.sockets.add(socket);

works, because for UdpSocket, the lifetimes are correct:

impl<'a, 'b> Into<Socket<'a, 'b>> for UdpSocket<'a, 'b> {
    fn into(self) -> Socket<'a, 'b> {
        Socket::Udp(self)
    }
}

I talked with the developer about this difference and he said it could be a bug.

I then made the modification:

impl<'a, 'b> Into<Socket<'a, 'b>> for TcpSocket<'a> {
    fn into(self) -> Socket<'a, 'b> {
        Socket::Tcp(self)
    }
}

and even though it compiles, my intellisense still gives the same error. Could be abug in the intellisense, I don't know, but it compiles.

what does 'b even mean here? I'm implementing something for the TcpSocket struct which has only one lifetime, how can 'b even compile and work here?

Perhaps it makes more sense if you view impls, including trait impls as syntactic sugar. In this case, the impl<'a, 'b> Into<Socket<'a, 'b>> for TcpSocket<'a> is actually a function definition for fn. If you streamline everything into an ordinary function definition syntax, it looks like

fn into_tcpsocket_socket<'a, 'b>(self_: TcpSocket<'a>) -> Socket<'a, 'b> {
    Socket::Tcp(self)
}



Another way to look at it in the case of traits: the parameters of traits are just as relevant as the Self type, so lifetimes that only appear in the parameter (by parameter I mean the T in Into<T>) are totally fine. In some sense, the only special treatment that Self gets over trait parameters is that you can have self arguments of type Self (or references to Self, box of Self and a few more).



Just tried this out and am a bit surprised myself that it compiles:

struct S;
impl<'a> S {
    fn f(self) -> &'a S {
        &S
    }
}

I guess the ā€œimpls are just syntactic sugarā€ view is the only thing that helps in this particular case.



....and now I’m noticing that even this compiles:

struct S;
impl<'a,'b,'c,'d> S {
    fn f(self) -> &'a S {
        &S
    }
}

or this

struct S;
trait Tr {}
impl<'a,'b,'c,'d> Tr for S {}

thank you! I think I understood now.

Can you clarify on this: Understanding an unconventional lifetime error (stuck on this for days) - #12 by lucaszanella
?

I'm having a really hard time thinking about lifetimes for things that do not use references at all

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.