Mutable borrows in a loop

help
1

My non-blocking socket receiver implementation uses serde and bincode to deserialize borrowed values from an internal buffer. Here is the condensed version:

struct Receiver {
    buf: Vec<u8>,
}

impl Receiver {
    fn try_recv<'a, 'de, T>(&'a mut self) -> Option<T>
        where 'a: 'de,
              T: serde::Deserialize<'de>,
    {
        // In my actual code, self.buf is filled from a non-blocking socket...
        self.buf.push(27);
        // ...and the minimum length is determined through other means.
        if self.buf.len() >= 10 {
            let v: T = bincode::deserialize(&self.buf).unwrap();
            Some(v)
        } else {
            None
        }
    }
}

fn recv<'a, 'de, T>(receiver: &'a mut Receiver) -> T
    where 'a: 'de,
          T: serde::Deserialize<'de>
{
    // Loop until an entire instance of T has been received
    loop {
        if let Some(v) = receiver.try_recv() {
            return v;
        } else {
            // Do other stuff, sleep, etc.
        }
    }
}

fn main() {
    let mut receiver = Receiver { buf: vec![] };
    let v = recv::<u32>(&mut receiver);
    println!("Received v={}", v);
}

But I get the following error:

error[E0499]: cannot borrow `*receiver` as mutable more than once at a time
  --> src/main.rs:25:26
   |
20 | fn recv<'a, 'de, T>(receiver: &'a mut Receiver) -> T
   |             --- lifetime `'de` defined here
...
25 |         if let Some(v) = receiver.try_recv() {
   |                          ^^^^^^^^-----------
   |                          |
   |                          mutable borrow starts here in previous iteration of loop
   |                          argument requires that `*receiver` is borrowed for `'de`

I think it should be possible to compile this with the proper lifetime annotations, because the mut reference to receiver, which is returned by the try_recv() call, is not carried to the next loop iteration.

Any help would be greatly appreciated.

2

In this simplified example, you can use DeserializeOwned, because your receiver doesn't have to borrow anything.

fn recv<T>(receiver: &mut Receiver) -> T
    where T: serde::de::DeserializeOwned,
{
    // Loop until an entire instance of T has been received
    loop {
        if let Some(v) = receiver.try_recv() {
            return v;
        } else {
            // Do other stuff, sleep, etc.
        }
    }
}

If your receiver does need to borrow, then you won't be able to do so with this code structure. You would instead want to separate the collecting of the buffer from deserialization. Something like this:

struct Receiver {
    buf: Vec<u8>,
}

enum Status {
    Ready,
    NotReady,
}

impl Receiver {
    fn try_recv(&mut self) -> Status {
        // In my actual code, self.buf is filled from a non-blocking socket...
        self.buf.push(27);
        // ...and the minimum length is determined through other means.
        if self.buf.len() >= 10 {
            Status::Ready
        } else {
            Status::NotReady
        }
    }

    fn deserialize<'de, T>(&'de mut self) -> T
        where T: serde::Deserialize<'de>,
    {
        bincode::deserialize(&self.buf).unwrap()
    }
}

fn recv(receiver: &mut Receiver) -> Result<(), ()> {
    // Loop until an entire instance of T has been received
    loop {
        if let Status::Ready = receiver.try_recv() {
            return Ok(())
        } else {
            // Do other stuff, sleep, etc.
        }
    }
}

fn main() {
    let mut receiver = Receiver { buf: vec![] };
    if recv(&mut receiver).is_ok() {
        let v = receiver.deserialize::<u32>();
        println!("Received v={}", v);
    }
}
1 Like
3

Not possible in my actual code, I want to use Deserialize to avoid unnecessary copying. Your suggestion to split the logic into two methods would work, thank you!

Can you explain why the compiler doesn't accept the original solution, respectively why it's not possible to set the lifetimes so that the code would be accepted? It seems to me that the code should be safe, but I've been wrong about that before. :wink:

4

I'll try my best to explain. Consider a reduced test case:

struct Flub<'a> {
    s: &'a str,
}

impl<'a> Flub<'a> {
    fn frobulate(&'a mut self) {}
}

fn foo<'a>(flub: &'a mut Flub<'a>) {
    for _ in 0..3 {
        flub.frobulate();
    }
}

I have a struct with a borrowed string. The frobulate method signature requires that self is borrowed for as long as the &str lives. This is due to the lifetime annotations making the link between self and self.s explicit.

Calling the frobulate method temporarily borrows flub. The temporary borrow will be dropped at the end of the loop, which violates the lifetime requirements.

   |
9  | fn foo<'a>(flub: &'a mut Flub<'a>) {
   |        -- lifetime `'a` defined here
10 |     for _ in 0..3 {
11 |         flub.frobulate();
   |         ^^^^------------
   |         |
   |         mutable borrow starts here in previous iteration of loop
   |         argument requires that `*flub` is borrowed for `'a`

To clarify, we can remove the loop and just call the method twice to see a different error message for the same issue:

struct Flub<'a> {
    s: &'a str,
}

impl<'a> Flub<'a> {
    fn frobulate(&'a mut self) {}
}

fn foo<'a>(flub: &'a mut Flub<'a>) {
    flub.frobulate();
    flub.frobulate();
}

   |
9  | fn foo<'a>(flub: &'a mut Flub<'a>) {
   |        -- lifetime `'a` defined here
10 |     flub.frobulate();
   |     ----------------
   |     |
   |     first mutable borrow occurs here
   |     argument requires that `*flub` is borrowed for `'a`
11 |     flub.frobulate();
   |     ^^^^ second mutable borrow occurs here

If we relax the lifetime requirements on foo and frobulate, this simple example compiles:

struct Flub<'a> {
    s: &'a str,
}

impl<'a> Flub<'a> {
    fn frobulate(&mut self) {}
}

fn foo<'a>(flub: &mut Flub<'a>) {
    for _ in 0..3 {
        flub.frobulate();
    }
}

The method signature for frobulate has now changed so that the caller doesn't have to use the same lifetime on both self and self.s. It can instead choose a shorter lifetime for the borrow on self.

This lifetime elision trick does not work in all cases. In the OP, there is a trait bound on T: serde::Deserialize<'de>. The lifetime annotation in this trait bound is required, which also means you will have to annotate the lifetime on &mut self. What you ended up with is basically the same as the following signature, which in turn is the same basic structure as the original frobulate example:

fn try_recv<'de, T>(&'de mut self) -> Option<T>
    where T: serde::Deserialize<'de>,

To answer your other question, you cannot force anything to live shorter or longer with any lifetime annotation. These are just "labels" for the compiler. Kind of like how variable names are labels for the programmer (though this is not a great analogy). They inform the compiler of how long borrows live in relation to one another, and that's about all.

3 Likes
5

In a very minimal answer, the issue is that lifetimes are a static / compile-time tool, so they cannot be very "dynamic" (i.e., change depending on the branches). That is:

    fn try_recv<'de, T> (self: &'de mut Receiver) -> Option<T>
    where
        T : Deserialize<'de>,
    {

  • (which is a simplification of your function signature)

  • Playground

is requiring that the Receiver be borrowed for the whole 'de lifetime. This is indeed necessary because in the "return Some(T)" branch, the returned T may indeed be borrowing from self during that 'de lifetime. But:

  • Although the 'de-long borrow only needs to be read-only shared (& borrow), because you happen to require a short-lived exclusive (&mut) borrow, you end up requiring that the whole 'de-long borrow be exclusive.

  • Even through the None branch, Rust considers that that None::<T> returned value may still be borrowing the input, exclusively (c.f., previous point), and for the whole 'de lifetime. This is a "limitation" of the type system.

The simplest solution to your problem is to do what @parasyte suggested, and split the mutation and the deserialize borrow in two, so as to start with a short-lived &'short_lived mut borrow that can thus happen multiple times within the loop, and only in the path that breaks the loop do require a long-lived borrow (which, on top of that can be reduced to a shared borrow).

The other option is to prove within the function API that when you return you have not "consumed all the input borrow", we could say, by "returning the remains":


impl Receiver {
    fn try_recv<'de, T> (self: &'de mut Receiver)
      -> Result<
            (T, &'de Receiver), // proof that we get back a `&'de Receiver`
            &'de mut Receiver, // proof that we get back a `&'de mut Receiver`
        >
    where
        T : Deserialize<'de>,
    {
        if self.buf.len() >= 10 {
            let v: T = bincode::deserialize(&self.buf[..]).unwrap();
            Ok((v, self))
        } else {
            Err(self)
        }
    }
}

fn recv<'de, T> (mut receiver: &'de mut Receiver) -> T
where
    T : Deserialize<'de>,
{
    // Loop until an entire instance of T has been received
    loop {
        match receiver.try_recv::</*'de,*/ T>() {
            | Ok((v, _receiver)) => {
                return v;
            },
            | Err(receiver_) => {
                receiver = receiver_;
                if true { todo!(); }
            },
        }
    }
}
2 Likes
6

Thanks @parasyte and @Yandros for the explanation. I'll go with the split methods – that way it's "symmetric" to the Sender implementation, which already has a try_send() and a serialize() method.

7

I love this because it allows passing the borrow all the way through every iteration in cases where there is no other choice! Thanks for bringing this solution to my attention.

1 Like
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.