Borrow still here after drop, why?

I want to write parser almost without copying bytes around, but I meet very strange error (the code bellow is senseless because of I try to make example as short as possible):

#[derive(Default)]
struct Buf {
    buf: Vec<u8>,
}

impl Buf {
    fn begin<'a, 'b, 'c>(&'a mut self, new_data: &'b [u8]) -> MyIter<'c>
    where
        'a: 'c,
    {
        self.buf.extend_from_slice(new_data);
        MyIter {
            buf: &mut self.buf,
            off: 0,
        }
    }
}

struct MyIter<'c> {
    buf: &'c mut Vec<u8>,
    off: usize,
}

impl<'c> MyIter<'c> {
    fn next<'d>(&'c mut self) -> Option<Result<&'d [u8], String>>
    where
        'c: 'd,
    {
        if self.off < self.buf.len() {
            let off = self.off;
            self.off = self.buf.len();
            Some(Ok(&self.buf[off..]))
        } else {
            None
        }
    }
}

fn main() {
    let mut buf = Buf::default();
    let mut it = buf.begin(&[1, 2, 3]);
    {
        it.next();
    }
    {
        it.next();
    }
}

The strange thing that it allows only one call of it.next(), if leave only one it.next() call all works fine.
But why, all references should be dropped because of it.next() surrounded by { } ?

error[E0499]: cannot borrow `it` as mutable more than once at a time
  --> src/main.rs:54:9
   |
51 |         it.next();
   |         -- first mutable borrow occurs here
...
54 |         it.next();
   |         ^^
   |         |
   |         second mutable borrow occurs here
   |         first borrow later used here

MyIter<'c> {
    fn next(&'c mut self)

When member function next is called on it. it will be borrowed until it is dropped. (Code lies a bit since this will always be less than whatever is borrowed by lifetime 'c.)

1 Like

This is because you've defined that the lifetime of the returned value must be identical to the lifetime of self (with exclusive borrows there's no subtyping flexibility). Because mut is an exclusive borrow, and you can have only one exclusive self, you can have only one excusive returned value existing.

fn next<'d>(&'c mut self) -> Option<Result<&'d [u8], String>>
    where
        'c: 'd,
  1. There is never anything good happening from adding a lifetime on &self, so don't do that.

  2. Mutable iterators can't be proven correct by the borrow checker. It has to ensure that returned mutable references don't overlap, but can't reason about the logic. It means you will have to use unsafe, or sometimes leverage some other (unsafe internally) construct like an other mutable iterator or split_at_mut (but if I remember correctly, even that is tricky and needs some unintuitive mem::swap).

1 Like

But what is mutable iterator? Intention of my iterator is providing read-only access to data as you can see: &'d [u8].

Or you mean fn next(&mut self) ? But all iterators has the same signature,
I just add lifetime to it.

If you add a relationship between a shared-immutable and exclusive-mutable lifetimes, you end up with something like exclusive-immutable.

Rust Iterator's next intentionally doesn't have any relationship between the iterator and the returned value. This is valid:

let iter = foo.iter();
let a = iter.next();
drop(iter);
using(a);

impl<'c> MyIter<'c> {
    fn next(&mut self) -> Option<Result<&'c [u8], String>>
1 Like
impl<'c> MyIter<'c> {
    fn next(&mut self) -> Option<Result<&'c [u8], String>>

This doesn't compile in my case, that is the reason why I have to specify exact lifetimes everywhere:

impl<'c> MyIter<'c> {
    fn next(&mut self) -> Option<Result<&'c [u8], String>> {
        if self.off < self.buf.len() {
            let off = self.off;
            self.off = self.buf.len();
            Some(Ok(&self.buf[off..]))
        } else {
            None
        }
    }
}
fn main() {
    let mut buf = Buf::default();
    let mut it: MyIter = buf.begin(&[1, 2, 3]);
    {
        let pack = it.next();
        println!("{:?}", pack);
    }
}

Compile error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements

I think it would fit Iterator, because the iterator only borrows the buffer, not owns it. But if you need to own it, then try using the interface of https://docs.rs/streaming-iterator/

The problem is that your interface — and the borrow checker only looks at interfaces, not implementations — allows unsafe behavior.

next(&mut self) {
   if random() { self.buf.clear() }
   …
}

would compile with this interface, but invalidate all references you've given out. That's why mutable iterators in general need a bit of unsafe code, e.g transmute lifetimes, to convince the borrow checker that they won't mutate the references they give out.


Wait a minute! It's not a mutable iterator! You have &mut buf for no reason! Make it shared &buf and it will all work just fine.

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.