How to couple lifetime expiration to function calls

help
1

Hi,

I am trying to develop a crate that internally uses C APIs calls that require a lot of heavy lifting with pointers and type casting to handle data. For performance, I'd like to iterate over a buffer of values without systematically copying it to a new data array (= reuse the buffer), however I can't figure out how to provide guarantees on an iterator for this buffer so that the iterator's lifetime stays between buffer writes.

Basically, what I'm trying to do is lifetime invalidation. Example :

// This should work
buffer.update();
let mut iter = buffer.to_iter();
for j in iter { // loop over the 
    println!("Value: {}", j);
}
// This should not compile
buffer.update();
let mut iter = buffer.to_iter();
for j in iter { // loop over the 
    println!("Value: {}", j);
    buffer.update();
}

Note that if it helps, update() returning the iterator is an option.
! have a hard time wrapping my head around lifetime specification in rust and could not figure out how to do this from the"advanced lifetime" documentation page and other researches.

Is this possible ? If it is, could you provide examples/show me how to achieve this result ?

2

Isn't this requirement the same as what e.g. Vec in the standard library also ensures? E.g.

fn main() {
    let mut buffer = vec![1, 2, 3];
    // This works
    buffer.push(42);
    let mut iter = buffer.iter();
    for j in iter {
        // loop over the
        println!("Value: {}", j);
    }
}

fn main() {
    let mut buffer = vec![1, 2, 3];
    // This does not compile
    buffer.push(42);
    let mut iter = buffer.iter();
    for j in iter {
        // loop over the
        println!("Value: {}", j);
        buffer.push(42);
    }
}

The "trick" is that while the .iter() method takes &'a self in order to return an Iter<'a, T>, the push (i.e. your update) method takes &mut self. Mutable borrows and shared borrows can't overlap, and the lifetime parameter 'a of the Iter<'a, T> couples it with the shared &'a self borrow, so that you cannot create the &mut self for the push call before you get rid of (or at least stop using) the Iter iterator.

1 Like
3

So, in terms of Rust code, the API might look like this to achieve what you want:

use std::marker::PhantomData;

struct Buffer(/* more fields... */);
struct Iter<'a>(PhantomData<&'a ()>, /* more fields... */);

impl Buffer {
    fn to_iter(&self) -> Iter<'_> {
        todo!();
    }
    fn update(&mut self) {
        todo!();
    }
}

type Todo = u8;

impl Iterator for Iter<'_> {
    type Item = Todo;
    fn next(&mut self) -> Option<Self::Item> {
        todo!();
    }
}

fn works(mut buffer: Buffer) {
    // This should work
    buffer.update();
    let mut iter = buffer.to_iter();
    for j in iter {
        // loop over the
        println!("Value: {}", j);
    }
}

fn doesnt_work(mut buffer: Buffer) {
    // This should not compile
    buffer.update();
    let mut iter = buffer.to_iter();
    for j in iter {
        // loop over the
        println!("Value: {}", j);
        buffer.update();
    }
}

(in the playground)

4

Thank you, I'm new to rust and could not figure out how vec did it, as when I tried something similar I did not know of PhantomData (my base type does not support lifetime parameters) and was not fully aware of the implicit mutability constraint on borrows. This helps a lot.

5

The PhantomData is mostly a stand-in so that the lifetime arguments gets used. You might not need it; e. g. if the iterator needs access to the original Buffer struct, then adding a &'a Buffer field to Iter<'a> might be useful and takes care of using the lifetime argument already. If the iterator is its own thing implemented in C, then adding such a PhantomData might make a lot of sense though. By the way, a PhantomData<&'a Buffer> could also make sense to more clearly communicate that the iterator borrows a Buffer.

Note that the to_iter taking &self (and not &mut self) assumes that it's okay to create and use multiple iterators in "parallel" / concurrently / in an interleaved manner, and also that it's okay to call other &self methods (should there be any) on the Buffer during iteration.

6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.