Lifetimes help for iterator returning reference

help
1

Hi Everyone,

I'm trying to implement an iterator over a sequence of bytes that uses only the stack and retrieves data in arbitrarily-sized chunks. This is for a file system for embedded rust.

I'm aiming for an interface like this:

let mut block = [0; 512];
let file_name = "hello_world.txt";
let fs = FileSystem::new( /* ... */ );
for file_data in fs.iter_file(&file_name, &mut block) {
    // `file_data` and `block` reference the same memory location
}

I'm having difficulty specifying the lifetimes for the Iterator. I'm getting the error "cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements".

I've trimmed my code down and you can find it on the playground here.

Thanks!

2

This isn't really a lifetime issue in that having the right lifetime annotations won't fix it. The problem is that you're trying to retain mutable access via block: &'b mut [u8] while lending out shared access via &'b [u8], and you intend to overwrite the data in each call to Iterator::next. This would invalidate the &'b [u8]s you've already lent out. Since you can actually retain iterator items between iterations—Iterator specifically forces you to have the lifetime of each Item be independent of the lifetime of self—this is a no-go.

You can actually solve the problem as long as you give away ownership of self.block on each call to Iterator::next, but of course then you wouldn't have a buffer to write into on the next iteration (playground). Maybe you could cheat by explicitly giving ownership of the buffer back to the iterator at the end of each iteration...

You could do this if there were a StreamingIterator trait to work with, but that's blocked on having generic associated types (generic associated type RFC). Technically, you can have your own StreamingIterator using higher-ranked trait bounds—like FileIterator<'a, b>: for<'c> StreamingIterator<'c>—but rolling your own like this wouldn't get you very wide support. Doing it with a generic associated type is preferred, so for now that's what we're waiting for.

1 Like
3

There is the streaming_iterator crate, which works even without generic associated types because it's directly specified to return a reference: fn next(&mut self) -> Option<&Self::Item>.

1 Like
4

Cool. I figured there would be some solution(s) in the wild, but I didn't get around to looking. Still won't give you the same support as a blessed solution (things that could work with a StreamingIterator instead require an Iterator), but getting the trait provided methods for free is nice.

5

Thank you both for your responses!

I didn't realize that the lifetimes of iterator items had to be independent of the lifetime of self. I'll look into implementing this using the streaming_iterator crate.

Thanks!