I'm facing an issue with expressing such a lifetime that allows a resulting value to outlive the object which returned it, but still be valid no more than the original object it belongs too. That issue is related to the infamous lending iterator problem.
So, there's an Encoding that can be passed to the Reader either by reference or by value. The Reader uses the given Encoding to get some Stage from it and return it to the caller of the read method. It works well till I use Reader directly, but wrapping it into an iterator requires addition of a lifetime.
Maybe I misunderstand something, but the lifetime 'a should extend the time when the Encoding is valid even if it's stored in a field (only if E is a reference). How to fix the issue without using unsafe? And would it be correct to transmute the stage result to workaround the issue if no safe Rust is possible here?
fn by_ref<E: Encoding>(enc: &E) -> Option<&Stage> {
let mut by_ref = Stages {
reader: Reader {
enc: enc,
_marker: PhantomData,
},
};
// the result can be returned
by_ref.next()
}
fn by_val<E: Encoding>(enc: E) {
let mut by_val = Stages {
reader: Reader {
enc: enc,
_marker: PhantomData,
},
};
// the result cannot be returned
by_val.next();
}
struct Stage {}
trait Encoding {
fn get_stage(&self) -> Option<&Stage>;
}
impl<E: Encoding> Encoding for &E {
fn get_stage(&self) -> Option<&Stage> {
(*self).get_stage()
}
}
struct Reader<'a, E: Encoding + 'a> {
_marker: PhantomData<&'a Stage>,
enc: E,
}
impl<'a, E: Encoding + 'a> Reader<'a, E> {
fn read(&mut self) -> Option<&'a Stage> {
self.enc.get_stage()
}
}
struct Stages<'a, E: Encoding + 'a> {
reader: Reader<'a, E>,
}
impl<'a, E: Encoding + 'a> Iterator for Stages<'a, E> {
type Item = &'a Stage;
fn next(&mut self) -> Option<Self::Item> {
self.reader.read()
}
}
error: lifetime may not live long enough
--> src/lib.rs:52:9
|
50 | impl<'a, E: Encoding + 'a> Reader<'a, E> {
| -- lifetime `'a` defined here
51 | fn read(&mut self) -> Option<&'a Stage> {
| - let's call the lifetime of this reference `'1`
52 | self.enc.get_stage()
| ^^^^^^^^^^^^^^^^^^
When you pass Encoding by value, it won't live anywhere else that outlives the Reader/Stages. Stages owns the reader, so when Stages is destroyed, it will destroy the Reader and everything in it.
The Iterator trait does not support lending iterators. It promises that always all returned elements outlive the iterator itself, and remain valid at the same time, even after the iterator is destroyed.
let a = iter.next()?;
let b = iter.next()?;
drop(iter);
use_both(a, b);
This code must be valid for every implementation of Iterator.
You would have to change the definition of the trait, to forbid it from owning any stages:
You could also switch to lending iterators. I know of at least three crates that provide this though none of them seem to be updated to take advantage of GATs that were stabilised a while ago.
The Iterator trait does not support lending iterators. It promises that always all returned elements outlive the iterator itself, and remain valid at the same time, even after the iterator is destroyed.
That's exactly what I expected to hear, but hoped that E: Encoding + 'a would mean that E can live exactly as 'a if it's a reference.
You could also switch to lending iterators.
That's not required actually. In my case Stage takes only 3 bytes and can be easily passed by value, so even Option<Stage> will take less space than Option<&Stage>. The question is mostly about lifetimes and meta programming (:
Are you expecting some relationship between E: 'a and the returned value? Note that lifetime bounds enforce a quality about types, not values.[1] And lifetimes in types typically represent the duration of some borrow, and do not represent the liveness scope of the thing being borrowed.
That is, I believe you are misunderstanding something, though it's hard to suss out exactly what the core of that misunderstanding is.
Meaning roughly "&'a E is a valid type"; "typeE is usable at 'a" âŠī¸
Expected is a strong word here, but let's say I wished that it would be possible. If E is a reference then the iterator is giving because the encoding outlives it and stages have the same lifetime scope as the encoding. If E is a value, then the iterator starts to be a lending one because it contains the E inside it.
The two methods in the code snippet were made to demonstrate a difference in behavior I wanted to achieve, and I even did it.
The thing I don't understand here is how I should express lifetimes for the Reader structure, so the read method won't require unsafe code to convert &'_ to &'a while these two lifetimes mean the same if E stores a value.
You can model "returns something which is maybe borrowed" with GATs or related patterns. For example, in terms of lending iterators, a lending iterator implementor can also returned non-loaned items.
trait LendingIterator {
type Item<'a> where Self: 'a;
fn next(&mut self) -> Option<Self::Item<'_>>;
}
// A actually-lending iterator can't implement Iterator, but all
// Iterator implementors can implement this trait
impl<I: Iterator> LendingIterator for I {
type Item<'a> = <Self as Iterator>::Item where Self: 'a;
fn next(&mut self) -> Option<Self::Item<'_>> {
<Self as Iterator>::next(self)
}
}
In a generic context, you'll still be constrained as-if lending is happening.
(At least, unless you also have bounds that ensure the item cannot be borrowing. But for now, any such bound I'm aware also inflicts an unwanted 'static requirement. Example.)
Refinement may some day also contribute a bit to this area (e.g. refine an implementation by returning a longer lifetime than required).
There is no Rust lifetime ('_) you can put on a struct that means "until I'm destructed (or moved or modified or...)". Much less for "until I'm next moved (or...)". I.e., there is no Rust lifetime you can put on a struct that represents the context-specific duration for which it can be borrowed everywhere else in the code base.
The borrow of a lending iterator call, say, doesn't exist outside the context of where the call happens. And neither does the associated lifetime.
Manipulating Rust lifetimes also cannot change how long values last. Borrow checking allows code to compile or not, but does not change the semantics of a program, such as when values get destructed.
(The word "lifetime" can be misleading in Rust. Perhaps replace it with "duration" or "region" in your head.)