In a current project, I've run into a dead end in the form of a mighty confusing issue with the borrow checker: When attempting dynamic dispatch via a trait object (i.e. Box<dyn Trait>), any method call that takes &'a self causes the dereferenced object (*thing) to be borrowed, and stay borrowed indefinitely (even past the end of its own scope)! Of course, the Box is dropped when it goes out of scope, meaning that thing is dropped while *thing is borrowed – causing an error.
My real-world case (the details of which you can see further down) concerns a complex trait and a method with a complex return type, but I've pared it down to a minimal example with a Box<dyn Trait> that returns a Box<dyn Iterator<…>> (Playground):
trait ReturnIter<'this> {
// This dyn Iterator<…> needs `+ 'this` specified, lest the compiler require
// it to be 'static. The issue happens with *any* return type (even a non-
// `dyn` one), but this illustrates why the lifetime bound is needed.
fn return_iter(&'this self) -> Box<dyn Iterator<Item = usize> + 'this>;
}
struct Thing {
numbers: Vec<usize>
}
impl<'this> ReturnIter<'this> for Thing {
fn return_iter(&'this self) -> Box<dyn Iterator<Item = usize> + 'this> {
Box::new(self.numbers.iter().map(|&n| n))
}
}
fn main() {
// In the real-world case, I'm using dynamic dispatch for polymorphism,
// which is why `thing` needs to be a trait object.
let thing: Box<dyn ReturnIter> = Box::new(
Thing {numbers: vec![1, 2, 3]}
);
let _iter = thing.return_iter();
}
This causes the compiler to yield the following error:
error[E0597]: `*thing` does not live long enough
--> src\main.rs:25:17
|
21 | let thing: Box<dyn ReturnIter> = Box::new(
| ----- binding `thing` declared here
...
25 | let _iter = thing.return_iter();
| ^^^^^ borrowed value does not live long enough
26 | }
| -
| |
| `*thing` dropped here while still borrowed
| borrow might be used here, when `thing` is dropped and runs the destructor for type `Box<dyn ReturnIter<'_>>`
What I've Gathered
thing.return_dyn()borrows*thing, sincereturn_dyntakes&self– that much is normal.- What's odd, however, is that it appears to stay borrowed past the end of the block when the type of
thingis a trait object (Box<dyn ReturnDyn>, in this case). - Which is an issue, since the
Boxalong with the contents are dropped when they go out of scope, as they should be (`*thing` dropped here while still borrowed). - This happens whatever the method called may be, as long as it takes
&'a self– my case is a bit more complicated due to returning aBox<dyn _>, which requires a lifetime, but the infini-borrow happens either way. - Curiously, this does not happen when
thingis declared as a concrete type (e.g.Box<Thing>) (Playground). - Even more curiously, it only happens when
&selfhas an explicit lifetime –&'this selfcauses the issue, but changed to&self, it doesn't happen.*
To summarize, this is an issue that only happens when you have ① a trait object that ② calls a method that takes &self ③ with an explicit lifetime for &self (e.g. &'a self).
* Changing to
&selfis unworkable in my case, since the returnedBox<dyn Iterator<…>>must have an explicit lifetime specified that's anchored to something else, lest it be assumed to be' static. (Using only&self, we instead get the errormethod was supposed to return data with lifetime `'this` but it is returning data with lifetime `'1`.) In my real-world case, as in the example, it must be anchored to&selfspecifically, since the iterator uses references to data within the struct.
The Real-World Case
Though that's the minimal version of the issue I'm having, I figure it could be relevant what I'm trying to achieve and how – so you can expand this drop-down for an overview of the real-world case:
Details…
The program I'm writing needs to be able to search through text encoded in fixed-width encodings of any width and endianness. To achieve this, I provide type parameters for which integer type to use (PrimInt from the crate num-traits) and which endianness to use (ByteOrder from the crate byteorder). There are also multiple different modes of searching, for which I use different types – meaning that the relevant methods must live in a trait:
pub trait SearchByteStream<'this, F: Read + 'this> {
fn in_byte_stream(&'this self, stream: F) -> Result<
Box<
dyn Iterator<
Item = Result<usize, std::io::Error>
> + 'this
>,
()
>;
}
impl<'this, N: PrimInt, E: ByteOrder + 'static, F: Read + 'this> SearchByteStream<'this, F>
for CodepointSearcher<'this, N, E>
{ /*… */ }
impl<'this, N: PrimInt, E: ByteOrder + 'static, F: Read + 'this> SearchByteStream<'this, F>
for RelativeSearcher<'this, N, E>
{ /* … */ }
impl<'this, N: PrimInt, E: ByteOrder + 'static, F: Read + 'this> SearchByteStream<'this, F>
for FormationSearcher<'this, N, E>
{ /* … */ }
In order to be able to select search mode, encoding width, and endianness at runtime, however, these need to used interchangeably. I've chosen to achieve this using polymorphism via dynamic dispatch:
let file = File::open(path)?;
let searcher: Box<dyn SearchByteStream<&File>> = select_searcher_type!(mode, int_type, endianness);
// Error! `*searcher` borrowed past the end of the block!
for result in searcher.in_byte_stream(&file)? { /* … */ }
I'm befuddled and left with the following questions:
- Why is this error happening? Why does the borrow outlive the scope? Why only when declared as a trait object?
- How might one make it work?
- Failing all else, what else could I do?