Why is this argument borrowed forever – only when by a trait object?

A few days ago, I ran into an issue where a trait object would be borrowed forever when a method was called on it. The solution there was to move a lifetime from the trait to the method itself.

Now I've run into a similar issue with the same method! Given the following method signature:

pub trait SearchByteStream<F: Read> {
    fn in_byte_stream<'this, 'f>(&'this self, stream: F) -> Result<
        Box<
            // The iterator reads `F` and references data in `&self`, so it must live as long as both.
            dyn Iterator<Item = Result<SearchResult, std::io::Error>> + 'f
        >,
        ()
    >
    where
        'this: 'f, // `self` should outlive `stream`.
        F: 'f; // `stream`, however, doesn't need to live as long as `self`.
}

And the following call site:

let searchers: Vec<Box<dyn SearchByteStream<&File>>> = /* … */;

for path in &files_to_search {
    let mut file = File::open(path)?;
    for searcher in &searchers {
        let results = searcher.in_byte_stream(&mut file);
    }
}

I get the following error:

error[E0597]: `file` does not live long enough
   --> source/main.rs:310:51
    |
304 |         let mut file = File::open(&path)?;
    |             -------- binding `file` declared here
...
310 |             let results = searcher.in_byte_stream(&file);
    |                                                   ^^^^^ borrowed value does not live long enough
...
345 |     }
    |     - `file` dropped here while still borrowed
...
348 | }
    | - borrow might be used here, when `searchers` is dropped and runs the `Drop` code for type `Vec`
    |
    = note: values in a scope are dropped in the opposite order they are defined

It appears that when searcher.in_byte_stream(&file) is called, file stays borrowed forever, even though the borrow should be released after the method call. What's even more mystifying is, if the declaration of searcher is switched out for a concrete type…

let searchers: Vec<Box<RelativeSearcher>> = /* … */;

…the issue no longer occurs. In the previous thread, @nerditation kindly pointed me in the direction of the “borrow something forever” problem, which appears to be what's happening here, but if that is the case, I don't quite understand why…! And unlike last time, there's nowhere to move the lifetime declaration to.

Does anyone know ① why file stays borrowed forever (since this issue keeps cropping up when I attempt dynamic dispatch, I really need to figure out what it stems from), and ② how to fix this issue?


The essential problem has to do with a Box<dyn Trait> with a method that returns a Box<dyn Iterator<…>>. I've prepared an interactive minimal example of the issue in Playground, in case that might help – it uses a trait named ReduceChunks instead of SearchByteStream and a method named chunk_results instead of in_byte_stream, but the principle is the same. Getting that minimal example to run should translate to the real-world case…!

Type of searchers contains &File, and a type can never have dangling pointers (with exception of for<'a> syntax typically used for callbacks), so this means &File must always outlive searchers. In your case file is dropped before searchers (inside the for path loop), so that is not allowed.

Move <F: Read> to the method too? When you substitute F for a type with a temporary reference, it's going to behave like a temporary reference with a lifetime you can't name, but that lifetime still applies restrictions, and since it's on the trait, it has to outlive instances of the trait.

Or keep <F: Read> on the trait, but make the method take &mut F. SearchByteStream<File> won't have the temporary lifetime, and you'll be able to name the &mut lifetime in the call if you have to.

"file dropped here while still borrowed" is a somewhat sloppy wording, because by definition things in Rust can't be dropped while they're still borrowed. What this is trying to say that the type system requires file to remain borrowed to satisfy your implicit lifetime requirements on the trait, but the structure of the code forces file to be dropped, and it's dropped too soon to satisfy the lifetime requirements (Rust can't make anything live longer, it only cross-checks whether type annotations match reality).

2 Likes

I see! The issue wasn't the dynamic dispatch in and of itself, but the fact that &File was part of the type. The same problem could've happened with a concrete type, then, I suppose.

I would've moved <F: Read> to the method, but that would've rendered the trait non-object-safe, which would've prevented it from being used for dynamic dispatch – rendering the whole exercise moot. Changing to &mut F along with SearchByteStream<File> worked beautifully, however! I truly cannot thank you enough for your kind and detailed explanation – beyond solving my current problem, it's really contributed to my grasp of the language.


In case someone else comes across this issue years down the line, here's a modified version of the Playground example that compiles as it should.

1 Like

Your fix...

fn chunk_results<'x>(&'x self, stream: &'x mut F) -> Box<dyn 'x ..>

...is probably what you want (without further redesign anyway).

But if you want to read more about the error, it's basically the same as what I'm talking about in this comment. Just replace Chars<'c> with &'f File or so.

Thus this also is a workaround for the OP...

//                        vvvvvvv               vv
let iter_builder: Box<dyn for<'f> ReduceChunks<&'f File>> = Box::new(ChunkSummer {});

...but improving the trait/method is probably the better fix.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.