Cannot infer an appropiate lifetime

Another day, yet another newbie needing help with lifetimes. :grinning_face_with_smiling_eyes:

I'm reading the book and trying to do a simple File walker for learning purposes, here's my code:

// The bound generic F: Read is so I can test it with io::Cursor 
// instead of a real file
struct FileWalker<'a, F: Read> {
    buf: String,
    r: BufReader<&'a mut F>,
}

impl<'a, F: Read> FileWalker<'a, F> {
    pub fn new(file: &'a mut F) -> FileWalker<F> {
        FileWalker {
            buf: String::new(),
            r: BufReader::new(file),
        }
    }

    fn read_line(&mut self) -> Result<&String, io::Error> {
        self.buf.clear();
        if self.buf.len() > MAX_BUF_READER_SIZE {
            self.buf.shrink_to_fit();
        }

        let result = self.r.read_line(&mut self.buf);
        match result {
            Ok(n) => return Ok(&self.buf),
            Err(e) => return Err(e),
        }
    }
}

impl<'a, F: Read> Iterator for FileWalker<'a, F> {
    type Item = &'a String;

    fn next(&mut self) -> Option<Self::Item> {
      match self.read_line() {
        Ok(s) => {
          if s.is_empty() {
            return None;
          }
          return Some(s);
        }
        Err(err) => {
          // TODO - Error handling
          panic!("{}", err);
        }
      }
    }
}

The idea was that it could walk the file with a BufReader so it doesn't load the entire file at once and then call my FileWalker from another function with something like this:

pub fn find(filename: &str, _query: &str) {
    let mut file = File::open(filename)
        .expect("Couldn't open file");

    let walker = FileWalker::new(&mut file);
    for line in walker {
        // Do something with each line, like matching a pattern and 
        // return the filepath
    }
}

But it won't compile: (Click on full error below)

error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/File_Walker.rs:43:18
|
43 | match self.read_line() {
| ^^^^^^^^^

Full error

error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/File_Walker.rs:43:18
|
43 | match self.read_line() {
| ^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime defined on the method body at 42:13...
--> src/File_Walker.rs:42:13
|
42 | fn next(&mut self) -> OptionSelf::Item {
| ^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src/File_Walker.rs:43:13
|
43 | match self.read_line() {
| ^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 39:6...
--> src/File_Walker.rs:39:6
|
39 | impl<'a, F: Read> Iterator for FileWalker<'a, F> {
| ^^
note: ...so that the types are compatible
--> src/File_Walker.rs:42:46
|
42 | fn next(&mut self) -> OptionSelf::Item {
| ______________________________________________^
43 | | match self.read_line() {
44 | | Ok(s) => {
45 | | if s.is_empty() {
... |
53 | | }
54 | | }
| |_____^
= note: expected Iterator
found Iterator

error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/File_Walker.rs:43:18
|
43 | match self.read_line() {
| ^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime defined on the method body at 42:13...
--> src/File_Walker.rs:42:13
|
42 | fn next(&mut self) -> OptionSelf::Item {
| ^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src/File_Walker.rs:43:13
|
43 | match self.read_line() {
| ^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 39:6...
--> src/File_Walker.rs:39:6
|
39 | impl<'a, F: Read> Iterator for FileWalker<'a, F> {
| ^^
note: ...so that the types are compatible
--> src/File_Walker.rs:42:46
|
42 | fn next(&mut self) -> OptionSelf::Item {
| ______________________________________________^
43 | | match self.read_line() {
44 | | Ok(s) => {
45 | | if s.is_empty() {
... |
53 | | }
54 | | }
| |_____^
= note: expected Iterator
found Iterator

I think I kinda understand the problem (correct me if I'm wrong), my FileWalker struct has a string buffer:

struct FileWalker<'a, F: Read> {
    buf: String, // <--------------------- here
    r: BufReader<&'a mut F>,
}

And I'm returning a reference to it here, which could outlive FileWalker, so it needs to be annotated if I'm not mistaken:

        let result = self.r.read_line(&mut self.buf);
        match result {
            Ok(n) => return Ok(&self.buf), // <----------- here
            Err(e) => return Err(e),
        }

And the borrow checker can't infer the lifetime for that reference, but how and where can I annotate it?

I could clone it but that's not how I should fix lifetime issues, right?. Another solution I can think of is to receive the string buffer from outside, e.g.: from the find() function, storing only a reference into FileWalker instead of an owned string buffer, but it defeats the purpose of having an iterable:

struct FileWalker<'a, F: Read> {
    buf: &'a mut String, // <------- Change
    r: BufReader<&'a mut F>,
}
impl<'a, F: Read> FileWalker<'a, F> { 
    pub fn new(file: &'a mut F, buf: &'a mut String) -> FileWalker<F> {  
        FileWalker {  //        ^^^^^^^^^ change
            buf, 
pub fn find(filename: &str, _query: &str) {
    let mut file = File::open(filename)
        .expect("Couldn't open file");

    let mut buf = String::new(); // <---- change
    let walker = FileWalker::new(&mut file, &mut buf);  // <----- change

    for line in walker {
        // Do something with each line, like matching a pattern and 
        // return the filepath
    }
}

That would make FileWalker a little bit pointless, wouldn't it?

I think the main problem has something to do with the structure of my concept. Maybe I'm used to abstracting in a different way of how Rust should work? How should I re-structure my code or annotate the lifetime for the ref without taking a buf from outside and without cloning?

Thanks in advance!

That's impossible by definition - when FileWalker is dropped, buf is dropped too, and any reference to it would dangle.

If you want to use the Iterator trait, you must clone the string. This is because the interface that the trait requires does not allow the returned items to borrow from the iterator.

You can see this in the standard library too. The Lines iterator produces new strings since it uses the Iterator trait. On the other hand, the read_line method which allows reuse of strings does not use the Iterator trait.

So... Am I right If I state that there is no way to annotate the lifetime of the references created in methods from owned struct fields (like 'buf: String' in FileWalker)?

I'm probably wrong but, if you could annotate the lifetime of those references (like '&self.buf') wouldn't the constraints force callers to use FileWalker and the iterable in the same scope (like the find() example in my first post) and it would be safe to use because FileWalker would never be dropped before the last iterable item?

If it's impossible by definition, how would you re-structure it in an idiomatic Rust way?

Change your goal. If you want a reference to a field of a struct, you should not try to have it outlive the struct containing that field.

1 Like

Oh, so you must clone. Is it worth the abstraction?

Is there any better and more idiomatic way to abstract it? Or the average experienced rustician would just use a function instead of such abstraction?

I mean, that's my goal!

I'm not trying to have it outlive the struct but to constraint it so it doesn't outlive it, but as you pointed out, it seems it is not possible with this implementation.

Could you give me an example of how would your implementation look like?

Thank you both for your quick responses!

Aren't you essentially trying to implement StreamingIterator, not Iterator?

If you define the next method as an ordinary method instead of as a trait method, it should work fine.

lol... from the streaming_iterator documentation:

The iterator APIs in the Rust standard library do not allow elements to be yielded which borrow from the iterator itself. That means, for example, that the std::io::Lines iterator must allocate a new String for each line rather than reusing an internal buffer.
streaming_iterator - Rust

That was quite straightforward, it solves all my questions, thank you so much!

Also, I have checked out the code of BufRead.lines() and the implementation is exactly how @alice pointed out: if you want to use Iterator as I wanted to do, you must clone. It's even simpler, it just creates a new String for every next():

#[stable(feature = "rust1", since = "1.0.0")]
#[derive(Debug)]
pub struct Lines<B> {
    buf: B,
}

#[stable(feature = "rust1", since = "1.0.0")]
impl<B: BufRead> Iterator for Lines<B> {
    type Item = Result<String>;

    fn next(&mut self) -> Option<Result<String>> {
        let mut buf = String::new();
        match self.buf.read_line(&mut buf) {
            Ok(0) => None,
            Ok(_n) => {
                if buf.ends_with('\n') {
                    buf.pop();
                    if buf.ends_with('\r') {
                        buf.pop();
                    }
                }
                Some(Ok(buf))
            }
            Err(e) => Some(Err(e)),
        }
    }
}

https://doc.rust-lang.org/src/std/io/mod.rs.html#2597-2623

Very enlightening, I am learning a lot. Thank you!

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.