Having problems with nested reference lifetimes

I'm using winnow for parsing alongside bumpalo, and I have some code that goes like this. The problematic part is right at the end.


#[derive(Debug, Clone, Copy)]
pub struct ArenaString<'a> {
    pub content: &'a str,
    pub arena: &'a Bump,
}
// Also a bunch of other traits that make ArenaString usable as a Parser input delegating to the underlying string

pub fn arena_separated<'a, O, O2, E, ParseNext, SepParser>(
    mut item: ParseNext,
    mut sep: SepParser,
) -> impl for<'c> FnMut(&'c mut ArenaString<'a>) -> winnow::Result<BumpVec<'a, O>, E> 
                  + Parser<ArenaString<'a>, BumpVec<'a, O>, E> 
                    // ^^ This ought to be blanket implemented but I'm explicitly asking for it just to be sure 
where
    E: ParserError<ArenaString<'a>>,
    ParseNext: Parser<ArenaString<'a>, O, E>,
    SepParser: Parser<ArenaString<'a>, O2, E>,
{ /* ... */ }

enum Metavar<'a, T> {
    Var(Span<'a>, &'static str),
    Literal(T),
}

impl<'a, T: Syntax<'a>> Syntax<'a> for Metavar<'a, T> {
    fn parse<'b>(input: &'b mut ArenaString<'a>) -> winnow::Result<Self> {
         /* ... */
    }
}

impl<'a> Syntax<'a> for Tags<'a> {
    fn parse<'b>(input: &'b mut ArenaString<'a>) -> winnow:Result<Self> {
        let ArenaString { arena, content } = *input;
        let semi = ArenaString {
            arena,
            content: ";",
        };
        let mut p = arena_separated(Metavar::parse, semi).map(Tags);
        p.parse_next(input)
    }
}

Tags::parse fails to compile with the following error:

177 | impl<'a> Syntax<'a> for Tags<'a> {
    |      -- lifetime `'a` defined here
178 |     fn parse<'b>(input: &'b mut ArenaString<'a>) -> winnow::Result<Self> {
    |                  ----- `input` is a reference that is only valid in the associated function body
...
184 |         let mut p = arena_separated(Metavar::parse, semi).map(Tags);
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |                     |
    |                     `input` escapes the associated function body here
    |                     argument requires that `'a` must outlive `'static`

This feels like it ought to work, because although I'm being passed a &'b mut reference, I can dereference it to get hold of a &'a T which is Copy, and return that inner object. As a result, I can't understand why it's complaining that anything must outlive the current function body or why the inner lifetime must outlive 'static. Can anyone help?

The lifetime <'a> seems like it should be separate, so I don't see what's the complaint about.

Maybe type inference picks wrong lifetimes? Does explicit Metavar::<'a, _>::parse help?

Maybe the message points to a wrong place? Does p.parse_next(input) have the right lifetimes?

I suggest adding #![warn(elided_lifetimes_in_paths)] to the crate.

The signature of parse_next is from winnow::Parser and is fn parse_next(&mut self, i: &mut I) -> Result<O, E>, in this particular case being, I=ArenaString<'a>, O=Tags<'a>.

#![warn(elided_lifetimes_in_paths)] doesn't seem to raise any warnings if I remove the call to arena_separated. Since I don't know how to call it without any error, I can't see if the call itself is missing something.

Specifying the type for Metavar::parse makes no difference

My guess is this would also compile:

let mut p = arena_separated(Metavar::parse, semi).map(Tags);
//  p.parse_next(input)
todo!()

That is, I don't see how 'b could be involved in the call to arena_separated otherwise.

Does this make any difference?

 pub fn arena_separated<'a, O, O2, E, ParseNext, SepParser>(
     mut item: ParseNext,
     mut sep: SepParser,
-) -> impl for<'c> FnMut(&'c mut ArenaString<'a>) -> winnow::Result<BumpVec<'a, O>, E> 
-                  + Parser<ArenaString<'a>, BumpVec<'a, O>, E> 
+) -> impl Parser<ArenaString<'a>, BumpVec<'a, O>, E>
 where

Removing the parse_next call doesn't make any difference.

Replacing the return type of arena_separated results in this error:

error: implementation of `FnOnce` is not general enough
   --> src\core.rs:166:5
    |
166 | /     move |input| {
167 | |         let first_item = item.parse_next(input)?;
168 | |         let arena = &input.arena;
169 | |         let mut container = vec![in arena; first_item];
...   |
174 | |         Ok(container)
175 | |     }
    | |_____^ implementation of `FnOnce` is not general enough
    |
    = note: closure with signature `fn(&'2 mut core::ArenaString<'_>) -> Result<bumpalo::collections::Vec<'_, O>, E>` must implement `FnOnce<(&'1 mut core::ArenaString<'_>,)>`, for any lifetime `'1`...
    = note: ...but it actually implements `FnOnce<(&'2 mut core::ArenaString<'_>,)>`, for some specific lifetime `'2`

Ah, the annotation was driving closure inference. This might fix it...

move |input: &mut _| { ... }

...but I'm still not sure if it matters. (My efforts at reproduction have been fruitless so far.)

That's a bit surprising. Apparently you can get it to compile by removing the call to arena_separated, though, correct? Can you baby-step towards the error? Like each of these lines?

        // let tmp = arena_separated(Metavar::parse, semi);
        // let mut p = tmp.map(Tags);
        // let r = p.parse_next(input);
        todo!() // /* replace with */ r

So replacing the closure opening with move |input: &mut _| does mean that the Fn trait isn't needed in the return value anymore.

In terms of stepping through, the following block does not work:

let ArenaString { arena, content } = *input;
let semi = ArenaString { arena, content: ";" };
let mut p = arena_separated(Metavar::<'_, Tag<'_>>::parse, semi);
// let temp = p.map(Tags);
// p.parse_next(input);
unimplemented!()

But does work if let mut p = ... is commented out. I'm not really sure how to break it down any further than that

That makes it seem like the lifetime on the &mut of input is still around during the call (assuming the error is the same), which it really shouldn't be. I don't have an explanation for that.

Can you spot any differences in the implementation headers or wrong guesses I've made (Syntax, Tags) in this playground?

Okay, I found a possible cause. Do you have this?

impl<'a> Parser<ArenaString<'a>, (), ContextError> for ArenaString<'static> {

If I make that change in my playground, I get:

error[E0521]: borrowed data escapes outside of associated function
  --> src/lib.rs:90:21
   |
86 | impl<'a> Syntax<'a> for Tags<'a> {
   |      -- lifetime `'a` defined here
87 |     fn parse<'b>(input: &'b mut ArenaString<'a>) -> winnow::Result<Self> {
   |                  ----- `input` is a reference that is only valid in the associated function body
...
90 |         let mut p = arena_separated(Metavar::parse, semi).map(Tags);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |                     |
   |                     `input` escapes the associated function body here
   |                     argument requires that `'a` must outlive `'static`

The implementation requires 'a = 'static. I was mislead by this line:

----- `input` is a reference that is only valid in the associated function body

The problem is in the type of *input not, input. It made me think the error was about 'b, not 'a.[1]


  1. And that statement is... sketchily phrased at best. External lifetimes are always longer than the function body. You can't assume they're any longer than immediately after the function ends, but if the lifetime was only valid in the function body, you could borrow a local variable for that lifetime, which you cannot... â†Šī¸Ž

1 Like

The definition I have for that goes,

impl<'a, I, O, E> Parser<I, O, E> for ArenaString<'a>
where
    I: winnow::stream::Compare<&'a str>,
    I: winnow::stream::StreamIsPartial,
    I: winnow::stream::Stream<Slice = O>,
    E: winnow::error::ParserError<I>,
{
    fn parse_next(&mut self, input: &mut I) -> winnow::Result<O, E> {
        self.content.parse_next(input)
    }
}

How do these compare? ('static in any position causes problems in my playground.)

//                           v                     vv
impl winnow::stream::Compare<&str> for ArenaString<'_> {
    // ...
}

//                                                       vv
impl<'a> winnow::stream::StreamIsPartial for ArenaString<'a> {
    type PartialState = &'a str;
    // ...
}

(Playground.)

That was finally it. My Compare implementation was against &'static str

This is clearly a poor/misleading error message. It would be great to file an issue for it, if the code could be minimized.

1 Like