Lifetime confusion with function parameter, serde deserialize

[bcs]

Just when I think I am understanding lifetimes the compiler reminds me that I know nothing.

Playground

I want to process a vector of Message that contain data that will be deserialized and processed by the function generic in Deserialize.

#[derive(Debug, Deserialize)]
struct Message {
    body: Option<String>,
}

fn process_message(d: Option<serde_json::Value>) -> bool {
    println!("process_message {:?}", d);
    d.is_some() && d.unwrap().is_object()
}

fn main() -> Result<(), String> {
    handle(process_message)
}

where the String body of Message will be deserialized into (in this case) just a serde_json::Value. Here's handle:

fn handle<'a,F,T>(f: F) -> Result<(), String>
where F: Fn(Option<T>) -> bool,
      T: Deserialize<'a>,
{
    let messages = vec![
        Message { body: Some("{\"foo\":\"bar\"}".to_string()) }
    ];
    
    for message in &messages {
        let input = if let Some(s) = &message.body {
            serde_json::from_str(&s).map_err(|e| format!("oops {:?}", e))?
        } else {
            None
        };
        
        if process_message(input) {
            do_something(&message);
        }
    }
    
    Ok(())    
}

This works fine. But if I replace

        if process_message(input) {
            do_something(&message);
        }

with the parameter for the function

        if f(input) {
            do_something(&message);
        }

I'm told off:

error[E0597]: `messages` does not live long enough
  --> src/main.rs:26:20
   |
18 | fn handle<'a,F,T>(f: F) -> Result<(), String>
   |           -- lifetime `'a` defined here
...
26 |     for message in &messages {
   |                    ^^^^^^^^^ borrowed value does not live long enough
27 |         let input = if let Some(s) = &message.body {
28 |             serde_json::from_str(&s).map_err(|e| format!("oops {:?}", e))?
   |             ------------------------ argument requires that `messages` is borrowed for `'a`
...
48 | }
   | - `messages` dropped here while still borrowed

I imagine the difference being that in the failed case the compiler has inferred the type of input to be T which has the 'a bound. But I don't know why this would be a problem because nothing in the function is intended to survive beyond the scope of the function. What is still borrowing messages at the end of the function?

Help appreciated.

Playground

[kpreid]

'a is a lifetime parameter. That means that the function must be valid for any caller-chosen selection of lifetime. But, in the version where you call f(input), you're

• passing input to f,
• so input must be Option<T>,
• so serde_json::from_str() must be deserializing a T,
• so given the declared bound T: Deserialize<'a>, the input to the deserialization must live for 'a,
• but the input in fact is a borrow of the local variable messages, which does not live for 'a.

We know that messages does not live for 'a because 'a is a (conceptually) caller-chosen parameter and any lifetime the caller can possibly specify would always outlive local variables in the callee.

The way to write this correctly is to not require the lifetime to be a parameter. Rather, we want to say that "T can be deserialized from local temporary data", which is equivalent to "T can be deserialized from data with any lifetime". That bound is written

fn handle<F,T>(f: F) -> Result<(), String>
where
    F: Fn(Option<T>) -> bool,
    for<'a> T: Deserialize<'a>,
{

In fact, this is so common that serde provides an alias for it, serde::de::DeserializeOwned.

fn handle<F,T>(f: F) -> Result<(), String>
where
    F: Fn(Option<T>) -> bool,
    T: DeserializeOwned,
{

[quinedot]

It's pretty much what @kpreid said, so I'll hide it, but...

What I wrote out before this thread got auto-blocked for 16 hours I guess?

(Not a huge deal but in case a mod reads this, that was a little annoying. At least it saved I guess.)

In short, use a higher ranked trait bound:

fn handle<F,T>(f: F) -> Result<(), String>
where F: Fn(Option<T>) -> bool,
      T: for<'a> Deserialize<'a>,

Which with serde is usually spelled:

fn handle<F,T>(f: F) -> Result<(), String>
where F: Fn(Option<T>) -> bool,
      T: DeserializeOwned,

When a function takes a lifetime parameter, the caller chooses the lifetime, and ^[1] the only thing you know for sure is the lifetime is longer than the function body (it lasts for the call-site expression or longer). I could call your original function with 'static for example; maybe that's all my closure works with, and your function signature said it's okay to do so.

The only way a caller can work with a lifetime local to your function body is if they can work with a lifetime with no lower bound (your function body is shorter than anything they could name), and we specify that by using the higher-ranked trait bound (HRTB) for<'a> ....

---

See also
https://serde.rs/lifetimes.html

---

1. modulo other bounds including inferred bounds ↩︎