Argument to a boxed function does not live long enough

Hi!

I have a function boxit takes a closure and then returns that closure in a box:

pub trait BoxFn<'a> {
    type Arg;

    fn boxit<F: Fn(&Self::Arg)>(f: &'static F) -> Box<dyn Fn(&'a Self::Arg)> {
        Box::new(move |arg| {
            f(arg);
        })
    }
}

And this is used like so:

pub struct MessageHandler;
impl<'a> BoxFn<'a> for MessageHandler {
    type Arg = String;
}

fn handler(message: &String) {
    println!("Message is {}", message);
}

fn main() {
    let boxed_fn = MessageHandler::boxit(&handler);
    let message = "Test".to_string();
    boxed_fn(&message);
}

However, the complier is complaining that message doesn't live long enough since it could still be borrowed when boxed_fn calls its destructor.

error[E0597]: `message` does not live long enough
  --> src/main.rs:40:14
   |
40 |     boxed_fn(&message);
   |              ^^^^^^^^ borrowed value does not live long enough
...
45 | }
   | -
   | |
   | `message` dropped here while still borrowed
   | borrow might be used here, when `boxed_fn` is dropped and runs the destructor for type `std::boxed::Box<dyn std::ops::Fn(&std::string::String)>`
   |
   = note: values in a scope are dropped in the opposite order they are defined

Why is message still borrowed when the box is destructed? Shouldn't the borrowing end when boxed_fn(&message); returns?

A note on a broader task I'm dealing with: BoxFn is reduced from the actual thing I'm trying to write, which is a function that takes a closure then returns another closure that takes a string, deserializes that string to Arg, then passes it to the input closure:

use serde::{Deserialize};

pub trait Handler<'de> {
    type Arg: Deserialize<'de>;

    fn implement<F: Fn(&Self::Arg)>(f: &'static F) -> Box<dyn Fn(&'de str)> {
        Box::new(move |s| {
            f(&serde_json::from_str(s).unwrap());
        })
    }
}

Well the thing is:

pub trait BoxFn<'a> {
               ^^^^
    type Arg;

    fn boxit<F: Fn(&Self::Arg)>(f: &'static F) -> Box<dyn Fn(&'a Self::Arg)> {
                                                             ^^^
        Box::new(move |arg| {
            f(arg);
        })
    }
}

The two lifetimes I marked are the same. Since the 'a is defined on the trait, this means it could be as large as the lifetime of the thing implementing the trait. And since you marked the argument as having this lifetime, it must live this long.

I'm new to rust so I'm still shaky on how lifetimes are inferred. What do you mean my "the thing implementing the trait"? The struct MessageHandler implements the thread, but since boxit is a static function I didn't create an instance of type MessageHandler, and so what would be the lifetime substituted for 'a there?

I'd super appreciate it if anyone could point at the relevant lifetimes rules involved in this.

TL,DR

"Fixed" lifetimes in argument position, can be "problematic" and universal (HRTB) lifetimes ought to be used.

Instead of:

  • Box<dyn Fn(&'a String)>

you should use:

  • Box<dyn Fn(&'_ String)>

    where '_ stands for an elided lifetime parameter, which in this case representes an universally quantified lifetime (HRTB):

    • Box<dyn for<'any> Fn(&'any String)>

To see why, let's start from:

pub
struct MessageHandler;

impl<'arg> BoxFn<'arg> for MessageHandler {
    type Arg = String;

    fn boxit<F : 'static> (f: F) -> Box<dyn Fn(&'arg String)>
    where
        F : Fn(&'_ String), // for<'any> F : Fn(&'any String)
    {
        Box::new(move |arg| {
            f(arg);
        })
    }
}

/// F = fn { handler }
/// for<'any> F : Fn(&'any String)
fn handler (message: &'_ String)
{
    println!("Message is {}", message);
}

fn main ()
{
    let boxed_fn = MessageHandler::boxit(handler); // Box<dyn Fn(&'arg String)>
    let message = "Test".to_string();
    boxed_fn(&message);
}
  • I have taken the liberty to rename 'a to 'arg to make it more readable, and removed the &'static indirection over the f: F argument, since it is not needed (F : 'static suffices for your signature).

So, now the question is:

what is 'arg?
how far does 'arg go / span?

Indeed, any lifetime parameter represents a lifetime, which can be seen as a span of the program.

Finding / choosing the exact lifetime span is Rust's job: an API bounds lifetimes, either by forcing them to reach some point, or by forbidding them from going further than some point. Rust then tries to find some "code section" / span that matches the given bounds, or triggers a compile error when it fails.

And that's what happens with your code:

  • boxed_fn : Box<dyn Fn(&'arg String)> is a variable whose type carries a 'arg lifetime. This means that 'arg cannot have ended when boxed_fn is used.

    'arg must span beyond the last usage of boxed_fn

    • Thanks to NLL (borrow checker 2.0 of sorts), the last usage point does not necessarily correspond to the point where the variable boxed_fn disappears. But when a variable has drop glue / a destructor, which is the case of Box<_> (the destructor needs to deallocate), then by definition the variable is used right before disappearing. This means that when the type has drop glue, NLL "does not" apply, and instead the old rules take place.

      'arg must span until boxed_fn is dropped.

  • Now, 'arg is also the lifetime of the borrow of the String that is fed to boxed_fn (that's what dyn Fn(&'arg String) means). And by definition a value cannot be borrowed beyond the point where it is dropped:

    'arg must end before message is dropped.

And in the code:

    let boxed_fn = MessageHandler::boxit(&handler);
    let message = "Test".to_string();
    boxed_fn(&message);
}

it is just not possible to meet both requirements. To better see it, know that variables are dropped in reverse order of their declaration. This means that the above code is equivalent to:

    let boxed_fn = MessageHandler::boxit(&handler);
    let message = "Test".to_string();
¤   boxed_fn(&message);
¤
    drop(message);
    drop(boxed_fn);
  ~
  ~
}
  • ¤ represents the span of "end before message is dropped" (but after the borrow starts);

  • ~ represents the span of "until boxed_fn is dropped";

As you can see, these spans do not overlap, hence the error (if they overlapped then Rust would be free to choose any point in the intersection and the code would compile). By the way, by writing down the drops explicitely, the error Rust shows is a little bit more explicit too:

error[E0505]: cannot move out of `message` because it is borrowed
  --> src/main.rs:41:10
   |
40 |     boxed_fn(&message);
   |              -------- borrow of `message` occurs here
41 |     drop(message);
   |          ^^^^^^^ move out of `message` occurs here
42 |     drop(boxed_fn);
   |          -------- borrow later used here
  • An example of the lifetimes overlapping can be seen if we invert the drop order:
    let message = "Test".to_string();
    let boxed_fn = MessageHandler::boxit(&handler);
¤   boxed_fn(&message);
¤   drop(boxed_fn);
¤~ // <--- 'arg can end here --+
 ~  drop(message);
}

The solution

By making the type of boxed_fn not carry an explicit lifetime, while still being "callable", the problem is solved.

And a way to have a "callable with an argument borrowed for some lifetime" interface without infecting the whole Box<dyn ...> type with that specific lifetime, is to just use a non-specific lifetime. Such a non-specific lifetime is "universally quantified" instead, meaning that the callable interface (and bound) does not apply to a specific borrow but to all possible borrows (hence the "universal").

This is what is called a HRTB, and can be written either with the explicit "for all" syntax:

Box<dyn for/*all*/<'borrow> Fn(&'borrow String)>

or, when dealing with Fn{,Mut,Once} traits specifically, it can be written with the "elided lifetime parameter" syntax, i.e., writing down the '_ elided lifetime parameter (explicit elision), or just not writing it at all (implicit elision):

Box<dyn Fn(&'_ String)>
Box<dyn Fn(&String)>

So the following BoxFn trait signature solves your problem:

pub
trait BoxFn {
    type Arg;

    fn boxit<F : 'static> (f: F) -> Box<dyn Fn(&'_ Self::Arg)>
    where
        F : Fn(&'_ Self::Arg), // for<'any> F : Fn(&'any Self::Arg),
    {
        Box::new(f)
    }
}
6 Likes