Help understanding a detail of lifetimes

help
1

I've been writing some rust code, and I hit a weird (to me, at least) case that lead to my program failing due to lifetimes not being handled as I would expect. Here's a minimal example that reproduces my issue:

fn foo() -> &'static str { "foo" }

pub fn bar_good<'a>(b: Option<&'a str>) -> &'a str {
    b.unwrap_or_else(|| foo())
}

pub fn bar_bad<'a>(b: Option<&'a str>) -> &'a str {
    b.unwrap_or_else(foo)
}

Playground link (note: the explicit lifetimes aren't required, but I think it makes this post clearer)

My intuition says that both bar_good and bar_bad functions should have the same behavior. For either one, we have an Option<&'a str>, and we call Option<&'a str>::unwrap_or_else, which expects something implementing FnOnce() -> &'a str as its argument. In both cases, I give it a function that returns a &'static str, and because functions are covariant on their output, this satisfies the requirement and the expression evaluates to &'a str.

With the bar_good function, this intuition is met, and everything works as I expect. However, with the bar_bad function, something doesn't match my intuition and I get the error:

error[E0521]: borrowed data escapes outside of function
 --> src/lib.rs:8:5
  |
7 | pub fn bar_bad<'a>(b: Option<&'a str>) -> &'a str {
  |                --  - `b` is a reference that is only valid in the function body
  |                |
  |                lifetime `'a` defined here
8 |     b.unwrap_or_else(foo)
  |     ^^^^^^^^^^^^^^^^^^^^^
  |     |
  |     `b` escapes the function body here
  |     argument requires that `'a` must outlive `'static`

For more information about this error, try `rustc --explain E0521`.

It seems like the fn() -> &'static str being passed as the argument to Option::unwrap_or_else convinces the lifetime checker that I must be calling Option<&'static str>::unwrap_or_else, and it rejects the code because 'a does not outlive 'static, not seeing that it could just loosen the lifetime requirement to 'a and the whole function now passes.

What is going on in the Rust compiler against my intuition that's making this program fail to check the lifetimes? Clearly, something about my intuition must mismatch with the real behavior, but I don't know what.

2

One thing to be aware of is that every function ("function items" like fn foo) has it's own (zero-sized, not a pointer) type. The type is not nameable. Each closure also has an unnameable type (the size depends on what it captures).

Function item types can be cast to function pointers, but are not themselves function pointers.

foo doesn't implement FnOnce() -> &'a str for any 'a,[1] it implements FnOnce() -> &'static str.

Your closure can implement FnOnce() -> &'a str for any 'a by using covariance of the obtained &'static within the closure body. It doesn't have to return &'static str. And if you make it do so...

b.unwrap_or_else(|| -> &'static str { foo() })
// error[E0521]: borrowed data escapes outside of function

These types that return &'static str are like a struct with no generic parameters. So they don't have supertypes you can implicitly coerce to, shortening the lifetime. Trait implementations themselves don't have variance, might be another way to put it.

Compare and contrast with bar here:

fn bar<'a>() -> &'a str { "bar" }

bar is like a struct with a generic lifetime parameter. When you pass the bar, you really pass bar::<'_> where a suitable lifetime can be inferred.

Alternatively you can cast foo to be a function pointer.

b.unwrap_or_else(foo as fn() -> &'a str)

// These also work:
b.unwrap_or_else(foo as fn() -> &'static str)
b.unwrap_or_else(bar::<'static>)

The difference here is that fn() -> &'_ str, like bar, is acting like a struct with a generic lifetime parameter, which is covariant in that lifetime. So this function pointer (and bar's type) does have a supertype that it can implicitly coerce to.

How exactly function item types and closures implement the Fn traits is pretty subtle. Here's another recent post about it in a more complicated setting.

  1. that's actually a bound that's not even recognized as well formed ↩︎

3 Likes
3

EDIT: see comment by @quinedot below

your reasoning about the subtyping relation is correct,FnOnce()-> &'static str is indeed a subtype of FnOnce() -> &'a str, fn foo() -> &'static str can satisfy the bound FnOnce() -> &'a str, the constraint can only be solved by unifying 'a and 'static.

an contrived example

if you can understand the following example, you'll get an rough idea:

fn get_ref() -> &'static isize {
    &42
}
fn option_unwrap_or_else_v1<T>(o: Option<T>, f: fn() -> T) -> T {
    o.unwrap_or_else(f)
}
fn option_unwrap_or_else_v2<T, F>(o: Option<T>, f: F) -> T where F: FnOnce() -> T {
    o.unwrap_or_else(f)
}
fn main() {
    let x = 0;
    // compiles
    let _ = option_unwrap_or_else_v1(Some(&x), get_ref);
    // lifetime error
    let _ = option_unwrap_or_else_v2(Some(&x), get_ref);
}

explanation

type inference happens before lifetime unification. when there are multiple generic type parameters, just like Option<T>::unwrap_or_else<F>(), the generic arguments is deduced at the call site independently.

in your example, at the call site, Self is Option<&'a str> (equivalent to say T is &'a str), F is fn() -> &'static str, these cannot be unified, since the F type contains a fixed lifetime (i.e. 'static), while the T type contains a variable lifetime (i.e. 'a), the compiler tries to unify them by extending 'a to match 'static and failed, thus the error message you see.

if you change the foo function, the following will work:

fn foo<'b>() -> &'b str { "foo" }

pub fn bar_good<'a>(b: Option<&'a str>) -> &'a str {
    b.unwrap_or_else(|| foo())
}

pub fn bar_not_so_bad<'a>(b: Option<&'a str>) -> &'a str {
    b.unwrap_or_else(foo)
}
1 Like
4

Traits aren't types, and implementations of traits doesn't imply a super/subtype relationship either, even when the trait/implementation involves lifetimes.

3 Likes
5

thanks for the correction.

6

One thing to be aware of is that every function ("function items" like fn foo) has it's own (zero-sized, not a pointer) type. The type is not nameable. Each closure also has an unnameable type (the size depends on what it captures).

Function item types can be cast to function pointers, but are not themselves function pointers.

So if I'm understanding you correctly, this is the point. fn() -> &'a str (which is a function pointer) is covariant on 'a, so a fn() -> &'static str can be used where a FnOnce() -> &'a str implementation is expected (because it coerces to fn() -> &'a str), but foo's type isn't of that pointer. There's a hidden, internal type that foo has which implements FnOnce() -> &'static str and can be cast to fn() -> &'static str, but the type of foo is not generic so it can't coerce to anything that implements FnOnce() -> &'a str. Is this correct?

7

That is correct.[1]

  1. Well, it can be cast to a function pointer, and that could take place as a coercion where a function pointer specifically is requested, but I think you get the point; the type of foo doesn't have a supertype that returns a shorter lifetime / implements FnOnce() -> &'a str. ↩︎

2 Likes
8

Just for reference, what @quinedot clearly explained is also written in the Rust Standard Library documentation, under Primitive Type fn » Creating function pointers:

When bar is the name of a function, then the expression bar is not a function pointer. Rather, it denotes a value of an unnameable type that uniquely identifies the function bar. The value is zero-sized because the type already identifies the function. This has the advantage that “calling” the value (it implements the Fn* traits) does not require dynamic dispatch. This zero-sized type coerces to a regular function pointer.

It is also explained more in details in

  • Functions - The Rust Reference

    When referred to, a function yields a first-class value of the corresponding zero-sized function item type, which when called evaluates to a direct call to the function.

  • Function item types - The Rust Reference

    When referred to, a function item, or the constructor of a tuple-like struct or enum variant, yields a zero-sized value of its function item type. That type explicitly identifies the function - [...] - so the value does not need to contain an actual function pointer, and no indirection is needed when the function is called.

    There is no syntax that directly refers to a function item type

    However, there is a coercion from function items to function pointers with the same signature, which is triggered not only when a function item is used when a function pointer is directly expected, but also when different function item types with the same signature meet in different arms of the same if or match.

2 Likes