Higher-ranked lifetimes trouble when passing generic fn as an impl argument

Hi folks,
I am struggling to convince rust to compile following piece of code which introduces trait, type with lifetimes implementing this trait, generic function working over this trait and then finally tries to pass this generic into a function instantiated with specific type.
Here is my code


/// Trait without lifetime
pub trait Trait {
    fn cb(&self, y: &[u8]);
}


/// X with lifetime implements Trait
pub struct X<'a> {inner: &'a [u8]}

impl<'a> Trait for X<'a> {
    fn cb(&self, y: &[u8]) {
        dbg!(y);
    }
}

/// generic dispath of Trait with unrelated lifetime involved
fn generic_dispatch<T: Trait>(x: T, y: &mut [u8]) -> T {
    x.cb(y);
    x
}

///                 problems here     vvv
pub fn takes_dispatch_x(d: impl Fn(X, &[u8]) -> X) {
  let buf = [];
  let x = X {inner: &buf};
  d(x, &[]);
}

fn main() {
    takes_dispatch_x(generic_dispatch)
}

The code obviously does not compile as it is because I don't have lifetime on X.
Trying to add one however leads to weird higher-ranked problems:

Replacing the line with pub fn takes_dispatch_x(d: impl for <'a, 'b> Fn(X<'a>, &'b [u8]) -> X<'a>) { (which imho should be the correct signature) p roduces this weird looking error:

error[E0631]: type mismatch in function arguments
  --> <source>:31:22
   |
18 | fn generic_dispatch<T: Trait>(x: T, y: &mut [u8]) -> T {
   | ------------------------------------------------------ found signature defined here
...
31 |     takes_dispatch_x(generic_dispatch)
   |     ---------------- ^^^^^^^^^^^^^^^^ expected due to this
   |     |
   |     required by a bound introduced by this call
   |
   = note: expected function signature `for<'a, 'b> fn(X<'a>, &'b [u8]) -> _`
              found function signature `for<'a> fn(_, &'a mut [u8]) -> _`
note: required by a bound in `takes_dispatch_x`
  --> <source>:24:33
   |
24 | pub fn takes_dispatch_x(d: impl for <'a, 'b> Fn(X<'a>, &'b [u8]) -> X<'a>) {
   |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `takes_dispatch_x`

error: aborting due to previous error

Replacing it with pub fn takes_dispatch_x(d: impl for <'a> Fn(X<'a>, & [u8]) -> X<'a>) { produces this:

error[E0631]: type mismatch in function arguments
  --> <source>:31:22
   |
18 | fn generic_dispatch<T: Trait>(x: T, y: &mut [u8]) -> T {
   | ------------------------------------------------------ found signature defined here
...
31 |     takes_dispatch_x(generic_dispatch)
   |     ---------------- ^^^^^^^^^^^^^^^^ expected due to this
   |     |
   |     required by a bound introduced by this call
   |
   = note: expected function signature `for<'a, 'b> fn(X<'a>, &'b [u8]) -> _`
              found function signature `for<'a> fn(_, &'a mut [u8]) -> _`
note: required by a bound in `takes_dispatch_x`
  --> <source>:24:33
   |
24 | pub fn takes_dispatch_x(d: impl for <'a> Fn(X<'a>, & [u8]) -> X<'a>) {
   |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `takes_dispatch_x`

error: aborting due to previous error

What am I doing wrong?
I obviously need to introduce lifetime on X (call it 'a) and the function should be generic over this lifetime (given that it is generic over Trait)

Thanks for any help and/or insights

1 Like

Solution

fn generic_dispatch<T: Trait>(x: T, y: &[u8]) -> T {
    x.cb(y);
    x
}

pub fn takes_dispatch_x(d: impl for<'a> Fn(X<'a>, &[u8]) -> X<'a>) {
  let buf = [];
  let x = X {inner: &buf};
  d(x, &[]);
}

fn main() {
    takes_dispatch_x(|x, y| generic_dispatch(x, y))
}

The full error offers the solution and you just try to add them as suggested.

help: consider making the bound lifetime-generic with a new `'a` lifetime
   |
23 | pub fn takes_dispatch_x(d: impl for<'a> Fn(X<'a>, &'a [u8]) -> X<'a>) {
   |                                 +++++++     ++++   ++           ++++

help: consider wrapping the function in a closure
   |
30 |     takes_dispatch_x(|x: X<'_>, y: &[u8]| generic_dispatch(x, &mut *y))
   |                      ++++++++++++++++++++                 ++++++++++++

Note: there is a difference between y: &mut [u8] vs &[u8] in d: impl Fn(X, &[u8]) -> X though.

3 Likes

Before going forward -- in case you didn't know, &mut [u8] and &[u8] are different types, so a function that expects a &mut [u8] argument can't meet a bound that says it has to take a &[u8] argument. From here on I'll just make them both &[u8] (but feel free to raise any questions on that point).

Incidentally I recommend #![deny(elided_lifetimes_in_paths)] so that lifetimes are less invisible.

That is the bound you want, but let's look at why you still can't pass in generic_dispatch directly with that bound.[1]

You have:

fn generic_dispatch<T: Trait>(x: T, y: &[u8]) -> T {

Let's note that functions with type parameters act like structs with type parameters. In this case that means, for example:

//              vvvvvvvvvv
generic_dispatch::<String>: Fn(String, &[u8]) -> String
// aka
generic_dispatch::<String>: for<'any> Fn(String, &'any [u8]) -> String

For every type T, there's a different function; there is no function that handles every possible type:

// This doesn't exist
generic_dispatch: for<'any, T> Fn(T, &'any [u8]) -> T

One might say generic_dispatch isn't a function, it's a function constructor -- you have to put a type in to get a function out.

If we replace String with X<...> we get:

// for any *particular* 'a
generic_dispatch::<X<'a>>: Fn(X<'a>, &[u8]) -> X<'a>
// aka
generic_dispatch::<X<'a>>: for<'any> Fn(X<'a>, &'any [u8]) -> X<'a>

Note how 'a isn't part of the higher-ranked binder (for<'any>).

So you had the right idea that T gets replaced by X<'a>. But it can only be replaced by an X<'a> with one particular lifetime 'a. Types that differ by lifetime are still distinct types.

Altogether, this means that generic_dispatch::<_> itself cannot meet this bound:

for <'a, 'b> Fn(X<'a>, &'b [u8]) -> X<'a>

Because generic_dispatch::<X<'a>> depends on 'a, it isn't higher-ranked over 'a.


The introduction of a closure works the same way this works:

fn generic_dispatch_x<'a>(x: X<'a>, y: &[u8]) -> X<'a> {
    generic_dispatch::<X<'a>>(x, y)
}

Unlike type parameters, unconstrained lifetime parameters on functions don't parameterize the function's type:

    // This errors
    generic_dispatch_x::<'static>(X { inner: &[] }, &[]);
error[E0794]: cannot specify lifetime arguments explicitly if late bound lifetime parameters are present
  --> src/main.rs:39:26
   |
39 |     generic_dispatch_x::<'static>(X { inner: &[] }, &[]);
   |                          ^^^^^^^
   |
note: the late bound lifetime parameter is introduced here
  --> src/main.rs:31:23
   |
31 | fn generic_dispatch_x<'a>(x: X<'a>, y: &[u8]) -> X<'a> {

(The parameter is "late bound" because it is unconstrained -- it doesn't appear in any where clauses or outlive bounds.)

So it is the case that

generic_dispatch_x: for<'a> Fn(X<'a>, &[u8]) -> X<'a>

Because generic_dispatch_x isn't a "function constructor"; it's just one function, and it can accept X<'a> with any lifetime.

(Playground.)


  1. Ignoring the &mut [u8]/&[u8] thing. ↩ī¸Ž

3 Likes

Thanks a lot for the explanation. I will still need to wrap my head around it but the explanation is very helpful!