Function pointers for (monomorphised versions of) functions that take AsRef<Path> arguments

1

Suppose I have two (or more) generic functions with identical signatures. They're only generic because one of the arguments is an AsRef<Path>. For example

use std::convert::AsRef;
use std::path::Path;

type Result = std::io::Result<()>;

fn operation_1<P: AsRef<Path>>(p: P) -> Result { todo!() }
fn operation_2<P: AsRef<Path>>(p: P) -> Result { todo!() }

fn direct(p: &Path) -> Result { operation_1(p) }

direct is included just to demonstrate that &Path does in fact satisfy a : AsRef<Path> bound, as one would expect. The above compiles (modulo complaints about nothing being public).

To factor repeated code out of some tests, I want to write a wrapper function that takes an &Path and either operation_1 or operation_2 as an argument, and it calls the function it was given (plus it does some other stuff that's not relevant here). As best I can tell, that wrapper would be written something like

fn wrapper
    <P: AsRef<Path>, F: Fn(P) -> Result>
    (f: F, p: &Path) -> Result
{ f(p) }

But if you paste this below the code above, it does not compile. You don't even have to call it, the compiler chokes on the body of the wrapper:

error[E0308]: mismatched types
  --> file_integrity/src/argh.rs:14:5
   |
12 |     <P: AsRef<Path>, F: Fn(P) -> Result>
   |      - this type parameter
13 |     (f: F, p: &Path) -> Result
14 | { f(p) }
   |   - ^ expected type parameter `P`, found `&Path`
   |   |
   |   arguments to this function are incorrect
   |
   = note: expected type parameter `P`
                   found reference `&std::path::Path`
note: callable defined here
  --> file_integrity/src/argh.rs:12:25
   |
12 |     <P: AsRef<Path>, F: Fn(P) -> Result>
   |                         ^^^^^^^^^^^^^^^

And now I'm stuck. I don't understand why &Path is considered not to match P in this context, when P: AsRef<Path> and passing &Path directly to a function with a generic argument of type P: AsRef<Path> is acceptable. Have I written the type signature wrong? (How else could it be written?) Have I tripped over some limit in the type system? Do I need to do something at the call site that wasn't necessary in direct, and if so, what is it, and why? :confused:

Incidentally, I also tried not having wrapper be generic, and instead manually monomorphizing at the call sites...

fn wrapper(f: fn(&Path) -> Result, p: &Path) -> Result { f(p) }
fn wrapped(p: &Path) -> Result { wrapper(operation_1::<&Path>, p) }

... and that didn't work either:

error[E0308]: mismatched types
  --> argh.rs:12:42
   |
12 | fn wrapped(p: &Path) -> Result { wrapper(operation_1::<&Path>, p) }
   |                                  ------- ^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |                                  |
   |                                  arguments to this function are incorrect
   |
   = note: expected fn pointer `for<'a> fn(&'a std::path::Path) -> std::result::Result<_, _>`
                 found fn item `fn(&std::path::Path) -> std::result::Result<_, _> {argh::operation_1::<&std::path::Path>}`
   = note: when the arguments and return types match, functions can be coerced to function pointers
note: function defined here
  --> argh.rs:11:4
   |
11 | fn wrapper(f: fn(&Path) -> Result, p: &Path) -> Result { f(p) }
   |    ^^^^^^^ ----------------------

(What on earth? What does for<'a> fn(&'a Path) even mean that's different from fn(&Path)?

2

Your first wrapper fails because it is asking for a function whose argument type is something its caller gets to choose. But in fact it itself is choosing the argument type to be &Path. The error is because the function signature you wrote would allow calling it like, for example, wrapper::<PathBuf, _>(f, p) but then a &Path would be passed instead of a PathBuf.

The right signature is one without a P type parameter:

fn wrapper<F: Fn(&Path) -> Result>(f: F, p: &Path) -> Result {
    f(p) 
}

Unlike your second attempt, there is still a F parameter. I'm not sure why the function pointer version fails.

1 Like
3

They work just fine if you will ensure that lifetimes match.

4

This gets closer to a working solution, but if you try to call it you get errors similar to, but better signposted than, the manual monomorphization case:

fn wrapper<F: Fn(&Path) -> Result>(f: F, p: &Path) -> Result { f(p) }
pub fn wrapped(p: &Path) -> Result { wrapper(operation_1, p) }

-->

error[E0308]: mismatched types
  --> argh.rs:12:38
   |
12 | pub fn wrapped(p: &Path) -> Result { wrapper(operation_1, p) }
   |                                      ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected trait `for<'a> Fn<(&'a Path,)>`
              found trait `Fn<(&Path,)>`
note: the lifetime requirement is introduced here
  --> argh.rs:11:15
   |
11 | fn wrapper<F: Fn(&Path) -> Result>(f: F, p: &Path) -> Result { f(p) }
   |               ^^^^^^^^^^^^^^^^^^^

error: implementation of `FnOnce` is not general enough
  --> argh.rs:12:38
   |
12 | pub fn wrapped(p: &Path) -> Result { wrapper(operation_1, p) }
   |                                      ^^^^^^^^^^^^^^^^^^^^^^^ implementation of `FnOnce` is not general enough
   |
   = note: `fn(&'2 Path) -> std::result::Result<(), std::io::Error> {operation_1::<&'2 Path>}`
            must implement `FnOnce<(&'1 Path,)>`, for any lifetime `'1`...
   = note: ...but it actually implements `FnOnce<(&'2 Path,)>`, for some specific lifetime `'2`

The note at the bottom clarifies (for me anyway) what the heck the difference is between for<'a> fn(&'a Path) and fn(&Path). Apparently, the lifetime elision rules mean that

fn wrapper<F: Fn(&Path) -> Result>(f: F, p: &Path) -> Result

requires the type signature of any function passed as f to satisfy "FnOnce<(&'1 Path,)>, for any lifetime '1", but the type signature of (the monomorphization of)

pub fn operation_1<P: AsRef<Path>>(_p: P) -> Result { todo!() }`

is actually something like

pub fn operation_1_m1(_p: &'a Path) -> Result { todo!() }`

where 'a is not a parameter. It's some concrete lifetime that the compiler doesn't know and is forced to assume is incompatible with the lifetime of p.

As @khimru suggests, a fix is to make the lifetimes involved in the arguments to wrapper explicitly the same: this compiles

fn wrapper<'a, F: Fn(&'a Path) -> Result>(f: F, p: &'a Path) -> Result { f(p) }
pub fn wrapped(p: &Path) -> Result { wrapper(operation_1, p) }

as does this

fn wrapper<'a>(f: fn(&'a Path) -> Result, p: &'a Path) -> Result { f(p) }
pub fn wrapped(p: &Path) -> Result { wrapper(operation_1, p) }

My remaining question: I don't understand why operation_1 is monomorphized the way it is, with respect to lifetimes. I sure thought I was writing a function that accepted a reference with any lifetime (longer than the lifetime of the call itself, anyway). Why is operation_1::<&Path> defined not to satisfy for<'a> fn(&'a Path)?

5

The point is that Fn* traits are special, because references in their signature stand in for HRTBs. Therefore this:

Fn(&Path) -> Result

is really just this:

for<'p> Fn(&'p Path) -> Result

so the fully-desugared signature of the function

fn wrapper<F: Fn(&Path) -> Result>(f: F, p: &Path) -> Result

is in fact

fn wrapper<'a, F: for<'p> Fn(&'p Path) -> Result>(f: F, p: &'a Path) -> Result

Meanwhile, when you declare an fn item like

fn operation_1<P: AsRef<Path>>(p: P) -> Result

then this is only generic (over whatever lifetime P may have); it is not a HRTB.
The type variable P thus will always be bound to a specific type (including a specific lifetime if it's a &Path) when you instantiate it.

Hence, in the following call:

wrapper(operation_1, p)

the function operation_1 will be monomorphized and instantiated with one specific type and lifetime, say &'q Path for some concrete, caller-chosen 'q, so its full type becomes:

wrapper(operation_1::<&'q Path>, p)

however, the signature of wrapper() still expects its functional argument to be higher-ranked over the lifetime! This is where the mismatch comes from.

4 Likes
6

I see how that works, but it seems inconsistent with the behavior of concrete functions. Here's another demo module:

use std::convert::AsRef;
use std::path::Path;

pub type Result = std::io::Result<()>;

pub fn concrete(_p: &Path) -> Result { todo!() }
pub fn generic<P: AsRef<Path>>(_p: P) -> Result { todo!() }

pub fn direct_concrete(p: &Path) -> Result { concrete(p) }
pub fn direct_generic(p: &Path) -> Result { generic(p) }

fn wrapper<F: for<'a> Fn(&'a Path) -> Result>(f: F, p: &Path) -> Result { f(p) }

pub fn wrapped_concrete(p: &Path) -> Result { wrapper(concrete, p) }
pub fn wrapped_generic(p: &Path) -> Result { wrapper(generic, p) }

All of the function calls in this demo are fine, except the one in wrapped_generic, which gets the same "implementation of FnOnce is not general enough" error as we've been discussing.

It seems inconsistent to me that concrete satisfies for<'a> Fn(&'a Path) -> Result but generic::<&Path> doesn't. Shouldn't they have exactly the same type signature, including whether or not the lifetime of the reference argument has been pinned down yet?

7

concrete notionally implements

impl<'a> Fn(&'a Path) for fn#concrete

Whereas generic notionally implements

impl<P: AsRef<P>> Fn(P) for fn#generic<P>

So there is no single generic::<T> type (where T is concrete -- Rust does not have types which are higher-ranked over type parameters) such that fn#generic<T>: Fn(&Path). Instead fn#generic<&'p Path> implements Fn(&'p Path) for one specific lifetime 'p, and so on.

Here's an analogous example that doesn't use function items or the Fn traits.

trait Trait<T> {}

struct HasTypeParam<P>(P);
struct HasLifetimeParam<'a>(&'a Path);
struct HasNoneOfThat(PathBuf);

impl<'a> Trait<&'a Path> for HasTypeParam<&'a Path> {}
impl<'a> Trait<&'a Path> for HasLifetimeParam<'a> {}
impl<'a> Trait<&'a Path> for HasNoneOfThat {}

fn foo<T: for<'any> Trait<&'any Path>>(_: Option<T>) {}

fn main() {
    // Implementation of `Trait` is not general enough
    // foo(None::<HasTypeParam<_>>);
    
    // Implementation of `Trait` is not general enough
    // foo(None::<HasLifetimeParam<'_>>);

    foo(None::<HasNoneOfThat>);
}

More technically, generic parameters of functions can be late bound or early bound. Only late bound parameters can satisfy a higher-ranked trait bound, and only lifetime parameters can be late bound.[1] Being early bound corresponds to be a generic parameter on the function item itself, like the "not general enough" examples above. If you can turbofish a parameter, it's early bound.

Moreover, lifetime parameters of functions which are also involved in explicit bounds on the function are early bound, so this will also fail.

//           vvvvvv Force 'p to be early bound
pub fn early<'p: 'p>(_p: &'p Path) -> Result { todo!() }
pub fn wrapped_early(p: &Path) -> Result { wrapper(early, p) }

For both late-bound cases, you can "move the binder" by wrapping the function in a closure which becomes higher-ranked.

pub fn wrapped_early(p: &Path) -> Result { wrapper(|p| early(p), p) }
pub fn wrapped_generic(p: &Path) -> Result { wrapper(|p| generic(p), p) }

It's a breaking change to go between being late or early bound. Going from early to late breaks turbofishing, and going from late to early breaks things like your wrapped_concrete example.

  1. So far. ↩︎

1 Like
8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.