Well, function items are pretty special. Let's start with something that has more visible language surface area, dyn Trait. Roughly what it comes down to is that in order to coerce to a dyn Trait, you need to have an impl Trait somewhere (either explicit or notionally, supplied by the compiler).
If you want a higher-ranked dyn Trait, the implementation itself is going to have to be generic over some lifetimes (which could be part of types) in relation to the type implementing the trait. So here:
struct StandAlone;
struct Contains<T>(*const T);
trait Trait<T> {}
impl<T> Trait<T> for StandAlone {}
impl<T> Trait<T> for Contains<T> {};
The implementation for Contains<&'static u8>, say, is an implementation of Trait<&'static u8> -- every concrete type has a distinct implementation, because the T must unify between the trait and the implementing type. Whereas for Standalone, there is an implementation for T -- if we had such a concept in rust, for<T>. At any rate, the same implementation applies to every &u8 -- that is, there is a single implementation for all &u8 -- for<'a> &'a u8. Thus, this compiles:
let _: &dyn for<'x> Trait<&'x u8> = &StandAlone;
But this does not:
let _: &dyn for<'x> Trait<&'x u8> = &Contains::<_ /* ?? */>(0 as _);
Now let's consider function items, but instead of coercion to function pointers, let's stick with coercion to Fn-like traits. There needs to be a notional implementation of the trait. Let's consider a couple of implementations:
fn foo<T>(_: T) {}
fn bar<T: ?Sized>(_: &T) {}
The implementations are roughly:
impl<T> Fn(T) for foo::<T> { /* ... */ }
impl<'x, T: ?Sized> Fn(&'x T) for bar::<T> { /* ... */ }
// aka impl<T: ?Sized> Fn(&'_ T) for bar::<T> { /* ... */ }
// aka impl<T: ?Sized> Fn(&T) for bar::<T> { /* ... */ }
In both cases, the T must unify between the trait and the implementing type. But in the implementation for bar, the lifetime is independent of the implementing type. Therefore this works:
let _: &dyn Fn(&u8) = &bar<u8>;
But this doesn't:
let _: &dyn Fn(&u8) = &foo::<_ /* ?? */>;
Can we influence these notional trait implementations? To some extent, yes. If you put a bound (even if trivial) on a lifetime, it will become a parmeter of the function item:
fn bar<'a: 'a, T: ?Sized>(_: &'a T) {}
// ...
// This now fails like `foo` does
let _: &dyn Fn(&u8) = &bar::<u8>;
What's going on? Well, if the bound wasn't so trivial, it might not be representable by the relatively coarse for<'any> higher-ranked types and bounds we have today. [1] It's part of a system to preserve soundness.
In compiler-implementation technical terms, this is a distinction between a function's parameters being late-bound or early-bound. Sometimes this arguably-an-implementation-detail surfaces in relatively obscure errors. Like the link goes into, a parameter is late-bound if it's a parameter of the trait in an implementation, but not parameter of the type. If it's a parameter of the type, it's early-bound. [2]
My mnemonic is "late-bound is trait-bound is higher-rank sound (is not turbo-fishable)."
So we have one potential answer to your question: A lifetime parameter must be late-bound in order to coerce into a higher-ranked lifetime.
Alright, let's consider this OP again:
fn foo<T>(_: T) {}
T here is certainly a parameter of the function item -- we can turbofish foo::<u8>, say. Can we go the other direction and make T late-bound, in some reverse of how we made 'a early-bound? Not today; type parameters are always early-bound. The impl Trait initiative document I linked above, though, suggest an alternative -- impl Trait in argument position could be late-bound. That is,
fn foo(_: impl Sized) {}
could have a notional implementation
impl<T /* : Sized */> Fn(T) for foo /* no parameters */ { /* ... */ }
And if that happens, presumably you could coerce it to a higher-ranked dyn type, ala Standalone above. One step beyond that, perhaps a witness of such a dyn Fn implementation for a function item would permit coercion to the analogous function pointer.
I haven't talked about higher-ranked function pointer types yet. I also haven't played with it sufficiently to be completely confident in this section, and I'm running out of steam to run down references, so this is going to be pretty terse.
Function pointers don't have generic parameters, and (like dyn Trait) can only be higher-ranked over lifetimes, not types. So you might think that you would need to start with a
for<'a_1, 'a_2, 'a_3, ..., 'a_n> $X
in order to coerce to a
for<'b_1, 'b_2, ..., 'b_m> $Y // m <= n
With some suitable replacement between $X and $Y as well. And I think this used to be the case. However, the compiler's concept of how subtyping works has evolved, and now it can prove not only that
// This first one is aka fn(&u8, &u8) */
for<'a, 'b> fn(&'a u8, &'b u8) is a subtype of
for<'c > fn(&'c u8, &'c u8)
// By letting 'a => 'c and 'b => 'c
But also that
for<'c > fn(&'c u8, &'c u8) is a subtype of
for<'a, 'b> fn(&'a u8, &'b u8)
// As you can use variance to unify to the lesser of `'a` and `'b`
And thus, they are the "same" type, and can coerce into each other.
There's some PRs or issues or blogs where Niko talks about this, but unfortunately I couldn't find them offhand.
What are the exact rules? I'm not sure. Does this equality hold everywhere? PR 72493 got us closer, but still no. Are the types actually normalized? Also no.
I think this issue implies it's not even in the dev guide proper.
I agree it can all be quite confusing, since you can end up with one type which is a subtype of another in some sense -- including some iterations of rustc's implementation as I understand it -- yet you cannot coerce between them (for the sake of language semantics, or coherence, et cetera).
I believe the problem in that playground is just one of higher-ranked inference, ala RFC 3216. Your implementations are for a concrete lifetime, so it's okay that the return type must be a concrete type -- it can still capture the lifetime; additionally, the implementing type doesn't have the return type (Fut) as an input (parameter), but instead as an output (an associated type of the Fn-like traits).
That said, async is not my forte, so I might be missing some deeper difference between the closure and the function. I didn't manage to find a way to coerce the closure. I think it's a known problem, but again I'm not an async expert. Maybe those macros can work around it; I didn't try.
However, if you needed to bind the return type under a higher-ranked bound like so:
where
// This stable syntax is already problematic as you can't elide `-> Fut`
F: /* for<'_> */ Fn(&[u8]) -> Fut,
// But even on unstable, you have a problem if you need this
Fut: SomeAbility,
Then that would be a problem if you want different input lifetimes to result in different output types, as Fut is a single type due to being a type parameter. So that situation is related.
I have played with this situation for Fn(&'a) -> R bounds quite a bit. I'll just refer to this earlier post (the rest of that thread covers similar ground as the recent parts of this one); it links to this thread which covers a solution (up to a point) for that case on stable. [3]
Or maybe it would be representable, but only by dint of implied bounds elsewhere. If you're tracking the discussion around the bad ergomics between HRTB and GATs, it's pretty related; there are a lot of rough edges in complicated combinations. Someday maybe we'll have
for<'a where 'b: 'a>style higher-ranked types. ↩︎Lifetimes are early-bound if they have bounds, or if they only appear in the return type. If they're unbound and in an input parameter, they're late-bound. "If they're explicit they're early-bound" would be more intuitive, but inadequate for unifying two input lifetimes, say, not to mention backwards incompatible. ↩︎
I have most of a writeup about a refinement to that particular use case (though not handy on this computer) -- it is one of a few extremely illustrative use-cases that have motivated a "lifetime reference" I've wish I could sink time into for months now. As this very reply demonstrates though, it's a very complicated and "why is the sky blue, but why, but why..." sort of topic, and knowledge must be both accumulated and adjusted as the compiler changes, due in no small part to the lack of proper documentation. ↩︎