HRTB connecting Fn inputs + outputs

Hi all,

I am trying to create a registry of callback functions, where each has arguments extracted from a common Input type. I have created a trait Args for the possible argument types to encapsulate the extraction logic, the results of which can refer to parts of the input and so has a lifetime parameter. Similarly, the Output type of these callbacks sometimes needs to return data from those arguments.

At the time we're building this registry, we don't yet have the input and so to create a list of the callbacks from Input to Output, I want to write something which wraps a body that takes Args with one that extracts them from the input, something like:

fn wrap<A, B>(body: B) -> impl for<'a> Fn(&'a Input) -> &'a Output 
   where for<'a> A: Args<'a>, B: Fn(A) -> &'a Output () { ... }

But we cannot write the HRTB for both A and B like that. Based on other answers to similar questions, it sounded like a possible path to resolve is by pulling out a trait for the body, something like:

struct Input<'a> {
    // ...
}

struct Output<'a> {
    // ...
}

struct Args<'a>: Sized {
    fn extract(c: Input<'a>) -> Self;
}

trait Body<'a> {
    type Args: Args<'a>;
}

impl<'a, A, B> Body<'a> for B where B: Fn(A) -> Output<'a>, A: Args<'a> {
    type Args = A;
}

fn wrap<A, B>(body: B) -> impl for<'a> Fn(Input<'a>) -> Output<'a>
where
    for<'a> B: Body<'a> + Fn(A) -> Output<'a>,
{
    move |c| body(<B as Body>::Args::extract(c))
}

// (trimmed for brevity)

(Playground)

But this doesn't work:

   Compiling playground v0.0.1 (/playground)
error[E0582]: binding for associated type `Output` references lifetime `'a`, which does not appear in the trait input types
  --> src/lib.rs:27:36
   |
27 |     for<'a> B: Body<'a> + Fn(A) -> Output<'a>,
   |                                    ^^^^^^^^^^

error: aborting due to previous error

error: could not compile `playground`

To learn more, run the command again with --verbose.

I've tried a number of different variations here but either end up with an error like this, or an error that the implementation of body is not general enough (I think because the Body impl has to re-state the bounds and runs into the same problem). I've also tried having the Body trait expose some call method itself, but either I have the same problems as above or I get errors about the Args type parameter being unconstrained despite appearing in the Fn type.

I'm pretty new to rust and feel like there is some fundamental thing I'm missing here about the goal or limitations I'm running up against; can anyone give suggestions about whether something like this is possible or pointers towards other code/discussions about something similar?

So, well, the main problem is that your Body trait does not work out as intended. The problem being: You can’t actually derive the argument type A from a function type B: Fn(A) -> .... It’s a bit complicated to try to fix your code when there’s no version without that Body trait. I’d appreciate a playground with the original example, i.e. something like the code

you originally referred to. I’m a bit confused about whether you’d prefer working with references &'a Input or generic types Input<'a> for example, but I guess that this distinction doesn’t really matter anyways. I’ll spend a bit more time figuring out if I can find a nice solution for you :wink:

Hm, this might work (playground).

2 Likes

Thanks, sorry! Here's the original thing I was basically trying to write. Rust Playground

Sorry for the &Input vs Input<'_> thing -- I was trying to produce a reduced example by removing the additional traits at some point.

Looks like this pretty much matches what I’ve guessed by now. See if you can understand what I’m trying to do in the playground I linked above and feel free to ask questions; it’s a bit of a mess, I’ll admit, but it seems to solve your problem.

Thanks, I will stare at it a bit and see if I can figure it out. Appreciate the super fast and helpful response!

So, IIUC, this solves the issue by making what used to be an assoc type become a generic type parameter, right?

That compiler error seems badly implemented, since it has so many false positives :weary:, so this hacky workaround seems to be warranted, and the "hacky quality", deserved (it does beat macros by a fair margin, though).


Funnily I tried to solve keeping the assoc type correctness, which lead me to taking the full HKT road, which then fails because of the infamous lazy normalization issue in the language. So I ended up going the syntactically speaking, type constructors are duck-typed HKTs road, by using macros, which worked quite well (Playground). Reminds me of how many libs out there use the "decorate the function or the closure with a macro" approach since it avoids the whole generic verbiage and higher-order-related "bugs" (lazy normalization, and now I discover the higher order assoc type false positives).

1 Like

By the way, before you even ask, the B: Fn(A) -> Output<'c> bound is exclusively to aid type inference. Without it, you’d need to explicitly specify the argument type of to_wrap in the test call (something like wrap::<Foo, _>(to_wrap)). With this bound, the compiler can infer that A has to be the argument type of to_wrap which it then finds out to be Foo<'c> for some lifetime 'c (hence also the extra lifetime parameter). Rust doesn’t actually care if it cannot find out anything else about that mysterious, unknown/unconstrained/ambiguous lifetime 'c since lifetimes are eliminated early in the compilation process anyways.

I’d not be surprised if a few lifetimes in my code could be replaced by the concrete lifetime 'static; e.g. the 'c lifetime of wrap or the 'c lifetime in the Args trait. Even if that works, I’m not sure if I’d find that any more elegant anyways.

Thanks! I will stare at that, too.

Just noticed that this kind-of was a question, so I’ll answer.

Yes that’s correct. And if it’s applied to OPs original code, it directly leads to the other error about the Body implementation. Actually, it’s kind-of sad that the other error is “masked” by this one since it’s totally unrelated.

Just wanted to say thanks (to both of you) again. I think I understand what's going on in the first version here. Basically seems like 2 things: one is replacing the associated type with a type parameter so it can be defined in the function bounds, and the other is allowing the creation of "Args with a different lifetime" through this two-lifetime trick. It's almost like a type OtherArgs<'a2>: Args<'a2> on the Args trait but lets us specify the lifetime in the function bounds and refer to the type in the body.

After @Yandros's comments about how people usually avoid this with a macro rather than fancy type juggling, I think that's probably the right thing for my case, too.

1 Like