Help finding a valid signature for this function

Hi,

I'm writing a program that uses Cap'n Proto and I've created a puzzle I haven't been able to solve. I don't think this is specific to Cap'n Proto, but it led me in this direction.

Here's a self-contained snippet that works:

fn declare_message_and_init_struct_directly() {
    let mut message_builder = capnp::message::Builder::new_default();
    let mut struct_builder = message_builder.init_root::<my_generated_struct::Builder>();
    struct_builder.set_data(b"abc123");
}

I found myself doing this multiple times with minor variations and wanted to see about rearranging it into a function that receives a closure for the part that varied. As a first step, I was able to move the init_root call into a separate function and pass the set_data call as a closure:

fn declare_message_directly_and_init_struct_through_closure() {
    init_struct(
        &mut capnp::message::Builder::new_default(),
        |mut struct_builder: my_generated_struct::Builder| {
            struct_builder.set_data(b"abc123");
        },
    );
}

fn init_struct<
    'message,
    A: capnp::message::Allocator,
    T: capnp::traits::FromPointerBuilder<'message>,
    InitFn: Fn(T),
>(
    message_builder: &'message mut capnp::message::Builder<A>,
    init_fn: InitFn,
) {
    init_fn(message_builder.init_root::<T>());
}

That's not any better at the call site, but it was an interesting exercise to find an appropriate trait in the capnp library and figure out how to express the lifetime bounds on it. The builder must be parameterized with a lifetime that does not outlive the capnp::message::Builder it was initially borrowed from, so I provided the function with a bound on that lifetime ('message) by passing it a mutable reference that outlives the capnp::message:Builder.

I then attempted to move ownership of the capnp::message::Builder into the extracted function:

fn declare_message_and_init_struct_through_closure() {
    declare_message_and_init_struct(|mut struct_builder: my_generated_struct::Builder| {
        struct_builder.set_data(b"abc123");
    });
}

fn declare_message_and_init_struct<
    'message,
    T: capnp::traits::FromPointerBuilder<'message>,
    InitFn: Fn(T),
>(
    init_fn: InitFn,
) {
    init_struct(&mut capnp::message::Builder::new_default(), init_fn);
}

// init_struct is reused from above

That's what I want the call site to look like in the end, but this is not valid as written.

error[E0597]: borrowed value does not live long enough
   --> src/lib.rs:179:26
    |
179 |         init_struct(&mut capnp::message::Builder::new_default(), init_fn);
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^          - temporary value only lives until here
    |                          |
    |                          temporary value does not live long enough
    |
note: borrowed value must be valid for the lifetime 'message as defined on the function body at 173:9..
.
   --> src/lib.rs:173:9
    |
173 |         'message,
    |         ^^^^^^^^
    = note: consider using a `let` binding to increase its lifetime

I don't know how to tell the compiler that I want the lifetime parameter 'message in my_generated_struct::Builder<'message'> to be the same as the body of the init_fn. (Adding a let binding as advised only expands the insufficient lifetime to the whole body of declare_message_and_init_struct.) The compiler seems to be interpreting 'message as a free parameter, having no constraint relative to what's going on when I call the passed Fn.

My specific question is how can I write a declare_message_and_init_struct that I can use as a Fn(Fn(my_generated_struct::Builder)) for any particular generated message's Builder? I can write this easily with any concrete foo::Builder<_> type because the compiler is happy to infer the lifetime of the struct builder within a function's body. But how do I get the same arrangement across a function call?

Thanks,
mvanbem

Can you change the signature of init_struct

fn init_struct<
    'message,
    A: capnp::message::Allocator,
    T: capnp::traits::FromPointerBuilder<'message>,
    // this is how you link the lifetime of 'message to the function
    InitFn: 'message + Fn(T),
>(
    message_builder: &'message mut capnp::message::Builder<A>,
    init_fn: InitFn,
) {
    init_fn(message_builder.init_root::<T>());
}

If you don't specify a lifetime, then a 'static lifetime is inferred.

Thanks, @RustyYato. I tried adding that 'message constraint to InitFn, but it didn't change anything. If I had to guess, the closure doesn't own anything, so there are no lifetimes in it to constrain.

Here's a cleaned up minimal repro of my problem on the playground:

Test (1) is what I started with. Test (2) is almost right, but I want to make it generic over all StructBuilders rather than manually specialized per concrete type. Test (3) is as far as I could get while making it generic. Test (4) is the least broken invalid attempt I could come up with. I'd like to fix up test (4) or find another way to do something similar.

Ah, I see.

I don't think you can go on to step 3, with these lifetime requirements.

Is there any reason why you need to have a reference to a Box instead of just storing the Box? That way you ca get rid of all of the lifetimes.

All of the declarations in the fake modules are made up to reproduce the problem. I added the Box<Any> to make it actually do something so it would compile without unused field warnings.

So, are you using an API which takes references?
And you want to initialize values using a closure, but those values have lifetimes attached?


edit: I missed the comment at the top on my last go through. That clears things up!

#4 is where you’d normally use HRTB, T: for<'message> StructBuilder<'message>, but this fails because the compiler thinks FooBuilder and BarBuilder don’t satisfy the bound. I believe this might be related to poor inference in closures that have an argument with a lifetime parameter. There are many HRTB related issues in Rust’s github, although I don’t know offhand which would be the one for this.

1 Like

I don't think it is possible to do #4, I think it is best to stick with #3. @vitalyd's explanation is exactly the issue. I don't think you can use HRTB (Higher Rank Type Bounds) in this case, it is too generic, so I would stick with #3. Another way to think of it is, there is no way to specify that the lifetime you are interested in is within the function, so you must say that the closure you take in must work for all lifetimes (which is what HRTB does), but in this case you would also need to specify the type via HRTB, which is currently not possible.

If it were possible the signature would be

fn declare_message_and_init<F>(f: F)
where F: for<'a, T: fake_capnp::StructBuilder<'a>> FnOnce(T)
      //^^^^ this is HRTB

But again that isn't currently possible

I recommend filing a Rust github issue so this example is tracked in case someone makes progress on expanding the lifetime inference in closures.

1 Like

Thanks to both of you for the explanations. I see how the nested where ... for<> clause would do what I want if it was possible. I'll take a look at the existing HRTB/closure bugs and add my example.