Return borrowable type that may borrow from outside the closure

I have this case that I have boiled down to this small sample below. I have a unit test that I want to parameterize by the the A used when calling b. Two such cases are using a top level instance of A, passed to the closure get_a, or creating a new one (so the top level A is ignored and a new one is just created. The first case fails with a lifetime error. The second, surprisingly just works, I would have expected the same compile error. How do I get both to pass?

See the playground link for the compile error.

use std::borrow::Borrow;

fn main() {
    // Return the provided reference
    borrow_a_call_b(|a| a);
    // Use the value provided instead
    borrow_a_call_b(|_a| A {});
}

fn borrow_a_call_b<F, B>(get_a: F)
where
    F: Fn(&A) -> B,
    B: Borrow<A>,
{
    let default_a = A {};
    get_a(&default_a).borrow().b();
}

struct A {}

impl A {
    fn b(&self) {}
}

(Playground)

The bound

F: Fn(&A) -> B

can be rewritten

F: for<'any> Fn(&'any A) -> B

and means that for any input lifetime 'any, the function must return the same type B. And types that differ by lifetime are distinct types, even if they only differ by lifetime.

Therefore it's impossible for a function that returns something that captures the input lifetime to meet the bound. If you have a return that captures the input lifetime, you can't mention the output as a simple generic type parameter like B -- those must resolve to a single type.

You can sometimes work around this with a custom trait.

However, even with the workaround, the closures need a lot of help to infer the desired signature -- hence the hr_bump helper and extra annotations.

4 Likes

That makes absolutely no sense to me :slight_smile:. I think I can vaguely make out that MaybeBorrowOne lets you constraint the return from the function to have a lifetime that matches the handle argument. I think that's what I believed I needed in my version, but couldn't figure out how to express it. It's a lot of jumping through hoops to get it, I don't quite follow, particularly why the for<'any>.

What does F { f } in fn hr_bump<F: Fn(&A) -> &A>(f: F) -> F { f } do?

The TL;DR is that abstracting over "borrowed or owned" is almost never simplistic. You might be better off just splitting up your use cases accordingly.

But let me try to walk through some of this in slow motion anyway.


This will be repeated below, but hr_bump is just the identity function that returns what you pass in, with some bounds on what you can pass in. The bounds influence the compiler's inference of what the closure does. The bounds are the point, but to answer the question, if we look at it without bounds we have

fn hr_bump<F>(f: F) -> F { f }

just an identity function. The { f } is just the body returning the argument.


Next a recap.

Our goal is to express something like these bounds:

    F: Fn(&A) -> B,
    B: Borrow<A>,

Except we can't use a type parameter like B, because with this signature:

fn example(_: &A) -> &A

When the input type is &'a1 A, the output type is &'a1 A, when input type is &'a2 A the output type is &'a2 A, and so on -- there's not one output type across all the input types, there's one output type per input type.

The bound can work with the example function for a single lifetime:

    F: Fn(&'a A) -> B,
    B: Borrow<A>,

Because here B can take on the "type value" &'a A. But your borrow_a_call_b does need the "for any lifetime" version (for<'any> Fn(&'any A) -> ??? or just Fn(&A) -> ???) because that's the only way for the function F to be callable with a local borrow like &default_a.

It turns out we can't use a F: Fn(...) -> ... bound at all, because it forces you to name the output type in some way. We can't use just a type parameter like B for the reasons above, and if you used something like

F: Fn(&A) -> &A

Then it wouldn't work with the closure that returns just A {}.

We need something else.


What I do in the playground as an alternative is to use custom traits to build up the bound that you want in such a way that you don't have to name the return type in the bounds on borrow_a_call_b. I start with the single lifetime version, and then expand that to the any lifetime version.

Here's the first trait:

trait MaybeBorrowOne<'a>: Fn(&'a A) -> <Self as MaybeBorrowOne<'a>>::Out {
    type Out: Borrow<A>;
}

The supertrait bound means that if F: MaybeBorrowOne<'a>, then

F: Fn(&'a A) -> F::Out

And the associated type bound means that

F::Out: Borrow<A>

Then we just implement it for everything that meets the supertrait and associated type bounds.

impl<'a, F, B> MaybeBorrowOne<'a> for F
where
    F: Fn(&'a A) -> B,
    B: Borrow<A>,
{
    type Out = B;
}

Note how we can write a bound like this:

F: MaybeBorrowOne<'a>

And we didn't have to mention the output type at all.

The next part is to have another trait for "can work with any lifetime" (a higher-ranked bound).

trait MaybeBorrow: for<'any> MaybeBorrowOne<'any> {}
impl<F: for<'any> MaybeBorrowOne<'any>> MaybeBorrow for F {}

This was just sugar really, we didn't have to use it. If we compare MaybeBorrowOne<'a> to your original bounds:

    F: for<'any> Fn(&'any A) -> B, // But we want `B` to be able to vary per
    B: Borrow<A>,                  // input lifetime, not be a single type

We could just write:

    F: for<'any> MaybeBorrowOne<'any>

The sugar just let's us write this instead:

    F: MaybeBorrow,

In either case, we don't have to restrict the output type to be the same across all the input lifetimes -- it might be the same, or it might be different for each input lifetime. Both are supported. The key is that we never equated the output type with a type parameter when we say

where F: for<'any> MaybeBorrowOne<'any>

because we don't have to mention the output type at all.[1]

And... that's it for replacing the bounds with something that can support both types of closures.


Sidebar: We could have left off the supertrait bound if we'd supplied the functionality some other way.

trait MaybeBorrowOne<'a> {
    type Out: Borrow<A>;
    fn do_the_call(&self, _: &'a A) -> Self::Out;
}

But then the bound wouldn't let you call against the bound variables directly; you'd have to do f.do_the_call(&some_a).


OK, that gets us this far. Now we're faced with a new problem: Rust thinks you wanted that closure to work with one particular input lifetime and not any input lifetime. This is a common problem for closures. The version I've left uncommented has an easy work-around: add an annotation that indicates the input type is a reference.

-    borrow_a_call_b(|_a| A {});
+    borrow_a_call_b(|_a: &_| A {});

But even the annotation doesn't work for the borrowing closure. It now thinks you want to take any lifetime but return one specific lifetime because... I have no idea why. (Frankly the compiler is really bad at inferring higher-ranked closures.)

It also can't "see through" the MaybeBorrow implementations deep enough that they influence how the compiler infers the shape of your closure. However, we pass the closure to something with Fn-type bounds, that is enough to influence the compiler.

So that's what hr_bump is about. If you ignore the bounds, it's just the identity function: it returns what you pass in. The bounds are there to influence the compiler's closure inference:

// hr_bump<F>(f: F) -> F { f } ... but with the closure "shape" we want
// expressed as trait bounds on `F`
fn hr_bump<F: Fn(&A) -> &A>(f: F) -> F { f }

And finally we get something that compiles.

The tricks workarounds in this section are well-known; they'll become familiar if you use closures in generic scenarios often enough.

If you use functions instead of closures, you can avoid this section where you have to hold the compiler's hand and walk it through the inference. If you need or want to use closures, you may not be gaining much, due to the compiler's closure inference limitations.


So my reply is an answer to "how do I write the bounds that covers both of these closures", but a better answer might be to take a step back and stop trying to handle both borrowing and owning scenarios in one go. It depends on which hassle you want to deal with.


  1. In the playground, the place where I "say" that is in the blanket implementation for MaybeBorrow. ↩ī¸Ž

3 Likes

Thank you @quinedot, really appreciated.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.