Need help understanding this lifetime/borrowck issue

help
1

I'm trying to make this code work:

use std::marker::PhantomData;

trait LuaRead<L> {
}

struct InsideCallback;

struct LuaMutex<'a, T> {
    lua: &'a InsideCallback,
    marker: PhantomData<*const T>,
}

impl<'a, T> LuaRead<&'a InsideCallback> for LuaMutex<'a, T> {
}

fn foo<T>() where T: for<'i> LuaRead<&'i InsideCallback>
{
}

fn main() {
    foo::<LuaMutex<i32>>()
}

(playpen link)

The error is when we call foo:

the trait bound `for<'i> LuaMutex<'_, i32>: LuaRead<&'i InsideCallback>` is not satisfied

(Note: Before you ask, yes I do need this exact design, please don't question it. If you want to take a look at the real code, it is here, here and here. I would be happy to obtain any feedback on a different design in the real code, but I avoided you this pain by simplifying the problem.)

Basically I'd like the type parameter of T to implement LuaRead with a reference to a InsideCallback created within the body of foo (not shown here).
Satisfying this requirement is exactly the purpose of the line impl<'a, T> LuaRead<&'a InsideCallback> for LuaMutex<'a, T>.
If you compare this line to the error message, it's exactly that. Yet the compiler says that the trait is not satisfied.

I opened an issue in which someone suggested to use two different lifetimes for the &'a InsideCallback and the LuaMutex<'a>.

...Which gives me this code:

use std::marker::PhantomData;

trait LuaRead<L> {
}

struct InsideCallback;

struct LuaMutex<'a, T> {
    lua: &'a InsideCallback,
    marker: PhantomData<*const T>,
}

impl<'a: 'm, 'm, T> LuaRead<&'a InsideCallback> for LuaMutex<'m, T>
{
}

fn foo<T>() where T: for<'i> LuaRead<&'i InsideCallback>
{
}

fn main() {
    foo::<LuaMutex<i32>>()
}

(playpen link)

For which the error is even harder to understand:

the requirement `for<'i> 'i : ` is not satisfied (`expected bound lifetime parameter 'i, found concrete lifetime`)

I don't even understand which "concrete lifetime" the compiler is referring to. The inferred lifetime at foo::<LuaMutex<i32>>() is equivalent to 'm in the trait impl, and the for<'i> is equivalent to 'a in the trait impl.
If I change the code to do foo::<LuaMutex<'static, i32>>(), the error message remains the same! If the "concrete lifetime" was this inferred lifetime here, then I feel that the error message would be different.

Any explanation for this error or any work-around would be appreciated.

2

Either of these 2 foo signatures compile:

fn foo<'a, T:LuaRead<&'a InsideCallback>>() 
fn foo<'a, T>() where T: LuaRead<&'a InsideCallback>

Are you trying to express something different?

3

Yes, I do need the HRTB here because the InsideCallback is created within foo.
Your suggestion works for the simplification here, but not in the real code.

(Just like for example this doesn't work: Rust Playground
But this works thanks to the HRTB: Rust Playground)

4

The key here is to think of generic parameters as part of the type: Type<'a> and Type<'b> are different types if 'a and 'b are different lifetimes. Same goes for references and traits: Trait<TypeA> and Trait<TypeB> are different traits.

First,

impl<'a, T> LuaRead<&'a InsideCallback> for LuaMutex<'a, T> {}

means that each concrete LuaMutex type implements LuaRead for one concrete reference to InsideCallback. For example, if we have a concrete type LuaMutex<span, T> (where span refers to an actual lifetime inferred during compilation), and apply this impl to it, we'd get an implementation of LuaRead<&span InsideCallback>. An implementation for a fully qualified, concrete trait.

On the other hand,

fn foo<T>() where T: for<'i> LuaRead<&'i InsideCallback>

means that a single concrete type T must have a generic implementation of LuaRead, that can take an arbitrary lifetime 'i.

But if we put this LuaMutex<span, u32> into foo, we get a conflict: it only implements LuaRead<&span InsideCallback>, but we require not a concrete, but a generic implementation - there is still 'i that compiler needs to put somewhere. Hence the error message: the impl we have is for a concrete trait, but we need a generic one.

To have such an implementation, lifetime parameter in the LuaRead must be free a free variable, unrestricted by the target type or outlive relations:

impl<'a, 'b, T> LuaRead<&'a InsideCallback> for LuaMutex<'b, T> {}

This wouldn't be a correct solution, of course.

5

What you're saying is true if I called foo::<'a, LuaMutex<i32>>() with a precise lifetime.
Since I'm not passing any lifetime, the compiler is supposed to automatically infer it.

If you take the first code, we have:

  • impl<'a, T> LuaRead<&'a InsideCallback> for LuaMutex<'a, T>
  • where T: for<'i> LuaRead<&'i InsideCallback>
  • foo::<LuaMutex<'_, i32>>()

For me the compiler should be able to understand that the unspecified lifetime in the call to foo is 'i, and then the actual span of 'i is determined based on the way we use the T inside foo.

6

'i is not a lifetime, it does not refer to the lifetime of the function body. It is a variable, which can (and will) take different values even within the same function, but T can not change. For example, it means that this compiles (Rust Playground):

fn foo<'a, T>(i: &'a InsideCallback) -> (T, T, T)
    where T: for<'i> LuaRead<&'i InsideCallback>
{
    let a = T::read(&i);
    
    let j = InsideCallback;
    let b = T::read(&j);
    
    static k: InsideCallback = InsideCallback;
    (a, b, T::read(&k))
}
7

You're essentially saying that this code shouldn't compile, yet it does: Rust Playground

It's a similar situation. &str also has a concrete and inferred lifetime, yet it matches for<'i>.

8

No, I'm not saying that. I'm saying that HRTB expresses requirement more powerful than you seem to think. where T: for<'a> Trait<'a> does not mean that T should implement Trait for some inferred value of 'a or another. It means it should implement it for absolutely all possible lifetimes ever, and all of them at the same time. Closures and function types do that for their lifetime parameters - the same closure value can be used in different places, with different concrete inferred lifetimes.

In the example you provide (Rust Playground), &str in the closure definition (line 11) does not have a concrete, inferred lifetime. It's lifetime is actually a variable. Which is exactly why the closure satisfies the for <'a> FnOnce(&'a str) bound.

On the other hand, &a on line 6 does have a concrete, inferred lifetime. And because F is required to work for any lifetime possible, the program compiles and works.

9

Let's take another example: Rust Playground
Here F can be variable and can work for any lifetime 'r. Yet it doesn't compile. Same error message as my original example.

If the problem is due to the fact that the lifetime of LuaRead is the same as the &str, then there is this one where the lifetime is decoupled but is more complex: Rust Playground

10

In the first example, the problem is not with F, but with T. And yes, the problem is indeed in the fact that lifetime of LuaRead is the same as the &str.

But decoupling via an outlives relation (the second example) is still not enough: it expands the set of accepted lifetimes, but still there are lifetimes for which the implementation does not exist for a given value of Foo.

11

I'm trying to find a work-around, but this doesn't work either for example: Rust Playground

I'm feeling a bit desperate. I'm not asking the compiler something crazy here.

12

I think what you "really" want is a higher-kinded type parameter. In other words, you don't want foo to be parameterized by LuaMutex<'a, i32> for any single 'a, but by the general concept of 'a -> LuaMutex<'a, i32>, which foo can then instantiate for multiple different 'as. Higher-kinded types don't currently exist in Rust, but you can simulate them with associated types:

trait LuaReadOnReferenceMaker<'i, C: 'i> {
    type Reader: LuaRead<&'i C>;
}
struct LuaMutexMaker<T>(PhantomData<T>);
impl<'i, T> LuaReadOnReferenceMaker<'i, InsideCallback> for LuaMutexMaker<T> {
    type Reader = LuaMutex<'i, T>;
}

fn foo<M>() where M: for<'i> LuaReadOnReferenceMaker<'i, InsideCallback>
{
    // use M::Reader
}

fn main() {
    foo::<LuaMutexMaker<i32>>()
}

That said, this is a somewhat complicated design, and there may be something simpler that would work for your use case; I haven't looked through your original code in detail. (edit: for example, you may be able to use just one trait with an associated type and methods that act on it, rather than what I did with a trait whose associated type is bounded on another trait)

13

My real code uses closures, and is very similar to Rust Playground in fact

The foo function is in my library, but the user chooses how to call foo. Therefore while your solution would work, I don't think it's very ergonomic.

However if there is no solution I guess I could use an Rc<InsideCallback> in order to remove the lifetime, with a flag inside the InsideCallback in order to panic if it is used outside of the callback (eg. if the user stores it in a TLS).

14

Looking at your code a bit more... in your PushOne impl, I guess an ideal Rust would let you write something like for<'i> P<'i>: LuaRead<&'i InsideCallback<'lua>>, where P would be a higher-kinded type parameter.

As a workaround, maybe you could have the callbacks take all parameters by reference? So you'd have FnMut(&P) -> R, for<'i> &'i P: LuaRead<&'i etc>... That'd be a burden on user code but not too big. This could benefit more than mutexes: I think you could also drop the Sized bound and impl LuaRead for str (instead of String), avoiding the copy you currently do on every string read.