Having trouble finding the right lifetimes for references to callback parameters

R3DP1XL · August 26, 2021, 1:20pm

Hi,

I've been trying to learn Rust by writing some ECS-like code, and having a lot of difficulty with lifetimes.
I've trimmed down my code so that it still produces the same type of errors I'm seeing, and put it in a playground.

What I'm trying to do is store a list of callbacks that will be invoked later on, and passed in some arguments that are retrieved from a context. It's the references to this context that I can't seem to get right.

The build errors:

error[E0491]: in type `&'a Ref<'entity, <T as SystemArgument<'a, 'entity>>::Component>`, reference has a longer lifetime than the data it references
  --> src/lib.rs:70:82
   |
70 |         let component_as_arg = <T as SystemArgument<'a, 'entity>>::into_arg_type(&component_ref);
   |                                                                                  ^^^^^^^^^^^^^^
   |
note: the pointer is valid for the lifetime `'a` as defined on the impl at 62:6
  --> src/lib.rs:62:6
   |
62 | impl<'a, 'entity, T> System<'entity> for Callback<T>
   |      ^^
note: but the referenced data is only valid for the lifetime `'entity` as defined on the impl at 62:10
  --> src/lib.rs:62:10
   |
62 | impl<'a, 'entity, T> System<'entity> for Callback<T>
   |          ^^^^^^^

error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
  --> src/lib.rs:70:82
   |
70 |         let component_as_arg = <T as SystemArgument<'a, 'entity>>::into_arg_type(&component_ref);
   |                                                                                  ^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime `'entity` as defined on the impl at 62:10...
  --> src/lib.rs:62:10
   |
62 | impl<'a, 'entity, T> System<'entity> for Callback<T>
   |          ^^^^^^^
note: ...so that the type `Ref<'entity, <T as SystemArgument<'a, 'entity>>::Component>` is not borrowed for too long
  --> src/lib.rs:70:82
   |
70 |         let component_as_arg = <T as SystemArgument<'a, 'entity>>::into_arg_type(&component_ref);
   |                                                                                  ^^^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 62:6...
  --> src/lib.rs:62:6
   |
62 | impl<'a, 'entity, T> System<'entity> for Callback<T>
   |      ^^
note: ...so that reference does not outlive borrowed content
  --> src/lib.rs:70:82
   |
70 |         let component_as_arg = <T as SystemArgument<'a, 'entity>>::into_arg_type(&component_ref);
   |                                                                                  ^^^^^^^^^^^^^^

Code: Rust playground

use std::cell::Ref;
use std::cell::RefCell;

#[derive(Default)]
struct Health(f32);

struct Entity {
    health: RefCell<Health>,
}

impl Default for Entity {
    fn default() -> Self {
        Self {
            health: RefCell::new(Health::default()),
        }
    }
}

trait RetrieveComponent<'entity, T> {
    fn retrieve(&'entity self) -> Ref<'entity, T>;
}

impl<'entity> RetrieveComponent<'entity, Health> for Entity {
    fn retrieve(&'entity self) -> Ref<'entity, Health> {
        self.health.borrow()
    }
}

trait SystemArgument<'a, 'entity> {
    type Component;

    fn retrieve_component(storage: &'entity Entity) -> Ref<'entity, Self::Component>
    where
        Entity: RetrieveComponent<'entity, Self::Component>;

    fn into_arg_type(borrowed_ref: &'a Ref<'entity, Self::Component>) -> Self;
}

impl<'a, 'entity, T> SystemArgument<'a, 'entity> for &'a T {
    type Component = T;

    fn retrieve_component(entity: &'entity Entity) -> Ref<'entity, Self::Component>
    where
        Entity: RetrieveComponent<'entity, Self::Component>,
    {
        entity.retrieve()
    }

    fn into_arg_type(borrowed_ref: &'a Ref<'entity, Self::Component>) -> Self {
        &*borrowed_ref
    }
}

trait System<'entity> {
    fn run(&self, storage: &'entity Entity);
}

struct Callback<T> {
    callback: Box<dyn Fn(T)>,
}

impl<'a, 'entity, T> System<'entity> for Callback<T>
where
    T: SystemArgument<'a, 'entity>,
    Entity: RetrieveComponent<'entity, T::Component>,
{
    fn run(&self, entity: &'entity Entity) {
        let component_ref: Ref<'entity, T::Component> =
            <T as SystemArgument<'a, 'entity>>::retrieve_component(entity);
        let component_as_arg = <T as SystemArgument<'a, 'entity>>::into_arg_type(&component_ref);
        (self.callback)(component_as_arg);
    }
}

fn print_health(health: &Health) {
    println!("print_health, health: {}", health.0);
}

fn main() {
    let entity = Entity::default();

    let mut systems: Vec<Box<dyn System<'_>>> = Vec::new();

    systems.push(Box::new(Callback {
        callback: Box::new(print_health),
    }));

    for sys in &systems {
        sys.run(&entity);
    }
}

Note that the stripped down code is a bit contrived
Normally there's also an impl of SystemArgument for &'a mut T, and a bunch of other stuff.

Any hints would be greatly appreciated, thanks in advance!
Marc

alice · August 26, 2021, 1:26pm

I don't think any of those traits need any lifetimes. Try this:

trait RetrieveComponent<T> {
    fn retrieve(&self) -> Ref<'_, T>;
}

trait SystemArgument {
    type Component;

    fn retrieve_component(storage: &Entity) -> Ref<'_, Self::Component>
    where
        Entity: RetrieveComponent<Self::Component>;

    fn into_arg_type(borrowed_ref: &Ref<'_, Self::Component>) -> Self;
}

R3DP1XL · August 26, 2021, 2:56pm

Thanks for the reply!

If I remove all the named lifetimes (which was how I started writing the code initially): Rust playground, I then get a lifetime error relating to the reference returned by SystemArgument::into_arg_type:

error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
  --> src/lib.rs:50:9
   |
50 |         &*borrowed_ref
   |         ^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime defined on the method body at 49:37...
  --> src/lib.rs:49:37
   |
49 |     fn into_arg_type(borrowed_ref: &Ref<'_, Self::Component>) -> Self {
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that the type `Ref<'_, T>` is not borrowed for too long
  --> src/lib.rs:50:9
   |
50 |         &*borrowed_ref
   |         ^^^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'_` as defined on the impl at 39:28...
  --> src/lib.rs:39:28
   |
39 | impl<T> SystemArgument for &T {
   |                            ^
note: ...so that reference does not outlive borrowed content
  --> src/lib.rs:50:9
   |
50 |         &*borrowed_ref
   |         ^^^^^^^^^^^^^^

quinedot · August 26, 2021, 9:17pm

(I only looked at your latest playground, not the original.)

In your implementation, the borrow of your argument is extended via the returned &T -- but type of the return value is Self (via the trait declaration). So in a sense, Rust is telling you "look -- I need some way to attach this borrowed lifetime to the return value that is compatible with the trait definition, so I can track it and make sure there aren't any borrowing violations".

You could add a lifetime to the trait definition, use it to restrict Self, and then implement SystemArgument<'t> for &'t T... but then it's going to infect everything else and probably cause more problems.

So perhaps it's better to change the method definition instead, to "surface" the borrow:

-    fn into_arg_type(borrowed_ref: &Ref<'_, Self::Component>) -> Self;
+    fn into_arg_type<'r>(borrowed_ref: &'r Ref<'_, Self::Component>) -> &'r Self;

And then change what you implement it for:

-impl<T> SystemArgument for &T {
+impl<T> SystemArgument for T {
     type Component = T;
+    //   ^^^^^^^^^^^^^ n.b. not sure this still makes sense...

     fn retrieve_component(entity: &Entity) -> Ref<'_, Self::Component> {...}

-    fn into_arg_type(borrowed_ref: &Ref<'_, Self::Component>) -> Self {
+    fn into_arg_type<'r>(borrowed_ref: &'r Ref<'_, Self::Component>) -> &'r Self {
         &*borrowed_ref
     }
 }

 struct Callback<T> {
-    callback: Box<dyn Fn(T)>,
+    callback: Box<dyn Fn(&T)>,
 }

That works for the example, but I don't know if it will work for your actual use case or if the changes won't make sense for other reasons.

R3DP1XL · August 27, 2021, 9:47pm

Thanks for looking into this @quinedot!

In my initial playground there is indeed a lifetime associated with type T in the SystemArgument implementation.

Unfortunately, as you suspected, I can't make the changes you propose in my actual code.

The main reason is that I want to implement SystemArgument for both generic references (&'a T) and generic mutable references (&'a mut T). In the context of my application, it makes sense to restrict callbacks to functions that only take references (mutable or immutable) as arguments. Obviously a lot of code is missing in the sample, to handle more arguments for example.

I've added a bit more code to the initial sample, in a new Rust playground, to make it more representative of this. The build errors are the same as initially reported.

use std::cell::{Ref, RefCell, RefMut};

#[derive(Default)]
struct Health(f32);

struct Entity {
    health: RefCell<Health>,
}

impl Default for Entity {
    fn default() -> Self {
        Self {
            health: RefCell::new(Health::default()),
        }
    }
}

trait RetrieveComponent<'entity, T> {
    fn retrieve(&'entity self) -> Ref<'entity, T>;
    fn retrieve_mut(&'entity self) -> RefMut<'entity, T>;
}

impl<'entity> RetrieveComponent<'entity, Health> for Entity {
    fn retrieve(&'entity self) -> Ref<'entity, Health> {
        self.health.borrow()
    }

    fn retrieve_mut(&'entity self) -> RefMut<'entity, Health> {
        self.health.borrow_mut()
    }
}

enum BorrowedReference<'b, T> {
    Shared(Ref<'b, T>),
    Exclusive(RefMut<'b, T>),
}

trait SystemArgument<'a, 'entity> {
    type Component;

    fn retrieve_component(entity: &'entity Entity) -> BorrowedReference<'entity, Self::Component>
    where
        Entity: RetrieveComponent<'entity, Self::Component>;

    fn into_arg_type(borrowed_ref: &'a mut BorrowedReference<'entity, Self::Component>) -> Self;
}

impl<'a, 'entity, T> SystemArgument<'a, 'entity> for &'a T {
    type Component = T;

    fn retrieve_component(entity: &'entity Entity) -> BorrowedReference<'entity, Self::Component>
    where
        Entity: RetrieveComponent<'entity, Self::Component>,
    {
        BorrowedReference::Shared(entity.retrieve())
    }

    fn into_arg_type(borrowed_ref: &'a mut BorrowedReference<'entity, Self::Component>) -> Self {
        match borrowed_ref {
            BorrowedReference::Shared(shared_ref) => &*shared_ref,
            _ => panic!("cannot be exclusive reference"),
        }
    }
}

impl<'a, 'entity, T> SystemArgument<'a, 'entity> for &'a mut T {
    type Component = T;

    fn retrieve_component(entity: &'entity Entity) -> BorrowedReference<'entity, Self::Component>
    where
        Entity: RetrieveComponent<'entity, Self::Component>,
    {
        BorrowedReference::Exclusive(entity.retrieve_mut())
    }

    fn into_arg_type(borrowed_ref: &'a mut BorrowedReference<'entity, Self::Component>) -> Self {
        match borrowed_ref {
            BorrowedReference::Exclusive(exclusive_ref) => &mut *exclusive_ref,
            _ => panic!("cannot be shared reference"),
        }
    }
}

trait System<'entity> {
    fn run(&self, entity: &'entity Entity);
}

struct Callback<T> {
    callback: Box<dyn Fn(T)>,
}

impl<'a, 'entity, T> System<'entity> for Callback<T>
where
    T: SystemArgument<'a, 'entity>,
    Entity: RetrieveComponent<'entity, T::Component>,
{
    fn run(&self, entity: &'entity Entity) {
        let component_ref: BorrowedReference<'entity, T::Component> =
            <T as SystemArgument<'a, 'entity>>::retrieve_component(entity);
        let component_as_arg =
            <T as SystemArgument<'a, 'entity>>::into_arg_type(&mut component_ref);
        (self.callback)(component_as_arg);
    }
}

fn increase_health(health: &mut Health) {
    health.0 += 1.0;
}

fn print_health(health: &Health) {
    println!("print_health, health: {}", health.0);
}

fn main() {
    let entity = Entity::default();

    let mut systems: Vec<Box<dyn System<'_>>> = Vec::new();

    systems.push(Box::new(Callback {
        callback: Box::new(increase_health),
    }));

    systems.push(Box::new(Callback {
        callback: Box::new(print_health),
    }));

    for sys in &systems {
        sys.run(&entity);
    }
}

quinedot · August 28, 2021, 6:00am

I'm not sure there's any way to "hide" your RefCell by e.g. trying to reborrow from &mut RefMut into a &mut T. Tinkering with it reminds me of this. Additionally the pattern that looks like

f(&'a mut Generic<'lifetime_you_actually_want>) -> Self<'a>

is going to be quite restricting (if workable at all) -- the inner lifetime becomes invariant and because the outer lifetime must be named, it's going to be forced to be longer than you want pretty much always (e.g. when 'a is a generic type parameter).

Trying to hide the lifetime is probably going to bite you in your callback signature, too. You really want Fn implementers that can work with any lifetime (dyn for<'a> Fn(...)).

Here's an example where I've pushed the abstraction into the implementation of System instead (and stopped hiding lifetimes again). Inside of run is where you want all your borrowing to be contained, so that the compiler can infer the smallest possible lifetimes.

Inference in main did suffer a bit. I didn't spend any time trying to make that part cleaner.

Are all your components Copy, like f32? If so, you could use Cell instead. Cell doesn't have the run-time tracking (and Drop implementation) that RefCell does.

If there's a larger motivation for SystemArgument being designed how you did, I don't understand it and this probably still won't work for you. I gave it a shot with returning &Cell<Self::Component> instead of Self (as that's a simpler type that works for both the mutable and immutable cases), but ran into a lot of rustc failing to normalize projections and gave up for now.

R3DP1XL · August 28, 2021, 9:59pm

Thanks again for the additional tests. I'm thinking my current design is not solvable as is, in terms of lifetime specifications.

I wanted to try this design of SystemArgument so that I wouldn't have to explicitly map out each mutable/immutable combination for each of the callback/functor arguments.

Also, I chose RefCell vs Cell to avoid the copies, since I don't know what the different actual Component types will be. I was expecting a performance gain by having the values reside in place. In the full code, the RefCell is actually on a vector of values, not individual values... but I wanted to simplify the sample code so I omitted this aspect.

In any case, I think the way you modified the sample could work in my scenario, and is actually closer to what I started out with. I was just hoping to simplify the code a bit, but I think I came up with a construction that can't actually work out...

system · November 26, 2021, 9:59pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Lifetime issue with generic callback parameter help	9	937	August 4, 2021
Lifetime problem - Cannot infer an appropriate lifetime help	2	365	November 9, 2020
Lifetime of callback arguments help	11	1973	January 12, 2023
Help with lifetime compile problem help	5	917	June 29, 2021
Help to understand lifetimes help	10	756	April 22, 2022

Having trouble finding the right lifetimes for references to callback parameters

Related topics