Borrowed value doesn't live long enough for trait, but does for concrete impl?

recatek · January 12, 2023, 3:32pm

I'm trying to build a simple ECS storage structure that can be queried for iteration based on type combinations, with both &T and &mut T access patterns.

use std::cell::{Ref, RefCell, RefMut};

#[derive(Default)]
pub struct SimpleWorld {
    column_1: RefCell<Vec<u8>>,
    _column_2: RefCell<Vec<u16>>,
    column_3: RefCell<Vec<u32>>,
    _column_4: RefCell<Vec<u64>>,
}

trait World {
    fn column<'w, T: Param<'w, Self>>(&'w self) -> T::Column<'w>
    where
        T::Column<'w>: 'w,
    {
        <T as Param<Self>>::column(self)
    }
}

impl World for SimpleWorld {}

trait Param<'p, W: World + ?Sized>: Sized {
    type Underlying;
    type Column<'c>: 'c;
    type Unpacked<'u>;

    fn column(world: &'p W) -> Self::Column<'p>;
    fn unpack<'a>(column: &'a mut Self::Column<'_>) -> Self::Unpacked<'a>;
}

impl<'p> Param<'p, SimpleWorld> for &'p u8 {
    type Underlying = u8;
    type Column<'c> = Ref<'c, [u8]>;
    type Unpacked<'u> = &'u [u8];

    fn column(world: &'p SimpleWorld) -> Self::Column<'_> {
        Ref::map(world.column_1.borrow(), |c| c.as_slice())
    }

    fn unpack<'a>(column: &'a mut Self::Column<'_>) -> Self::Unpacked<'a> {
        column.as_ref()
    }
}

impl<'p> Param<'p, SimpleWorld> for &'p mut u32 {
    type Underlying = u32;
    type Column<'c> = RefMut<'c, [u32]>;
    type Unpacked<'u> = &'u mut [u32];

    fn column(world: &'p SimpleWorld) -> Self::Column<'_> {
        RefMut::map(world.column_3.borrow_mut(), |c| c.as_mut_slice())
    }

    fn unpack<'a>(column: &'a mut Self::Column<'_>) -> Self::Unpacked<'a> {
        column.as_mut()
    }
}

trait System<'p, W: World, Input> {
    fn run(&mut self, world: &'p W);
}

impl<'p, F> System<'p, SimpleWorld, (&u8, &mut u32)> for F
where
    F: FnMut(&u8, &mut u32),
{
    fn run(&mut self, world: &'p SimpleWorld) {
        let mut p1 = <&u8 as Param<'p, SimpleWorld>>::column(world);
        let mut p2 = <&mut u32 as Param<'p, SimpleWorld>>::column(world);

        let p1_slice = <&u8 as Param<'_, SimpleWorld>>::unpack(&mut p1);
        let p2_slice = <&mut u32 as Param<'_, SimpleWorld>>::unpack(&mut p2);

        for (a, b) in p1_slice.into_iter().zip(p2_slice.into_iter()) {
            self(a, b)
        }
    }
}

fn main() {
    let world = SimpleWorld {
        column_1: vec![1, 2, 3].into(),
        _column_2: vec![4, 5, 6].into(),
        column_3: vec![7, 8, 9].into(),
        _column_4: vec![10, 11, 12].into(),
    };

    (|a: &u8, b: &mut u32| println!("{} {}", *a, *b)).run(&world);
}

This code works fine when System is implemented for functions with concrete types, but when I go to generalize it by replacing that impl with this one:

impl<'p, F, T1, T2> System<'p, SimpleWorld, (T1, T2)> for F
where
    F: FnMut(T1, T2),
    T1: Param<'p, SimpleWorld> + 'p,
    T2: Param<'p, SimpleWorld> + 'p,
    <T1 as Param<'p, SimpleWorld>>::Unpacked<'p>: IntoIterator<Item = T1>,
    <T2 as Param<'p, SimpleWorld>>::Unpacked<'p>: IntoIterator<Item = T2>,
{
    fn run(&mut self, world: &'p SimpleWorld) {
        let mut p1 = <T1 as Param<'p, SimpleWorld>>::column(world);
        let mut p2 = <T2 as Param<'p, SimpleWorld>>::column(world);

        let p1_slice = <T1 as Param<'_, SimpleWorld>>::unpack(&mut p1);
        let p2_slice = <T2 as Param<'_, SimpleWorld>>::unpack(&mut p2);

        for (a, b) in p1_slice.into_iter().zip(p2_slice.into_iter()) {
            self(a, b)
        }
    }
}

I get the following error about the lifetimes of p1 (and p2):

error[E0597]: `p1` does not live long enough
  --> <source>:75:63
   |
63 | impl<'p, F, T1, T2> System<'p, SimpleWorld, (T1, T2)> for F
   |      -- lifetime `'p` defined here
...
75 |         let p1_slice = <T1 as Param<'_, SimpleWorld>>::unpack(&mut p1);
   |                        ---------------------------------------^^^^^^^-
   |                        |                                      |
   |                        |                                      borrowed value does not live long enough
   |                        argument requires that `p1` is borrowed for `'p`
...
81 |     }
   |     - `p1` dropped here while still borrowed

Is there something I can do with these where clauses to preserve the lifetimes here? It seems to me like p1 should live long enough as a Ref or RefMut, but for whatever reason I can't enforce that. Failing trait clauses, is there an unsafe way to get around this lifetime definition limitation without UB?

CAD97 · January 12, 2023, 4:55pm

The issue comes down to that in the specific impl you have

impl<'p, F>
where
    F: FnMut(&u8, &mut u32),

but in the generic impl you have

impl<'p, F, T1, T2>
where
    F: FnMut(T1, T2),
    T1: Param<'p, SimpleWorld> + 'p,
    T2: Param<'p, SimpleWorld> + 'p,

When you write F: impl FnMut(&mut T), this is a higher-ranked bound that F can accept any lifetime. With T1: Param<'p>, though, you only have that &'p mut _: Param<'p>, and no other lifetime satisfies the bound.

The fix will take the general shape of specifying T1: for<'a> Param<'a>, such that you can call unpack with a lifetime other than 'p (e.g. the reference to a local).

recatek · January 12, 2023, 5:43pm

Unfortunately I've tried all sorts of for<'a> combinations in the generic impl's where clause, like so:

    F: FnMut(T1, T2),
    T1: for<'a> Param<'a, SimpleWorld>,
    T2: for<'a> Param<'a, SimpleWorld>,
    for<'a, 'b> <T1 as Param<'a, SimpleWorld>>::Unpacked<'b>: IntoIterator<Item = T1>,
    for<'a, 'b> <T2 as Param<'a, SimpleWorld>>::Unpacked<'b>: IntoIterator<Item = T2>,

Those do compile for the body of the function, and do ensure the lifetime lives long enough, but then I can no longer auto-match this impl to the closure |a: &u8, b: &mut u16|{} down at the bottom in main() and I don't fully understand why.

error[E0599]: no method named `run` found for closure `[closure@<source>:92:6: 92:27]` in the current scope
  --> <source>:92:55
   |
92 |     (|a: &u8, b: &mut u32| println!("{} {}", *a, *b)).run(&world);
   |                                                       ^^^ method not found in `[closure@<source>:92:6: 92:27]`
   |
   = help: items from traits can only be used if the trait is implemented and in scope
note: `System` defines an item `run`, perhaps you need to implement it
  --> <source>:59:1
   |
59 | trait System<'p, W: World, Input> {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It seems I'm stuck with either one error or the other here. Is matching parameters like that to for<'a> where clauses not supported yet?

CAD97 · January 12, 2023, 7:07pm

I unfortunately can't really offer any tips; it's quite difficult to determine any nonimpl due to "impl is not general enough" unless you're lucky enough to be in a case where the compiler can point it out. The best I can offer is to look at bevy_ecs's traits and see if you can figure out how yours differ from theirs (since theirs obviously are working). Perhaps the world.access(closure) makes resolution easier than (closure).retrieve(world)?

Looking at the trait definition more closely, my intuition suggests that you do only need T1: Param<'p>; you call column only with the 'p-having input, and unpack is lifetime generic. I think the overconstraint is T1::Unpacked<'p> and that's what's trying to overextend the reference given to unpack to last for 'p.

I'd try for<'u> <T1 as Param<'p>>::Unpacked<'u>: IntoIterator, and then be out of ideas. I'd guess that bound won't work and require something like for<'q where 'p: 'q>, but that's not writeable yet. An extra trait for unpack to put the 'u lifetime in the trait list might work, so you can instead say T1: for<'u> UnpackParam<'u>, for<'u> <T1 as UnpackParam<'u>>::Unpacked: IntoIterator, but I honestly don't know. Higher-ranked lifetime bounds take a lot of experimentation to get right, and a lot of extremely subtle reasoning to prove sound if used to encapsulate an unsafe-requiring^[1] implementation.

A fun thing to do is to make a fully safe implementation (using others' unsafe-powered collections/abstractions) of an API surface to prove it's a sound interface, and then replace that safe implementation with an unsafe-accelerated implementation. You can still make mistakes, but it's a lot easier to show sound than vocabulary which can't be expressed without unsafe. And sometimes the safe implementation is good enough that you don't need the unsafe acceleration. ↩︎

system · April 12, 2023, 7:07pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Retrieving components as a concrete type from an ECS help	7	1199	January 12, 2023
Compiler unable to infer lifetime when calling RefCell::borrow() help	3	379	August 18, 2021
Newbie question on borrowing help	7	655	January 12, 2023
Issue with trait object lifetime help	7	432	March 18, 2023
Trait abstracting over Mutex and RefCell? help	5	2072	January 12, 2023

Borrowed value doesn't live long enough for trait, but does for concrete impl?

Related Topics