Lifetime issue with a 'constructor' function that takes no inputs and returns a struct with a lifetime

I'm having trouble with lifetimes again...

Over at https://github.com/neon64/opal/blob/master/opal_driver_gl/src/device.rs I have a struct called Device<'r> that has a lifetime parameter. I think that 'r should last for the same length as the struct itself, so I can call a method that takes &'r self, however the compiler keeps saying that I needs to last for the 'block suffix' of whatever the line of code previous to when I call Device::from_gl() (that's the function that returns a new Device struct).

In an extreme example, I managed to comment out all the dependencies of Device::from_gl() so that it took zero arguments, and rustc still complained that the references needs to be valid for the block suffix following some completely unrelated piece of code.

src/main.rs:229:1: 229:7 error: `device` does not live long enough
src/main.rs:229 device.test();
                ^~~~~~
src/main.rs:207:9: 381:2 note: reference must be valid for the block suffix following statement 3 at 207:8...
src/main.rs:207     }));
src/main.rs:208
src/main.rs:209     /*let limits = Limits::from_context(&gl);
src/main.rs:210     let global_state = GlobalState::from_context(&limits, &gl);
src/main.rs:211     let default_state = DefaultState::from_context(&limits, &gl);*/ // these aren't even used anymore
src/main.rs:212
                ...
src/main.rs:213:38: 381:2 note: ...but borrowed value is only valid for the block suffix following statement 4 at 213:37
src/main.rs:213     let device = GlDevice::from_gl(); // look no dependencies

I've tried to make a smaller example of this issue on the Rust playground and it works absolutely fine which is strange.

This time I have given up on trying to produce an isolated example as they never seem to work. Instead I've just put my entire repository up on GitHub in the hope that someone might see where I've made a mistake.

If anybody could see sense in this issue then that would be fantastic, as I feel that I'm so close and yet so far from actually being able to wrangle lifetimes instead of just being directed by the all-powerful compiler. :smile:

[quote="neon64, post:1, topic:2962"]
Device<'r>
...
&'r self
[/quote]This is a trap. It's extremely restrictive and you most likely don't want this.

Apparently, the smallest possible lifetime 'r starts at 207:9. The &'r self in the signature requires that device be borrowed for this whole lifetime. It's not possible because 'r outlives device (whose lifetime starts at 213:38). You might say it's just a technicality, but I'm not sure.

I'd look into removing 'r from &self in the trait, possibly implementing the trait for &'r Device instead of Device itself.

I think I need to take &'r self because in the device methods I'm storing a reference to the Device in the returned struct.

For example take this method:

/// Creates a shader object
fn create_shader(&'r self, info: R::ShaderCreationInfo, source: &R::ShaderSource) -> Result<R::Shader, ShaderCreationError>;

In my implementation I have the following struct that gets returned:

pub struct Shader<'a> {
    id: u32,
    ty: Type,
    functions: &'a gl::Gl, // <--- this gets taken from the `Device`
    pub shader_entity_index_mapping: HashMap<u32, (ResourceType, String)>
}

How can I borrow self.functions for the lifetime of the device without taking &'r self?

Do you know why 'r has to start at line 207? It seems like it is just always one line before I create the device which is really fishy.

When you say implement a trait for &'r Device does that mean I can write impl<'r> DeviceTrait<'r, Resources> for &'r Device<'r> or something similar? How is that different to what I have at the moment?

As you can see I'm a real noob at lifetimes.

EDIT:

My last resort would be to use reference counting inside the Shader, Buffer struct etc so that they don't need to have any lifetimes on them. But surely i can prove at compile time that the references are safe.

How is that different to what I have at the moment?

It might be the same... At least try to decouple the two lifetimes here:

impl<'r, 'x: 'r> DeviceTrait<'r, Resources> for Device<'x>

(the actual thing might have to be more complex) to allow borrowing the device for shorter periods than its lifetime parameter.

Thanks for the help. Decoupling the lifetimes gives me a bunch of lifetime errors in the implementation of the methods that I can't be bothered to work out at the moment, but I'll keep that suggestion for the future.

Regardless, I don't think the implementation of DeviceTrait really matters though, because I commented it out completely and inserted a test method in the Device struct:

pub fn test(&'r self) {
    // do nothing
}

And even that is not callable.

What I find peculiar is that the small example I made on the Rust playground works but the full code doesn't. I'm not sure what I'm doing differently in each case.

What i'm really interested in is if I write a function like this:

fn make_object() -> Object<'a> {
    Object {
         empty: Vec::new()
    }
}

And then call it somewhere:

let object = make_object();

How does Rust decide on that <'a> lifetime considering there aren't any input parameters. Can 'a be anything, or does it start where make_object() was called?

In an inherent method I just don't see what good &'r self can do you.

As I explained I'm pretty sure i need &'r self because I'm going to store that reference inside another struct.
Do you have any idea why the example on the playground works but mine doesn't? They both have methods taking &'r self.

I've not been able to conjure an example that would reproduce this error.

Still I'm not convinced that returning a reference to Device requires &'r self. I must be missing a reason why fn foo<'a>(&'a self) -> &'a Device<'r> is not sufficient.

And if almost nobody can guess the exact value of 'r in an instance of Device, the design might require simplification or at least some kind of ownership diagram.

I had pretty much given up and decided to go down a different path, however I tracked down why my example fails to compile!

Most of the structs stored by the 'resource cache' in my Device implement Drop (because they're wrapping OpenGL resources). Normally recursive references work fine like in my playground example:

pub struct SomethingWithALifetime<'r> {
    baz: HashMap<String, Bar<'r>>
}

impl<'r> SomethingWithALifetime<'r> {
    fn talk(&'r self) {
        println!("{:?}", self.baz);
    }
}

But as soon as I add an implementation of drop to Bar<'r>, it doesn't work due to the changes to dropck that go by the name sound generic drop. For a generic type to soundly implement drop, its generics arguments must strictly outlive it. <--- thanks Rustonomicon

Of course then it all makes sense, as rustc wants 'r to outlive Device<'r>.
I thought I'd be able to get around this with the #[unsafe_destructor] attribute but apparently it doesn't exist anymore.
Surely it should be possible with some unsafe magic to prove to rust that I'm not doing anything dirty in my constructors?

Now that i think about it, maybe I am trying to do the impossible and that rustc is correct not just over-protective: When device goes out of scope, vertex_array_cache goes out of scope, thus destructors of resources (e.g.: VertexArray<‘r>) are called. These need to access device one last time to release their resources (specifically, they need the device.functions). ?So that would be a use after free i suppose?
Is there a way to force it that vertex_array_cache is dropped before Device not after?

2 Likes

In addition, if I remove the RefCell<...> part and just have HashMap<A, B> where A or B implements drop, it works. So it seems to be a combinations of Drop and RefCell

Good job tracking this down! Maybe the last missing piece is this:

UnsafeCell<T>, Cell<T>, RefCell<T>, Mutex<T> and all other interior mutability types are invariant over T (as is *mut T by metaphor).


It does feel that Device should not own a collection of references to objects that reference it back. You could look into splitting the cache out of it or using Weak.

1 Like

Thanks for the info. I skimmed over the page but I'm still struggling to understand variance/invariance. If RefCell/Cell/Mutex etc. is invariant over T does that mean that RefCell<Foo<'a>> can't accept Foo<'static> or something? I feel this is a topic I really need to look into.

Theoretically, if Rust didn't check ownership/lifetimes, then my Device struct should work if:

  1. Device is created with an empty list of cached resources
  2. When device goes out of scope, vertex_array_cache is freed before any other fields so that the circular references inside those cached VertexArray<'a> structs don't become dangling pointers.
  3. Then the rest of Device is freed

That sort of thing should be possible at runtime (probably in an 'unchecked' language like C++), but I accept that since Rust is about proving safety at compile time it probably won't work.
I agree that Device shouldn't own this 'resource cache', however it would just be really convenient if it did because otherwise I'd need to pass a reference of the cache to all functions that want to use it -- definitely doable, but not desirable imo.

let device = Device::new(ResourceCache::new())
let cmd = device.create_command_buffer().clear(...).draw(...).build();

That is much better than this:

let resources = ResourceCache::new()
let device = Device::new()
let cmd = device.create_command_buffer().clear(...).draw(&resources, ...).build();

Especially considering draw() will be called thousands of times inside other methods.

1 Like

Could the convenience angle be solved with a proxy struct?

struct NiceDevice<'r> {
    device: &'r Device,
    cache: ResourceCache<'r>,
}

Now Device is free to outlive everything else and its methods or trait impls can be duplicated on this struct.

1 Like