Lifetime troubles with Windows and Surfaces

help
1

Hi everyone,

I'm trying to polish up my Rust skills and wanted to play around with winit and wgpu. While doing so, I ran into a curious lifetime issue that puzzles me. This question is a distillation of what I was struggling with there.

The situation is as follows: First we create a bunch of Windows. Then we create for each Window a Surface for rendering into that window. The Surface borrows the window, so it has a lifetime parameter tracking the lifetime of the Window.

My idea was now to have a CommonOwner that holds all of the Windows and Surfaces.

Here's what that looks like:

use std::cell::OnceCell;

struct Window;

#[allow(dead_code)]
struct Surface<'a> {
    window: &'a Window,
}

impl<'a> Surface<'a> {
    fn new(window: &'a Window) -> Self {
        Self { window }
    }
}

struct CommonOwner<'a> {
    v: Vec<(Window, OnceCell<Surface<'a>>)>,
}

impl<'a> CommonOwner<'a> {
    fn new() -> Self {
        Self { v: vec![] }
    }

    fn create_window(&mut self) -> i32 /* id */ {
        self.v.push((Window, OnceCell::new()));
        (self.v.len() - 1) as i32
    }

    fn create_surface(&self, id: i32) {
        let (w, c) = &self.v[id as usize];
        let s = Surface::new(w);
        c.set(s);
    }
}

This does not work. The compiler is complaining:

error: lifetime may not live long enough
  --> <source>:32:17
   |
20 | impl<'a> CommonOwner<'a> {
   |      -- lifetime `'a` defined here
...
30 |     fn create_surface(&self, id: i32) {
   |                       - let's call the lifetime of this reference `'1`
31 |         let (w, c) = &self.v[id as usize];
32 |         let s = Surface::new(w);
   |                 ^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`

First of all: I don't get this error. What does it mean? CommonOwner has a lifetime of 'a, and &self is obviously a CommonOwner<'a>. Why would the compiler think that there is two different lifetimes here? I can't wrap my head around this.

The straightforward fix is of course, to change the signature of create_surface to include the lifetime:
fn create_surface(&'a self, id: i32).

But here's my second problem: In my case, create_surface comes from a trait which I don't control, so I cannot add a lifetime to &self without triggering a lifetime mismatch error with the trait. I also can't find a way to wrap the call away, because the lifetime always keeps bubbling up to the entry point controlled by the trait (which makes sense, I guess). But now I'm stuck and don't know what would be a sensible way to resolve this.

I'm figuring the self-referential nature of the CommonOwner is probably the root of the problem here, but I don't know how one would model such a scenario idiomatically in Rust.

Here's a godbolt link with the full example code.

1 Like
2

it sound like a classic self-referential data structure, which you cannot do in rust (without special treatments, that is)

&self is short for self: &Self, which in turn is short for self: &'_ CommonOwner<'a>. everytime you write &Type, there's always a (elided) lifetime there.

typically, references (or types that wrap references) are not meant to be stored freely, they are usually used within certain scopes, which are represented by the lifetimes in their types.

in cases where you do need to store references, you must store them seprately from the referents, which, depending on the use case, might need complete overhaul to the structure of your designs.

one big issue with storing references is that, it prevent you from moving the referent values, incidentally, you cannot return them from a function anymore.

for example:

fn foo() -> Window {
    let window = create_window();
    let surface = create_surface(&window);
    // now you are in pickle:
    // as long you want to keep the `surface` alive, the `window` cannot be "moved"
    ???
}

one possible design is use Rc (or Arc), but I thin you'd better rethink how to design the Surface type, and don't use references at all.

2 Likes
3

Simplest answer:

It's telling you to put 'a in your &self reference: &'a self.

4

That would be a self-referential struct, which is an anti-pattern in Rust. Given the second half of your post, perhaps you realize this.

The error is about the outer lifetime of the &'outer self, which is not 'a. &self is a &'outer CommonOwner<'a>, not a CommonOwner<'a>.

There may be a misconception here: Rust lifetimes (those '_ things) represent borrows, and are analyzed at compile time. They do not represent the liveness scope of values/objects, which is a dynamic property.

Or there may be a misconception about what Rust references are, depending on your language background. Rust references are closer to compiler checked C pointers than they are to C++ references.

Unfortunately, I don't know enough about the crates involved to offer a definitely-actionable suggestion for a fix. One high-level possibility is only creating the Surfaces on demand. Another is to do all your work below the stack location from creating the Window-borrowing Surfaces.

5

So, to make sure I understand this: The lifetime on the outer ref ('_) is saying: This is how long CommonOwner is alive. The lifetime parameter 'a is saying: This is how long whatever CommonOwner depends on is alive. Is that correct?

That's fine for me, at least for now. All the windows get created upfront and they stay never move once the surface creation has started. As you said, I could use Rc to mitigate this further, but at this point, that's not even necessary. The structure is sound, as far as I can see, and the lifetimes check out. I just don't know how to write that down.

Unfortunately, the surface type here is wgpu::Surface, which is not under my control. I also feel that this is the correct design, as the Surface will indeed become worthless if you drop the window.

6

How can I do that without triggering the lifetime mismatch with the trait?

7

The lifetime on the outer ref is saying, self is borrowed for the outer lifetime.

The lifetime parameter 'a is saying: CommandOwner contains a borrow that's valid for 'a.[1]

If by "alive" you mean "the object hasn't been destructed yet", that's not what Rust lifetimes are about.

  1. this is not the only meaning of a lifetime parameter, but that's what it means in this case ↩︎

2 Likes
8

in this case, it's possible, just create the windows in the lowest callstack frame (e.g. from within main directly), and move all the app code to a higher frame, and just pass those windows as arguments, example:

fn main() {
    // create all windows
    let window1 = todo!();
    let window2 = todo!();
    let window3 = todo!();
    // put them in a container if you like, this example use a Vec
    let windows = vec![window1, window2, window3,];
    run_app(&windows);
}
// all references to windows never escape scope of this function
fn run_app(windows: &[Window]) {
    // create the surfaces here
    let surfaces: Vec<Surface<'_>> = windows.map(...).collect();
}
1 Like
9

That's a classic trap, requiring &'a self/&'a mut self when the type you're implementing already contains 'a. Doing that will make those structs borrowed until they go out of scope, which will create a whole bunch of new unsolvable issues.

4 Likes
10

I did indeed. Unfortunately there is tons of info out there why self-referential structs are bad (which is also quite obvious to me), but very little about what to use instead. :sweat_smile:

I think that clicked now. Surface borrows Window. That borrow bubbles up to CommonOwner. The borrow checker has no way of knowing that that outer borrow ultimately refers to something owned by CommonOwner, so it assumes it's an unrelated lifetime.
Now create_surface() is storing the surface that depends on the lifetime 'a which the borrow checker can't look into. So it's complaining: "Hey, who tells me this 'a thing that the surface you just stored here depends on won't go away while self is still around".

If the borrow checker was able to realize that 'a really refers to something that lives inside the CommonOwner itself, things would be fine, but I can't tell it that because of the trait.

Thanks, this has been helpful!

Creating the surface on demand here means: whenever I need one, I create one, and then I destroy it again right away once I leave the local context? Yeah, that's not an option here. The Surfaces need to carry state.

I was also thinking of doing something along your second option, but there's a bunch of async machinery happening during the wgpu initialization that makes this challenging. I basically have to give up the call stack frequently to allow event processing, and the window creation has to happen inside of an event processing handler with winit.

11

That would be great, but this doesn't work within the constraints of the example where the trait fixes the signature for create_surface, does it?

What I could do though, is move all the windows to a global. It's not nice, but it works.

12

if you store them as globals, what's the point to contain the windows as field of CommonOwner then? why do you insist making a self-referential type? just don't store the windows, or just store references if you need accesses to them.

struct RenderTarget<'b> {
    window: &'b Window,
    surface: Surface<'b>
}
fn main() {
    let window = todo!();
    let render_target = RenderTarget::new(&window);
    //...
}

think twice, thrice, or however many times, about ownership. if you mix ownership and references, you eventually run into the self-referential problem.

13

I've been playing with wgpu & winit recently. Turns out that if you put the Window into an Arc and use that when creating the surface, you get a Surface<'static>, which removes the need for self-referencing. Essentially, the relevant part of my root structure looks like this:

struct State {
    window: Arc<winit::window::Window>,
    surface: wgpu::Surface,
}
5 Likes
14

If you have a mental model that's working for you, perhaps ignore everything I'm about to write.

The borrow checker isn't trying to reason about how long self is still around during runtime, which would be some global analysis of a dynamic property. Instead, it analyzes when places (like variables or fields) are borrowed in terms of code location, and checks if any uses of places conflict with those borrows.

Lifetimes and their relationships (in addition to how places are used) are how the compiler figures out the borrows.

    fn create_surface<'s>(self: &'s CommonOwner<'a>, id: i32) {
        // w: &'i Window
        let (w, c) = &self.v[id as usize];
        // s: Surface<'u>
        let s = Surface::new(w);
        c.set(s);
    }

The analysis in this case is something like

  • 'a: 's (implied the input types)
  • 's: 'i (via definition of Index::index and covariance of &_)
  • 'i: 'u (via definition of Surface::<'u>::new and covariance of &_)

But for c.set to succeed, you also need

  • 'u: 'a

Which in turn would require that 'a, 's, 'i, and 'u are all the same lifetime.

And you could do that, but as other comments have mentioned, the results won't be useful.[1]

Why does CommonOwner<'_> have to remain borrowed forever in order for this to succeed? If it didn't remain borrowed, you could replace self.v with an empty Vec and cause self.c to contain a dangling reference, using only safe code, as one example.

As another example, for all the compiler knows, even just moving CommonOwner<'_> could cause the reference to dangle. There's nothing in the language to distinguish a borrowed heap location from a borrowed inline field.

Until the language grows some new functionality, self-referential structs -- as in, built with actual Rust references -- are not useful in any practical way.

  1. The OnceCell makes the lifetime of CommonOwner<'_> invariant. ↩︎

15

Thanks. This is indeed the proper solution in this case.

It doesn't work for my initial reproducer, but that is because, as others have pointed out, my reproducer was showing a design that is just not sensible in the first place.

Thanks for the detailed explanation. I am aware of the static nature of the borrow checker, apologies if my ramblings gave a different impression.

A big thank you to everyone who took the time to respond! I feel that this really helped me improve my mental model about the implications of self-referential borrowing and where the current limits of the borrow checker lie.

1 Like
16

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.