Type erasure and dyn lifetime in a NonNull/Box struct field

help
1

I'm new to Rust, and I am trying to understand dyn object lifetimes when inside a Box or NonNull. My goal is to write an adapter for a C API. I have a few C struct types:

#[repr(C)]
struct ConcreteA {
    // generated by bindgen
}

#[repr(C)]
struct ConcreteB {
    // generated by bindgen
}

that all implement the same trait:

trait Trait {
    // common methods
}

impl Trait for ConcreteA {
    // implementation calls C functions
}

impl Trait for ConcreteB {
    // implementation calls different C functions
}

The structs are allocated and initialized in C, so I created a generic "handle" smart pointer type that owns an instance of one of those structs:

use std::{ops::Deref, ptr::NonNull};

#[repr(transparent)]
struct ConcreteHandle<T: Trait>(NonNull<T>);

impl<T: Trait> ConcreteHandle<T> {
    // The real code calls into C to allocate and initialize the memory; using a
    // Box is for illustration.
    fn new(v: T) -> Self {
        Self(Box::into_non_null(Box::new(v)))
    }
}

impl<T: Trait> Drop for ConcreteHandle<T> {
    // The real code calls into C to free the memory; this is for illustration.
    fn drop(&mut self) {
        drop(unsafe { Box::from_non_null(self.0) });
    }
}

impl<T: Trait> Deref for ConcreteHandle<T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        unsafe { self.0.as_ref() }
    }
}

Semantically, the handle more-or-less is the underlying concrete object—the lifetime of the handle and the lifetime of the underlying object are tied together. All good so far.

My trouble starts when I try to create a type-erased handle (to make it easier to hold a heterogeneous collection), like this (playground):

#[derive(Debug)]
#[repr(transparent)]
struct TypeErasedHandle(NonNull<dyn Trait>);

impl<T: Trait> From<ConcreteHandle<T>> for TypeErasedHandle {
    fn from(h: ConcreteHandle<T>) -> Self {
        Self(NonNull::new(h.0.as_ptr()).unwrap())
    }
}

impl Drop for TypeErasedHandle {
    fn drop(&mut self) {
        drop(unsafe { Box::from_non_null(self.0) });
    }
}

impl Deref for TypeErasedHandle {
    type Target = dyn Trait;

    fn deref(&self) -> &Self::Target {
        unsafe { self.0.as_ref() }
    }
}

I get this error:

error[E0310]: the parameter type `T` may not live long enough
  --> src/main.rs:63:14
   |
63 |         Self(NonNull::new(h.0.as_ptr()).unwrap())
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |              |
   |              the parameter type `T` must be valid for the static lifetime...
   |              ...so that the type `T` will meet its required lifetime bounds
   |
help: consider adding an explicit lifetime bound
   |
61 | impl<T: Trait + 'static> From<ConcreteHandle<T>> for TypeErasedHandle {
   |               +++++++++

For more information about this error, try `rustc --explain E0310`.

If I do what the error suggests and add 'static to the type bound, then the toy example compiles and runs, but I have several questions:

  1. Doesn't adding 'static (as suggested by the error) limit the usefulness of the TypeErasedHandle type? Doesn't it disallow conversion from a useful subset of ConcreteHandle<T>?

  2. How can I get an intuitive understanding of what it means to add + 'static to a type bound like this? I understand the use of 'static for objects that are truly required to have indefinite lifetime, but that doesn't apply here; the lifetime should match the lifetime of the TypeErasedHandle struct. (Right?)

  3. If I add a generic lifetime parameter like this:

    struct TypeErasedHandle<'a>(NonNull<dyn 'a + Trait>);

    then all users of the TypeErasedHandle type must supply a lifetime, which doesn't seem proper given the type's semantics. I want the lifetime to always be "the lifetime of the TypeErasedHandle struct itself". Is it not possible to express that?

  4. Should I store the output of pointer.to_raw_parts() instead of the fat pointer itself? (has the same problem)

  5. Would you take a different approach altogether?

2

I probably got something wrong, but If your struct implements Deref that means it is already a reference, so you could probably use it dynamically e.g:

    let a_ref: &dyn Trait = &*ConcreteHandle::new(ConcreteA {});
    let b_ref: &dyn Trait = &*ConcreteHandle::new(ConcreteB {});
    let list:Vec<&dyn Trait> = vec![a_ref, b_ref];

Why do you need a separate struct TypeErasedHandle for it?

PS: Could probably also be

impl<T: Trait> AsRef<T> for ConcreteHandle<T> {
    fn as_ref(&self) -> &T {
        unsafe { self.0.as_ref() }
    }
}
////
    let handle = ConcreteHandle::new(ConcreteA {});
    let a_ref: &dyn Trait = handle.as_ref();
3

Thank you for your answer.

I want something that owns the underlying object. The Deref is for implementation convenience, not because the handle is semantically equivalent to a reference. I could alternatively do something like this:

struct TypeErasedHandle(NonNull<dyn Trait>);

impl<T: Trait> From<ConcreteHandle<T>> for TypeErasedHandle { /* ... */ }

impl Drop for TypeErasedHandle { /* ... */ }

impl Trait for TypeErasedHandle {
    // forward every method to `self.0`
}
4

technically, dyn Trait is NOT a single type, but a type constructor: the real type always has a lifetime attached to it. see:

the usual notation are convenient shorthands, e.g. &'a dyn Trait is short for &'a dyn Trait + 'a, Box<dyn Trait> is short for Box<dyn Trait + 'static>.

NonNull<dyn Trait> is short for NonNull<dyn Trait + 'static>, similar to the Box case.

in your example, it's ok to put the 'static bounds on the ConcreteHandle since the pointee of the concrete type does not contain any rust lifetimes. this should compile:

impl<T: Trait + 'static> From<ConcreteHandle<T>> for TypeErasedHandle {
    //...
}
5

The phrase “the lifetime of” is a great way to get confused when talking about Rust, and the rest of your post demonstrates this confusion. The elements of Rust that are called “lifetimes” have almost nothing to do with how long a value exists; rather, they refer to how long a borrow (an invocation of the & operator or equivalent) is active. I strongly recommend that you avoid the phrase “the lifetime of” and try to find something more specific to say in each case instead.

Not unless one of the types that implements the trait contains a lifetime, which usually only happens when the values of that type contain references somewhere, which I would imagine is probably impossible or highly atypical for bindgen-generated structs. (By “contain” I mean, for example, that SomeType<'a> contains the lifetime 'a.)

A lifetime bound on a type means that the type cannot contain a lifetime that is shorter than that bound. Most types meet most lifetime bounds by not containing any lifetimes (in particular, not containing any references). For example, the bound u32: 'static holds because u32 contains no references. Therefore, you can coerce a u32 to a dyn Trait + 'static if u32 also implements the trait. The same logic should apply to your generated struct.

You should probably try generalizing ConcreteHandle instead of making a separarte TypeErasedHandle:

#[repr(transparent)]
struct ConcreteHandle<T: Trait + ?Sized>(NonNull<T>);

This change allows ConcreteHandle<dyn Trait + 'static> to exist. This might not work out for your needs, but if it does, it’s much simpler.

6

That's incredibly helpful. To help me reform my mental model, can you confirm whether the following statements are correct:

  • A reference of type &'a T (or &'a mut T) borrows a T value, and can be safely dereferenced until 'a. (The referenced T value will remain valid until at least 'a.)
  • A value of type Foo<'a> can be safely used until 'a. (The value contains—directly or indirectly—zero or more borrows, possibly phantom, and all of the borrows can be safely dereferenced until 'a.)
  • A type bound like T: 'a means that the T value must remain valid until at least 'a. (If the value contains any borrows, all of those borrows can be safely dereferenced until 'a.)
  • A type bound like T: 'a + 'b means that the T value must remain valid until at least the longer of 'a and 'b.

It's unclear to me what the type Foo<'a, 'b> says about how long it is safe to use a value of that type. My guess: The value can be safely used until the sooner of 'a and 'b. But maybe it's more subtle than that if an access can be limited to a certain part of the value?

So adding a 'static lifetime bound doesn't mean, "the value must be alive indefinitely", but rather, "the value must not have a built-in expiration date." Correct?

Interesting idea! I'll play with that and see how it goes. Thank you!

7

It means T can't contain any lifetimes that aren't 'static, which generally means that T (or its fields, etc) is not borrowing anything.[1]

The first step is to recognize that Rust lifetimes (those 'a things) generally denote the duration of some borrow, and not the liveness scope of any particular value. They're a compile-time, descriptive, type-level property. They're not a runtime property or a property of any particular value.

The second step is to know that lifetime bounds are defined syntactically. For example,

Ref<'a, T>: 'b

holds if and only if 'a: 'b and T: 'b. If T happens to be a generic, and whatever it expands to happens to have lifetimes or more type generics, this repeats recursively.

For 'static in particular, it means that the only lifetime that may appear in the fully expanded left-hand side is 'static.

As a concrete example, String: 'static as it trivially upholds that property. That doesn't mean that Strings never drop and deallocate, because we're talking about type level properties, not value liveness scopes.

If you have a dyn Trait + 'b, that means the type of the original value which coerced to dyn Trait + 'b met a : Trait + 'b bound.

Yes... but that answer may be misleading. Values of type &'a T don't have to stick around for all of 'a, and generally speaking the uses of references determine the lifetime in their type. The lifetime in the type does not determine how long the value is around and available to actually get dereferenced.

Let me try to make that more clear.

Note that usually if you can name the lifetime 'a, like in this function body...

fn example<'a>(s: &'a str) -> &'a str { s }

...the lifetime denotes some duration that lasts at least until the function has returned. Within the function this means little beyond "it's longer than the function body".[2] That is not irrelevant -- it means for example that you can never borrow a local for 'a (as all locals drop or move by the end of the function call, which are incompatible with being borrowed).

But it doesn't tell you how long the return value will be around / dereferenceable. The practical meaning of the signature is "so long as the return value is in use, *s remains borrowed". In effect how the return value is used will determine the borrow duration of *s in the context of the call site.

Lifetimes you can't name, like the borrow duration of locals, are inferred based on how the borrowing value is used, and how values with types which are related by a borrowing relationship get used.[3] So to continue the example...

fn ex2() {
    let mut s = String::new();  // L1
    let b = example(&*s);       // L2: let b: &'x = example::<'y>(&*s)
    // s.push('!');             // L3
    println!("{b}");            // L4
    s.push('!');                // L5
}

A borrow of *s gets created on L2, this borrow ends up connected to the lifetime in the type of b,[4] uses of b keep the lifetime in the type of b active which in turn keeps the borrow of *s active.

So *s is borrowed on L3 and L4 due to the use in L4. If you uncomment L3 you will get a borrow checker error. But nothing requires the borrow to be active on L5, so this compiles as-is.

Where we used b determined the lifetimes in question. We didn't assign some lifetime to b's type that said b could be dereferenced on L4 but not L5.

Yes, modulo the same notes as above.

Generally yes and probably how I'd recommend thinking about it. Lifetimes can technically appear without a borrow being made, purely through the type system, but I wouldn't worry about it when learning.

Definitely no, again consider that String: 'static. It means the type T is valid for 'a. It also means that any lifetime 'b or other generic 'U in the type represented by T meets a 'a bound.

As for the parenthetical, "yes... but that may be misleading", the same as talking about references themselves.

It means both T: 'a and T: 'b must hold.

Think of 'a and 'b as sets -- it may be the case that 'a is not longer than 'b and 'b is not longer than 'a 'b is not a subset of 'a and 'a is not a subset of 'b.

With that mindset, 'a + 'b is the union of 'a and 'b.

If the code is sound, it's generally fine to use values whenever the type is valid, and the compiler throws an error if you try to use a value in a place where its type isn't valid. In this case, that would be the intersection of 'a and 'b.

In practice, uses of values of type Foo<'a, 'b> usually determine what 'a and 'b can possibly be or result in borrow checker errors, like the examples above.

It's a type-level property, not a value property. It does effectively say "the value contains no borrows". (With some small print like "except maybe borrows of static or leaked memory.") It does mean "you can coerce the value to a dyn Trait + 'static" (assuming Ty: Trait). It does mean, "you can leak values into a &'static mut _ and keep it them around the rest of the runtime if you want to".

  1. At least, not in a way one usually cares about. It might technically be borrowing static or leaked memory. ↩︎

  2. especially since no relationship with other lifetimes is being expressed here ↩︎

  3. Even more accurately, no concrete lifetime need be computed. All that's required is proving some lifetime that doesn't have conflicts exists. ↩︎

  4. 'y: 'x -- "if 'x is active, so is 'y" ↩︎

8

True.

True.

“The T value must remain valid” is false or confusing depending on exactly what you mean.

This bound means that a T’s owner may keep it around and use it until 'a ends. It does not obligate the owner to keep it around until 'a ends.

Well, this contains the same confusion as the previous point, but other than that, yes. However, note that this is a simple logical consequence of “T: 'a and T: 'b”. That’s all the + means.

No, you have it right. However, there may be functions which take a Foo<'a, 'b> and return Bar<'a> or Bar<'b>, in which case the longer borrows Foo had may outlive the shorter of the two lifetimes, even though Foo doesn’t.

Exactly!