A question about pinning

If I have a structure in memory that needs to have a constant starting address, Pin<P> seems to provide the guarantee needed (so long as safe code is used). However, using this type requires that P is a pointer and be Deref'able. As such, this is my understanding of what I should do if I want a structure to have a constant starting address:

use std::ops::Deref;
use std::pin::Pin;

struct MyStructInner {
    item1: usize,
    item2: usize
}

struct MyStruct {
    inner: MyStructInner
}

impl Deref for MyStruct {
    type Target = MyStructInner;
    fn deref(&self) -> &Self::Target {
        &self.inner
    }
}

fn main() {
    let mut my_vec = Vec::new();
    let my_item = MyStruct { inner: MyStructInner { item1: 0, item2: 0 } } ;
    my_vec.push(Pin::new(my_item));
    // now, accessing my_vec[0] should mean that the address stays constant so long as the object isn't removed from the vector?
}

However, something seems fishy about this... am I not thinking about pinning correctly? Should I make an Inner type when I need to make a pinnable object? Or, should I pin a mutable reference instead?

Well, because MyStructInner is Unpin, none of the pinning guarantees matter. The pin api is a clever one that utilizes unsafe in a few critical sections to make guarantees. Importantly, the compiler doesn't check if a type is actually pinned.

For your type, if you add a field of type PhantomPinned, you'll see the unsafe api that allows these guarantees. PhantomPinned opts out of Unpin and this allows you to rely on memory address stability if MyStructInner is behind a Pin<_>.

Look into the pin_utils crate to see how stack pinning works.

I think @RalfJung knows a lot about pinning, so he can explain it in more depth.

2 Likes

Vec doesn't give you any guarantees about addresses of its content, whether that's Pin or anything else. Everything in the Vec can move at any time the Vec decides to reallocate.

For Pin to ensure the pinned address doesn't change, it has to be given something that has an address in the first place. You're passing the struct by value, so there's no address to talk about!

So the first step is to give the struct an address by allocating memory for it with Box:

let my_item_at_some_address = Box::new(MyStruct { yada, yada });

Address of the Box is stable, and can be relied on (to the same extent as any address from malloc). If you need this address only for your purposes, and you trust yourself not to intentionally break that in clever ways, then this may be enough, and you don't even need to pin it.

2 Likes

Pin is a wrapper for a pointer P (e.g., P = Box<_>, P = &_, P = &mut _) that may inhibit the API of P (e.g., "remove" DerefMut), when the pointee does not have the Unpin property (marker trait). Moreover, Pin's contract also requires that the pointee will not be overwritten without calling its destructor.

So, if we forget about the Drop property (e.g., assume the pointee T does not needs_drop::<T>()), there are two things for the Pin to play any role:

  1. There must be indirection through some pointer P.

    The most common pattern is to have the data live on the heap and be "pinned" there (Pin<Box<_>>) thanks to:

    • Box::pin() to pin data from the stack to the heap,

    • Into<Pin<Box<_>>>::into() to "tag" heap-allocated data (Box<_>) as Pinned;

    The other option is to pin stack-allocated data in place (Pin<&mut _>), thanks to:

  2. For the Pin property to do anything, the pointee needs not to be Unpin. In nightly this can be achieved by doing impl !Unpin on your struct, and in stable you need to add a PhantomData-like field: PhantomPinned (I think that the name PinSensitive describes the role of this marker trait better).


Given these invariants, the only moment where you may rely a value of type T : !Unpin not to change, is when having a "witness" Pin<Ptr<T>>.


In practice, the Pin API is specially contrived, and not really something that helps that much, since usually custom wrapper types with their custom APIs end up needed. It was mainly added out of the fact that it was perfect for the Future API w.r.t. to the compiler-generated self-referential structs. So the main interaction with this API is Future-based.

3 Likes

Note that pinning does not add support for unmovable types to Rust. Instead, it provides support for pointers to unmovable data. So there is no way to say "no instance of my struct can ever be moved". But there is a way to say "please give me a pointer to an unmovable instance of my struct". That's why Pin is very tied to pointer types.

While pinning was primarily motivated by futures, its usefulness is not restricted to futures. It is needed by any API that wants to impose on the user restrictions of the form "do not move this instance of this type ever again (and also call drop before invalidating its storage)". Futures need that, but so do intrusive collections.

2 Likes

IIUC, the only way Pinning provides an additional guarantee w.r.t. intrusive collections is by allowing mutable pointers within the intrusive collection, right? (otherwise &'a Thing<'a> tricks lead to stack-pinning APIs that do not require Pin). The issue right now is that the non-aliasing semantics of Pin<&mut _> need to be clarified (I have a semi-clear mental model of this issue, that I would love to get clarified: if, for data pinned by a Pin pointer, "the backing memory is the real owner of the data", and since memory "can be aliased" (...?), then this data is inherently aliased, right?). I stumbled upon this "philosophical" question when trying to implement myself intrusive collections as part of the examples of the #[easy_pin] crate / attribute.

I don't think you can have intrusive collections (in the style I describe in my blog post) without Pin. Notice that this API lets you add a stack-allocated element to a collection where the collection outlives that stack frame. That is only possible because of the drop guarantee of Pin and AFAIK not possible without pinning. (There might be some closure-based API one could try but that will be really, really unergonomic.)

I have heard about this trick before, but I am far from convinced that it will be guaranteed to work in any future version of the borrow checker. We've cases before where people made tacit assumptions about Rust never accepting more programs than it does right now, and then that code breaking later. Such negative reasoning is very fragile.

Yeah but that's most orthogonal to what was discussing here.

1 Like

Challenge accepted?

Oh, also this requires extra care and a macro because otherwise you do not get the "drop" properties of Pin even with current versions of Rust (ever since ManuallyDrop is a thing, you can have such "self-referential" references to things that are not dropped). So at this point you start re-inventing the stack-pinning part of Pin.

Furthermore, as you said yourself, this does not support pinning things on the heap. So overall, relying on this "self-referential" trick gives you a less ergonomic and weaker API that moreover only works for shared references. In contrast, Pin lets the user do more, supports mutable references and also expresses intent much more clearly.

1 Like