Pin tutorial are confusing me

From what I've gathered reading tutorials, an often repeated statement is Pin prevents an object from being moved. Just ctrl+f "move" at these turorials 1 2 3 4 and in the async book where it says

Pinning makes it possible to guarantee that an object implementing !Unpin won't ever be moved.

Okay, so I'm someone who knows nothing about Pin. I have no background context to work off of so I read that with a pure mind ready to take everything literally. So I go and write

    fn move_me(v: Pin<Box<Vec<u8>>>) {}
    let mut v = Box::pin(Vec::<u8>::new());
    move_me(v);

No error.. I read the Pin docs and figure out that P in Pin<P> must implement !Unpin or not implement Unpin. Vec implements Unpin. So Box<Vec> implements Unpin. Looking at the docs, I can stuff a struct with a PhantomPinned field to mark it as !Unpin.

    struct NotUnpin { _p: PhantomPinned }
    fn move_me(n: Pin<Box<NotUnpin>>) {}
    let mut n = Box::pin(NotUnpin{ a: 1, _p: PhantomPinned::default() });
    move_me(n);

Again no error. an object implementing !Unpin won't ever be moved. After looking and looking at example after example I notice all of them are using &mut Pin<...>.

    let mut n = Box::pin(NotUnpin{ a: 1, _p: PhantomPinned::default() });
    move_me(&mut n);

Still no error. Well if we're taking an exclusive reference &mut Pin<...> that means we are trying to mutate it. What the heck lets try

    let mut n = Box::pin(NotUnpin{ a: 1, _p: PhantomPinned::default() });
    self.a = 1; // cannot assign to data in dereference of `Pin<Box<NotUnpin>>`

Finally! Proof-by-elimination. So the word move is some kind of metaphor for mutation. Or rather, mutation is a metaphor for move? When I assign over old data, I just think I'm copying data from one variable to the other. Move here is such an overloaded term I don't know what is being referred to. Move as in simple copy or move semantics copy-and-destroy or or.. Ambiguity hurts learning, especially if you don't know the disambiguating context that the teacher knows.

The other theme I keep hearing is that Pin allows you to construct self-referential structs. Again, while this is a metaphorically true statement, it isn't technically true. There are a million safety issues related to self-referential structs not unique to Rust. After pouring through Pin it seems to me that it instead allows you to safely use (as opposed to construct) already 99% safely defined self-referential data structures. You don't just define a data structure with self-referential pointers, then stick the data structure in a Pin and walla. Which is what the statement allows you to construct self-referential structs found in many tutorial implies. No, actually literally states.

2 Likes

Please clarify whether your goal is to provide feedback on the tutorial, or to receive an explanation for some specific things you find confusing.

1 Like

Tutorials say I can't move things when they are pinned. I demonstrated that I can move things when they are pinned. I attempted to reason about this contradiction by surmising it is because the word move is not being used to refer to move semantics. As an expert at being ignorant of what Pin does, I can assert with expertise that other ignorant readers have a hard time with Pin because the ambiguous terminology used to describe it's actions in the language.

1 Like

You are not wrong. Unfortunately, the particular tutorial you are reading is an abandoned project, so there is not really a good place to leave feedback that someone will act upon.

2 Likes

Here you don't move the NotUnpin. You move the (pinned) Box, and Box implements PinUnpin. (Besides: Pin is for pinning the value pointed-to by the pointer and not for pinning the pointer itself.)

Try this:

use std::marker::PhantomPinned;
use std::pin::Pin;

fn main() {
    struct NotUnpin { _p: PhantomPinned }
    fn move_me(_n: Pin<Box<NotUnpin>>) {}
    let n = Box::pin(NotUnpin{ _p: PhantomPinned::default() });
    move_me(n);
    let m = Box::pin(NotUnpin{ _p: PhantomPinned::default() });
    fn move_me2(_m: NotUnpin) {}
    //move_me2(*m); // doesn't work
    //move_me2(*Pin::into_inner(m)); // doesn't work either
}

(no Playground as it's currently giving me errors)

1 Like

The reason you can move the box is not that it implements Unpin. The box has not been pinned at all (only its contents have been pinned), so whether or not it implements Unpin is irrelevant to whether you can move the box.

3 Likes

Yeah I figured that afterwards, thanks for pointing this out explicitly. That's why I added:

That's what you mean, right?

That's true in the sense that moving Box does not move it's value on the heap. Only the Box pointer itself gets moved. But this requires knowing that context specifically when learning about Pin. Because the first thing that stands out when you learn Rust is ownership. You start to understand that variables can be borrowed or moved. So as a reader of tutorials, ignorant of the special memory related contexts of using Pin, you are unaware which is which. You have to be ignorant of the context because knowing the context implies you already know about Pin.

1 Like

You never explicitly answered my question on whether you wanted explanations of how these things actually work (which is ironic considering that you are complaining about ambiguity), but currently it seems like you want to provide feedback on the tutorial.

In general, I agree that the tutorial is confusing on the areas you have pointed out.

3 Likes

It seems to me what's being shown here is that with Pin you can't move out or dereference the inner value from Pin. Mutating the inner value requires a deref. It's starting to come together.

No no I actually do want to an explanation. I just don't know where to begin other than to vent in frustration. I don't know what specific questions to ask. I have a starting point, an ad hoc self-referential struct. But I'm basically going around to different tutorials and docs until I have a clear picture of what Pin does so I can ask clarifying questions. I don't want to just say "teach me all there is to Pin" out of basic decency :slight_smile:

1 Like

One way to phrase what pin guarantees is that, when you pin a value of type T at some memory location, then that memory location will continue to contain a valid value of type T until you run the destructor on that memory location. Furthermore, the type T is given full control over all modifications to that memory location — for example, the user cannot swap it for some other value of type T.

7 Likes

You can dereference, but not mutably dereference (edit: unless the value pointed-to implements Unpin):

use std::marker::PhantomPinned;

fn main() {
    struct NotUnpin { _p: PhantomPinned }
    let n = Box::pin(NotUnpin{ _p: PhantomPinned::default() });
    let _r: &NotUnpin = &*n; // works
    //let _m: &mut NotUnpin = &mut *n; // fails
}
2 Likes

And this is done by simply ensuring that you can't mutably deref the T? As in, there isn't any magic machinery or special allocator stuff the compiler does to somehow protect that memory location. Just simply disallows you from using it in the wrong way (deref mut)? This is purely a language semantics thing, like the borrowing rules?

Yeah, there's no magic. It's just that there's no safe way to go from Pin<&mut T> to &mut T, so you cannot obtain a mutable reference to the value, so you can't do anything stupid.

9 Likes

I struggled a lot with understanding Pin and Unpin, and I still don't overlook it very well. I think what helped me most is to understand that in Pin<P>, the P is actually a pointer-like type and the pointed-to value (edit: which is to be pinned in memory) has the type <P as Deref>::Target. E.g. P could be Box<NotUnpin> and <P as Deref>::Target would be NotUnpin.

2 Likes

Well, formally you could say that Pin is a pointer-like type too, because it implements Deref. But it's a zero-cost wrapper, and its target isn't pinned in memory (but the target of the target is). (hope I got this right)


Of course, only if the target of the target is !Unpin, it gets pinned in memory.

The central rule that makes Pin necessary is that, if your program has only a single unsafe block, then if your program does something that causes undefined behavior, then the unsafe block is at fault, no matter what. It doesn't matter how stupid the non-unsafe code is.

To see why this rule is problematic without Pin, consider what happens if you use unsafe to create a self-referential struct on the stack. When the owner of the struct moves the struct (i.e., the data gets copied over to a new memory location), then your self-referential pointers still point at the old location. If the struct unsafely uses those self-referential pointers, that's a use-after-free, which is undefined behavior. There's no way for a struct to know if you've moved it, so you can't avoid this. (Self-referential structs on the heap are completely possible without Pin, by the way.)

Now, note here that the user of the struct didn't do anything unsafe. Moving stuff you own is safe. Therefore, the unsafe code in the struct's implementation must be at fault.

How do we fix this? Well, if we force the user to write an unsafe block before they can use the struct, then there's no problem. If the user moves it, then the unsafe block that the user wrote is at fault, and there's no problem. It's not our fault anymore.

It turns out, that's all what Pin does. The main constructor Pin::new_unchecked is unsafe. The caller promises to not move the struct, and now they gain the ability to use methods that take self: Pin<&mut Self>. Tada!

So the truth is that Pin is just a tool for shifting the blame if something goes wrong.

Well, there are some convenience methods like Box::pin that let you pin something without unsafe. However, in this case, the unsafe is just in the implementation of Box::pin instead, and it is justified due to the API design of Pin making it impossible to do anything that would result in the contents being moved to a new memory location.

Point is, if you ever see a Pin<&mut Self>, then you know that someone, somewhere, has written an unsafe block that promises that this particular value is pinned. And that it would be ok to rely on this promise in your own unsafe code.

22 Likes

Thank you that absolutely demystified it for me. Now I have eliminated an entire set of possible theories about what's going on behind Pin. In the end, it's just a type-safe api for operating on a type meant to be self-referential. Because self-referential data structures are extremely sensitive. Crazy how something so simple can confuse so much. I mean, I'm sure figuring out if the api and it's constraints would lead to safe code was a tough task, but the end product isn't that difficult. If there is no magic, then I should be able to understand how to use Pin by simply grokking the docs.

This reminds of the notorious nature of learning monads. Very simple, almost dumb structure in programming. But the drama of them being this major thing I think keeps many people from simply understanding them. Like monads don't deserve any burden in understanding. Maybe in category theory but not programming languages. But when you read about stuff like this, you are told grand big ol things about them, but the ambiguities keeps you from believing what you are being told. Sometimes you just need to be told there is no magic, no implicit behind the curtains stuff going on, this is what it is full stop. The only thing left to do is practice using it and burn it into memory.

7 Likes

Yeah needing the inner type to be a pointer type makes it clear what the intention is in handling it.