What does move and Pin really do?

[andy-yx-chen]

My understanding is move is similar to std::move, is that correct? if that's the case, why do we need pinning? as Pinning - Asynchronous Programming in Rust, it saids

    struct ReadIntoBuf<'a> {
        buf: &'a mut [u8], // points to `x` below
    }

    struct AsyncFuture {
        x: [u8; 128],
        read_into_buf_fut: ReadIntoBuf<'what_lifetime?>,
    }

it mentions that,

However, if AsyncFuture is moved, the location of x will move as well, invalidating the pointer stored in read_into_buf_fut.buf
Does it mean array is stored as c/c++ array, instead of pointers? If that's the case, can it be replaced with Arc instead of Pin?

[alice]

You can find a very thorough explanation of pin and futures here.

[Ixrec]

@andy-yx-chen

My understanding is move is similar to std::move, is that correct?

Not really. In C++, std::move() performs a cast to rvalue reference, which in some cases allows a move to take place, but it's very different from the move itself. In Rust, the r/l/x/pr/plvalue reference taxonomy is completely absent and Rust references have no special behavior when it comes to move semantics, so the very concept of an "rvalue reference cast" has no meaning.

What is true is that a "move" in C++ is conceptually pretty similar to a "move" in Rust, but the rules for when they happen and what they do are completely different (in particular, Rust's rules are far simpler). Understanding Ownership - The Rust Programming Language is the canonical explanation of basic ownership and borrowing in Rust, including moves, so just read that.

@andy-yx-chen

if that's the case, why do we need pinning?

That's largely a separate issue, but it mostly boils down to "actually C++ would need it to if it wanted to guarantee memory safety". The core problem that Pin solves is that some types simply cannot be moved safely, but Rust assumes anything can be moved. C++... sort of does too, but it also assumes that you the programmer will simply know not to move things that silently lead to UB after being moved.

@andy-yx-chen

Does it mean array is stored as c/c++ array, instead of pointers?

If you're talking about the x: [u8; 128], then yes, that really is a fixed-size array inside the AsyncFuture struct without any pointers or other indirection, just like in C or C++.

@andy-yx-chen

If that's the case, can it be replaced with Arc instead of Pin?

In principle, yes, and there are languages that essentially do this to handle features like async/await or coroutines. But of course that requires potentially doing a dynamic memory allocation on every single async/await, which would not be a "zero-cost abstraction".

For more on why async/await works the way it does, and needs to work the way it does to be truly zero-cost, see alice's link.

[andy-yx-chen]

@Ixrec

ular, Rust's rules are far si

haven't read through it yet, but looks nice

[andy-yx-chen]

Thank you so much, very detailed, best community ever!

[kornel]

I'd say Option::take() is closer to std::move(), and Pin is a totally different concern.

To me Pin is a type-system marker for "if you memcpy this content to another address it may not make sense anymore". I don't know if C++ has an equivalent of that. In C that's just "well, duh!" assumed about every non-trivial object.

If the content of Pin is marked as safe to move anyway (Unpin), then Pin does nothing.

If the content may be relying on being at a specific address (e.g. it's self-referential) then Pin won't let you see it until you acknowledge with an unsafe block that it's dangerous to copy or move that data to a different address, and you promise you won't, because in this case the borrow checker can't help you.

[Michael-F-Bryan]

@kornel

To me Pin is a type-system marker for "if you memcpy this content to another address it may not make sense anymore". I don't know if C++ has an equivalent of that. In C that's just "well, duh!" assumed about every non-trivial object.

I really like this explanation! That's probably the most intuitive formulation I've heard so far.

[Yandros]

It's not a complete definition (but I agree that is a very nice short way to describe the main point of using Pin ) though, since Pin interacts very heavily with Drop semantics: Pin implies not only that the pointee has a "fixed address" ... but also that such address must remain valid until the pointee is dropped. That is, Pin is also a way to opt-out of compile-time lifetimes / borrows while still keeping some invariants; c.f. my post about it:

@Yandros

/// The following code causes Undefined Behavior (Run with MIRI to confirm)
fn main ()
{
    use some_lib::{BackPtr, SmartInteger};

    let back_ptr = BackPtr::new();
    {
        let mut smart_integer = SmartInteger::new(42);
        {
            let at_smart_integer = unsafe {
                Pin::new_unchecked(&mut smart_integer)
            };
            at_smart_integer.register_self_to(&back_ptr);
            assert_eq!(back_ptr.deref_read(), Some(42)); // OK
        }
        mem::forget(smart_integer); // swap isn't even needed, forget suffices
    }
    dbg!(back_ptr.deref_read()); // UAF
}

• Playground

• Since someone having ownership of the value after the lifetime of the pinned reference can thus cause unsoundness without using unsafe , it means that the unsoundess must lie somewhere else: in the Pin::new_unchecked call.

Suffices to say, Pin is very complex abstraction; the property of "preventing ill-formed mempcys" can be more easily provided by immutable heap pointers than by these advanced Pinned pointers. Only use the latter if you do need some form of (restricted) mutation anyways, and/or if you want to support such stack pointers (what most other languages do not even dare do).

[Michael-F-Bryan]

@Yandros

It's not a complete definition (but I agree that is a very nice short way to describe the main point of using Pin ) though

I'd use it as more of a "lies to children" thing, in the same way you are taught F = ma when starting physics even though relativity says that's not the whole truth.

So introduce the point about not memcpying, then once people understand that you can extend it to include Drop guarantees.

How about this:

If you memcpy the thing behind this pointer to another address or invalidate it before it gets dropped, the thing may not make sense anymore.

I'm not sure how you'd explain pin projection in a one-liner though...

[alice]

@Michael-F-Bryan

I'm not sure how you'd explain pin projection in a one-liner though...

How about "specific types can decide which of its fields inherit the no-memcpy property"

[cfsamson]

I wrote about Pin recently since I wanted to understand it thoroughly myself. It explains Pin from a slightly different angle than the async book, and might be helpful. It's not exactly a one-liner though...
.

I just wanted to expland a bit on the use of Arc instead of Pin. If we agree that a move of AsyncFuture would invalidate any internal references (read_into_buf_fut points to x so it's self-referential), using Arc won't help since you can trivially get a mutable reference to an Arc if there are no strong/weak pointers to it using Arc::get_mut() in safe Rust. By doing that you can move the data and invalidate any internal pointers.

It's a little more complex than this as well and not easy to explain in just a few sentences.

[andy-yx-chen]

My understanding is, in case of AsyncFuture, we can make it to Arc, then it's safe to move, my sense is it should not be a problem, as in today's C++ with lamda callbacks, most of scenarios are storing context in shared_ptr, there is no reason that Arc or something else to enforce the owership could break, but saving another unnecessary allocation convinced me