Pinning in plain English

[Tamschi]

I tried to strike a balance between making it easy to read and being thorough, so (in hindsight ) the most important points are front-loaded and you can read on if you'd like to learn more.

Abstraction Haven

It's not a complete guide, since there's actually no how-to, but I hope it works as general overview (and maybe encourages others to box their Futures a bit less often, or to support !Unpin types in a few more places).

[robber]

Great read! I initially struggled a lot with the Pin reference documentation (I especially get confused by the "pinning is structural" terminology). This post would have been very helpful for me! Thanks for putting this together! I have it bookmarked for the future

Thanks!!

P.S. Minor notes on two points I had to read over a few times which could maybe be reworked:

• A "pin" type that enforces "the value can't be moved" towards safe Rust.

^unclear to me what this is saying. Especially the "towards safe Rust" bit

Confusing table entry:

• rarely. If yes, then pinning isn't meaningful.

[steffahn]

Wow, that's a remarkably good post!

[Tamschi]

I'm glad it's helpful, or would have been if it had come earlier at least
And thank you for the criticism, as that's what makes posts like this possible in the first place!

"towards safe Rust"

You're right, that wasn't the best wording so I've adjusted it.

The difference between safe and unsafe Rust here is that unsafe Rust can convert from Pin<&mut T> into &mut T using Pin::into_inner_unchecked (or a reinterpret-cast), though doing so is unsound in many cases.

I didn't mention that explicitly since it seemed like a detail not too relevant to the concept of pinning itself, and it seems reasonably easy to discover in the docs if you implement a type that's !Unpin or a pinning type manually.

"rarely. If yes, then pinning isn't meaningful."

I've adjusted it to say " rarely in practice, as pinning with T: Unpin is meaningless."

Pin<&T> or Pin<&mut T> with T: Unpin may appear to the consumer of a generic API, but this isn't something I've come across so far. (I'll have it as side effect of my dependency injection framework though, which will dynamically provide even instances of Unpin types as Pin<&T>.)

[Tamschi]

Thank you!

It actually wasn't that much work compared to my other posts, since I didn't program anything for this one and usually go into more detail. Maybe less is more?

I'd have been tempted to add a few more expandable asides, but alas I don't get to use custom CSS or HTML here.

---

The main difficulty difficulty with writing posts like this is having enough friends who complain to you about stumbling blocks, and then getting them to read and ideally pick apart your first draft(s).

The rest is, for me at least, just stopping myself from going into too much detail. I'm not good at writing Q&A or dialogue-style posts in the first place, so concise-ish explanations like this come more naturally to me.

[Tamschi]

I updated the "The Problem" section to give a bit more context for those coming from other programming languages and smooth out the initial terminology pile. Please let me know if that's still fine or if it's too verbose now.

[steffahn]

@steffahn

Wow, that's a remarkably good post!

@Tamschi

Thank you!

It actually wasn't that much work compared to my other posts, since I didn't program anything for this one and usually go into more detail. Maybe less is more?

Let me give slightly more detail: I’m unable to actually evaluate the post from the perspective of someone unfamiliar with Pin (although I’ve never before seen the idea to put collections C into Pin<_>, instead of just types that implement Deref). What I do find remarkable though is the fact that an introduction that's trying to make Pin more easy to learn still doesn’t oversimplify anything. In fact, on some points the post is more accurate than the standard library docs itself, because the std docs point out lots of things simply as “you must not do this or that” while the correct / more accurate description is often “you must not do this or that, unless it’s your own type that’s pinned and you know the precise circumstances in which pinning is or isn’t necessary for soundness”. Exactly your point in “Pinning is a matter of perspective”. Similarly, the overview of and references to some use cases of pinning are super important.

---

Anyways, let me point out a few points that might be improved nonetheless

This is possible because moves in Rust are already pretty explicit once references are involved: The underlying assignment maybe be hidden inside another method, but there is no system that will move heap instances around without being told to do so (unlike for example in C#, where pinning is a runtime operation integrated into the GC API.)

The only exception to this are types that are Copy, a trait which must be derived explicitly for each type for which implicit trivial copies should be available.

(link)

I don’t quite understand what you mean by “The only exception to this are types that are Copy”. First of all, it isn’t clear what “this” is referring to, i.e. I’m wondering “exception from what exactly?” when reading this sentence. (After all, the last point of the previous sentence was some explanation about GC in C#, so that’s definitely not the thing being referenced.) And even if I relate this to “there is no system that will move heap instances around without being told to do so”, it doesn’t make sense. Maybe it relates to “moves in Rust are already pretty explicit” – but Copy doesn’t really concern moves... maybe something about “explicitness” is related to Copy, I don’t know.

In fact, the only primitive type that is explicitly !Unpin is core::marker::PhantomPinned , a marker you can use as member type to make your custom type !Unpin in stable Rust.

(link)

In my mind, the term “primitive type” only refers to these types, as well as tuples, arrays, slices, string slices, references, pointers, function pointers, i.e. the ones that you don’t have to import (even without the prelude), and the ones with special syntax. One page in the reference even only lists boolean, numbers, char, str, and the never type; and calls the rest of non-user-defined types “builtin types”, which then would also include function types, closure types, trait object types, and abstract impl Trait return types.

Maybe instead of talking about “primitive types” you should use “standard library types”, or something like that?

However, as the type of the pinned instance itself does not change, it can remain visible "unpinned" inside the module that implements a pin in the first place.

Pin<_> hides the normal mutable API only through encapsulation, but can't erase it entirely.

This means that safe code in that module can often move an instance even after it appears pinned to code outside of it, and extra care must be taken to avoid such moves.

(link)

I’m not entirely sure what kind of example you have in mind here. It’s certainly the case that when you’re implementing an !Unpin type and using unsafe code, then some of your safe code might violate properties necessary for soundness. But that’s more of a general rule for unsafe code, right? Quite often, unsafe implementations of e.g. some data structure relies on encapsulation for soundness, and the effect is that safe code in the same module has the power to violate soundness requirements, too. But maybe you’re saying with pinning, it’s particularly easy to accidentally violate the necessary requirements in safe code because moving values is a common thing to do in Rust and often fairly implicit? Nonetheless, it could be more clear that “extra care must be taken” only applies when your code also uses unsafe, since without it you either cannot work with something like Pin<&mut T> at all, or you’ll have to use safe abstractions like pin-project, but in the latter case no extra care must be taken anymore.

[Tamschi]

Ah, my documentation is actually always like this now, as long as I get around to writing it at all. There was admittedly a jump in quality earlier this year when I started taking my open-source work more seriously, but the level of precision is normal for me.

I mainly put the difference to the standard library docs down to giving myself more space and not trying to future-proof a feature where any breaking change could cause a lot of compatibility issues.

I'm still taking this as very high praise and motivation to publish more posts like this, if I see another topic where there's similar confusion and I can't find a good resource to link to.

(Technically I could write another about how to make a collection pinning like this, but as long as arbitrary_self_types hasn't landed on stable, I'd still have to resort to extension traits. I also don't need anything really simple in that area right now and using a full-on map crate (see below) as example is maybe a bit much.)

@steffahn

(although I’ve never before seen the idea to put collections C into Pin<_> , instead of just types that implement Deref )

That's original content, or at least I'm pretty sure it is
I have a rudimentary implementation here and it took me a few tries to get this entirely correct.

I believe it raised a few eyebrows when I brought it up in the Community Discord, and I'm admittedly jamming something off-axis into an API not created with it in mind, but it works perfectly and makes complete sense, doesn't it?

---

@steffahn

I don’t quite understand what you mean by “The only exception to this are types that are Copy”. First of all, it isn’t clear what “this” is referring to, i.e. I’m wondering “exception from what exactly?” when reading this sentence. (After all, the last point of the previous sentence was some explanation about GC in C#, so that’s definitely not the thing being referenced.) And even if I relate this to “there is no system that will move heap instances around without being told to do so”, it doesn’t make sense. Maybe it relates to “moves in Rust are already pretty explicit” – but Copy doesn’t really concern moves... maybe something about “explicitness” is related to Copy , I don’t know.

I've clarified it like this just now:

@Tamschi (new)

The only exception to this, where a & -referenced instance's copy can appear with a new address implicitly, are types that are Copy . This trait isn't auto , which means it must be derived explicitly for each type for which implicit trivial copies should be available.

@steffahn

[…]

Maybe instead of talking about “primitive types” you should use “standard library types”, or something like that?

You're right, the word I was looking for was "built-in". I clarified PhantomPinned a bit differently, too.

@steffahn

I’m not entirely sure what kind of example you have in mind here.

My TipToe struct is an embeddable reference-counter that is !Unpin and interior-mutable.

The implementation is a little convoluted, but all directly related code (so minus the smart pointers' memory management) needs only safe Rust features as a result.

This also goes for consumers, which can implement the unsafe trait IntrusivelyCountable using only safe Rust statements, as seen here: tiptoe - Rust

Maybe this isn't an actual example though, insofar that implementing that trait already counts as unsafe Rust code by itself

---

You can see all changes I just made in context here: update post: Pinning in plain English · Tamschi/Abstraction-Haven-Backup@e8c3a20 · GitHub

[Tamschi]

@steffahn

I’m not entirely sure what kind of example you have in mind here. It’s certainly the case that when you’re implementing an !Unpin type and using unsafe code, then some of your safe code might violate properties necessary for soundness. But that’s more of a general rule for unsafe code, right? […]

I remember now what that was about: If you implement a pinning collection, rather than a pinning pointer, you're likely to share a bunch of code between the "pin" and non-"pin" variants of certain methods.

If your collection is a compound of collections that aren't pinning-aware, then it's not feasible to use pinning internally. The pinning API is just a signature-changing proxy then.

If it's interior-mutable via Mutex or Cell, then that can easily give you a safe API that could move values you present as pinned outside your module. The pinning API just calls the non-pinning one in that case, and unsafely upgrades the return values.

I've clarified my post in that regard: update post: Pinning in plain English · Tamschi/Abstraction-Haven-Backup@47b6d83 · GitHub

[vague]

Your article is so much clear that I can't help translating it into Chinese here. I really appreciate it!

[Tamschi]

Awesome, thank you!
I've added a link to your translation to the original post to make it easier to find.

I appreciate the list of headings to the side. Hashnode can technically generate one, too, but that would take up the entire screen. I'll try to add navigation similar to yours when I get access to custom CSS.