pin-project/pin-project-lite is the primary way Rustaceans project a pin from a type, T, to one of its fields when the field is ?Unpin. Is there an idiomatic way to project a pin to a field of a field when the innermost field is ?Unpin?
use core::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
use pin_project::pin_project;
struct Foo<T>(T);
#[pin_project]
struct Bar<T>(#[pin] Foo<T>);
impl<T: Future> Future for Bar<T> {
type Output = T::Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// This is more concise and relies on the same amount of
// `unsafe` code as the commented out portion.
// SAFETY:
// This is okay because `self.0.0` is pinned when `self` is.
let p = unsafe { self.map_unchecked_mut(|f| &mut f.0 .0) };
p.poll(cx)
//let projected_foo = self.project().0;
//let p = unsafe { projected_foo.map_unchecked_mut(|f| &mut f.0) };
//p.poll(cx)
}
}
I realize this is rather contrived. For example Foo would likely be defined in a separate crate; and unless Bar knows Foo doesn't implement Unpin for T: ?Unpin, then PhantomPinned would need to be added to Bar to prevent it from auto implementing Unpin.
In this example, I think§ Choosing pinning to be structural for field… only needs to be amended such that "inner field" is substituted for "field" (e.g., item 1 requires Bar to not implement Unpin when Foo's field is ?Unpin).
Edit
The primary motivation behind this question is comprehension. Most (all?) of the time pin-project would be enough. I'm trying to understand if the rules mentioned in the pin module only apply to fields directly contained in the type or can they be generalized to any field in the object graph. If they can't be generalized, why?
There is nothing special about Foo either. If Bar is unable to project to Foo's T field, then that would seem to mean that even if Bar contained a primitive type that doesn't explicitly provide a pin projection method, it would be unable to project to the contained data. For example if Bar contained (T, T2), tuples don't provide pin projection methods; thus Bar would seemingly not be able to construct a Pin<&mut T>. I find that hard to believe.
Can this be guaranteed though, if Foo doesn't provide explicit structural pinning itself? And if it does provide, well, I think this together with pin_project on Bar should be enough?
If Bar doesn't know for sure Foo doesn't implement Unpin, then it wouldn't have a choice but to add PhantomPinned to be safe. Are you saying that even with PhantomPinned, Bar::poll is not safe?
I don't quite understand your use case here. if Bar needs to be !Unpin, just add PhantomPinned, why do you worry about Foo?
just to clarity, ?Unpin is different than !Unpin.
the bound ?Unpin will accept any type, if it is Unpin or not, while PhantomPinned is explicit !Unpin, which let you intentionally opt-out the compiler derived Unpin (auto-)trait.
My "use case" is purely about comprehension. I'm trying to understand how pinning works and what is required. I use ?Unpin because I would want Bar to work for all types (i.e., types that are Unpin and !Unpin). For types that are Unpin, Bar doesn't even need to rely on unsafe code; so it is only interesting when the types are !Unpin.
!Unpin is mostly useful when your type contains some form of self references, the synthesized Future types for desugared async functions are the main use cases. for "normal" types, you don't need to worry about Unpin at all, and that's why it's an autotrait.
even when you need !Unpin, PhantomPinned itself cannot prevent your type from being moved. what !Unpin really does is that, when your type is behind a Pin<Ptr>, you cannot get a mut reference back, that's it.
You can only soundly construct pifFoo offers structural pinning for its .0 field. If Foo does not, you have no guarantee it won't move f.0.0, thus violating the pinning you are promising here. “Foo doesn't implement Unpin for T: ?Unpin” is a necessary but not sufficient condition for Foo to structurally pin the .0 field; it also has to not have any operations that would move out of the .0 field.
I think that ideally, you (as the author of Bar) would use unsafe only to project into Bar's .0 field, and then use an accessor method defined by Foo to project into Foo's field, rather than doing both in one unsafe step. But I’m not greatly familiar with the idioms of actual pinning code, just the theory, so I don't know whether examples or counterexamples to this exist.
Interesting. So you are saying that if Foo doesn't explicitly define a projection method (i.e., a function that returns Pin<&mut T>), external code can never (safely/soundly) do this projection themselves? What if there is no ability to "move" the contained T using Foo's public API? Is that OK, or would you still have to worry about Foo's internal API?
I would really like a counterexample that "proves" this. I'm trying to understand pinning, but clearly there is something I'm missing. I don't feel bad that it's confusing seeing posts like this from well-respected members. @Yandros's response was especially comforting:
Structural Pin-ning, or more precisely, the lack thereof, just adds up to the confusion. For instance, even if it is intuitive that Pin<P<Box<T>> does not pin T, this pattern becomes less clear for Pin<P<(T, U)>: "if (T, U) is pinned, then surely both T and U should also be pinned" corresponds to our human intuition (at least it did for me).
The best thing to fight against an intuition (again, at least for me) is a counterexample. I, for instance, appreciated the exploit_ref_cell example.
I really relate to both of those paragraphs because they applied to me.
Well, it could also just be documented that a public field is to be treated as structurally pinned. I wouldn’t recommend that since the implications aren’t obvious and so it would be difficult to use correctly, but it would be sound to do so, I think.
You can always treat it as if the T can't be moved. It's just not very robust to do so.
IMO, one of the big reasons Pin is hard to understand is that it doesn't do anything itself: it just conveys a promise from one piece of unsafe code (owner of a value) to another (address-sensitive value), and just like any other unsafe contract, it’s possible to write a program that works because it happens to not violate the contract, which is not the same thing as a program that is robust under maintenance.
Here is one part of the module that is perhaps causing confusion for me:
If need to truly pin a value of a foreign or built-in type that implements Unpin, you’ll need to create your own wrapper type around the Unpin type you want to pin and then opts-out of Unpin using PhantomPinned.
To me this seems as though one can "override" a type's pinning guarantees so long as you are careful to follow the 4 requirements mentioned. So even though a type implements Unpin (i.e., pinning properties don't apply to it), an external type can provide those pinning properties so long as it is careful. I thought my Bar example was doing just that especially if it contains PhantomPinned thus opting out of being Unpin in the case Foo implements Unpin unconditionally.
This. Consider the "Structural Notice of Destruction" example, for instance; one generally can't assume Foo's destructor was written a certain way. And even if it's okay today, Foo's owner could change their implementation in some way that violates your unsafe assumption tomorrow. (Without using unsafe of their own, even.)
This should apply to any type though; thus if Bar contained Result<T, E> instead of Foo<T>; it would not be allowed to project to T since Result doesn't offer a projection method, correct?
As a different question, why should a type not implement Unpin unconditionally when it doesn't rely on pin projection?
You may also in this situation impl Unpin for Struct {}even if the type of field is not Unpin. Since we have explicitly chosen not to care about pinning guarantees for field, the way field’s type interacts with pinning is no longer relevant in the context of its use in Struct.
What are the ramifications if instead of Result only implementing Unpin when T and E do, it always implemented Unpin? Downstream code seemingly cannot perform pin projection on the contained data anyway for the same reason Bar cannot project to Foo's T field; and even if it could, unsafe code would still be needed to get a Pin<&mut T> (when T doesn't implement Unpin).
Not if you made the Result structually pinned (as you have in the OP), is how I understand it.
Let's say Bar contained a Result<T, E>. You've provided a safe method to get from Pin<&mut Bar> to Pin<&mut Result<T, E>>. It's up to Result<T, E> to decide if there's a safe way to get from there to Pin<&mut T>... or to &mut T. If they decide on the latter, and you also provided the safe projection from Pin<&mut Bar> to Pin<&mut T>, one could (without unsafe) create the Pin<&mut T>, then the Pin<&mut Result<T, E>>, then the &mut T, and then swap out the T without calling drop.
What if you hadn't provided a way to get a Pin<&mut Result<T, E>>? Then, as I understand it, you could provide the Pin<&mut T> projection (provided you meet all the other requirements).
So if I didn't use pin_project for Bar—which I only did to show that it doesn't help—then Bar::poll is sound so long as Bar doesn't expose Foo or the T in Foo in a way that would allow T to be moved? That is what I thought; however your original gripe was Foo's internal API especially how it implements Drop. I don't think that matters as long as Foo and T are not exposed by Bar, correct?
I just find it hard to believe that if Bar were Bar<T>((T)) (i.e., it contained a 1-ary tuple instead of Foo) that it wouldn't be able to pin project T even when it doesn't implement Unpin when T doesn't, and it doesn't expose the tuple or T in a way that would allow one to move T. I think items 2 and 3 in the pin module discussing what it necessary for a type to pin project to one of its fields doesn't really apply to Bar since it's not an "intrusive" data structure.
This would suggest that the description in the pin module applies to any of the data that is contained by the type and not just the fields the type directly contains. This is the heart of my question.
Assuming Bar<T>((T, )); to match the 1-tuple description
If you don't expose the tuple, at all, nor T in a way that would allow one to move T off &pin mut Bar<T>, then offering a pinning projection to the inner T ought to be fine, if you make sure not to be Unpin when T isn't (e.g., by adding a PhantomPinned):
you know that the T memory is contained within that of (T, ), inline;
you know that T is dropped when Bar<T> is (since (T, )'s drop glue does drop its T)
T : !Unpin => Bar<T> : !Unpin thanks to the assumed added PhantomPinned.
So there is no way to break the drop guarantee of Pin with your projection.
However, if you were to expose the (T, ) field in some fashion from a Pin<&mut Bar<T>> (e.g., a &pin mut Bar<T> -> &pin mut (T, ) "pinning" projection), then you'd be running the risk of its owner (here, the stdlib), adding new APIs or trait impls which would allow moving the T from there (e.g., they could make (T, )alwaysUnpin, allowing for &pin mut (T, ) -> &mut (T, ) which in turn exposes &mut T).