Pin<Box<T>> Auto-dereference pinned container

TL;DR:

Is it possible to abstract away the .as_mut() call for Pin<Box<T>> types?

let mut container: Pin<Box<Container>> = Container::new("A");
container.as_mut().set_name("B".to_owned());

Why? I am trying to create an ergonomic API for a tree-like data structure. I am still a beginner with Rust, so I try to explain my reasoning as well. Please watch for misconceptions along the way.

Considering this simplified "pinned container" which has a collection of nodes, each with a pointer to the parent container.

struct Node {
    name: String,
    parent: *mut Container,
}

struct Container {
    name: String,
    nodes: Vec<Pin<Box<Node>>>,
    pinned: PhantomPinned,
}

(Note that the nodes themselves are pinned as well, as eventually they will have children with parent references as well, but I have removed them for simplicity.)

The container itself cannot be moved, as the children nodes are effectively self-referential. So, when the "user" creates a new container they are actually given a pinned version on the heap, i.e. a value of type Pin<Box<Container>>.

As a result, you cannot implement methods that mutate the container with the usual &mut self parameter:

    fn set_name(&mut self, name: String) {
        self.name = name;
    }
error[E0596]: cannot borrow data in a dereference of `Pin<Box<Container>>` as mutable
   --> src/main.rs:104:5
    |
104 |     container.set_name("A".to_owned());
    |     ^^^^^^^^^ cannot borrow as mutable
    |
    = help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Pin<Box<Container>>`

Instead, you can implement methods with a self parameter such as self: Pin<&mut Self>:

    fn set_name(self: Pin<&mut Self>, name: String) {
        let container = unsafe { self.get_unchecked_mut() };
        container.name = name;
    }

However, the Pin<Box<Container>> does not auto-dereference to a Pin<&mut Container>, you have to explicitly call .as_mut():

Compiling playground v0.0.1 (/playground)
error[E0599]: no method named `set_name` found for struct `Pin<Box<Container>>` in the current scope
  --> src/main.rs:99:15
   |
99 |     container.set_name("A");
   |               ^^^^^^^^ method not found in `Pin<Box<Container>>`

Having to explicitly call .as_mut() on the values is of course not great for the consumers of the API.

Is it possible to make the Pin<Box<Container>> and down the tree (Pin<Box<Node>> ... ) automatically dereference to Pin<&mut T> without an explict .as_mut() or otherwise abstract it away for the API users?


For a complete code listing see:

I don't think .as_mut() is your problem here, ultimately. At the end of the day, you can't access a Container simultaneously via a reference (like self) and via the parent pointer; there are multiple other discussions about this in this forum and on github. Whenever you access it via the reference, the compiler will disregard updates done via the pointer because of its uniqueness assumption. And once you develop your code further, this is almost impossible to avoid.

Putting the object in a Pin<Box<>> may mask that, but doesn't change that. In particular, the Pin part doesn't help at all (contrary to intuition), and the Box<> may currently mask it. Even if you get code that passes MIRI in its current version, once you turn track-raw-pointers on in Miri, it'll be flagged.

In practice, until Rust explicitly introduces a primitive that informs the compiler that all or part of a referenced object is also aliased via a raw pointer, I think this means that writing any code that uses both references and raw pointers is a fool's errand. (Example). This is a bit of a sad state of affairs for it means that code that necessitates raw pointers will ultimately be mostly if not exclusively C style.

2 Likes

If your goal is writing a tree datastructure, why involve Pin at all? Have a look at the atree crate.

Thanks for the suggestion @jjpe. I have actually had the most success with a generational-arena crate for this particular API, but ultimately I had to introduce runtime borrow checking, interior mutability, and a lot of indirection.

I was investigating other options, and the possibility of using raw pointers to the effect of an "intrusive" data structure seemed like an option.

Thank you for the references @godmar, there is a lot to digest there and sadly none of it seems to extol the utility of Pin with respect to raw pointers.

Again, I am still bit of a foundling with Rust so please forgive me but I am not sure I understand your assessment.

I thought that because the children are owned by the Container, any mutable access to the parent pointer would require the Container already be mutably borrowed, which would preclude simultaneous access.

Overall, I am absolutely ready to accept your assessment. I am out of my depth. It just seems like, because the Container itself is Pin<Box<_>>, and the pointers are truly self referential this is a different problem than the intrusive doubly-linked list and Pair example. I don't see those as self-referential at all.

Pin is generally only necessary for types that are visible in your public api for which it is the user of the library that needs to ensure that they are pinned. For internal types, you can just use raw pointers and not move it.

Pin isn't special in any way, it's just a container that denies (safe) mutable access to the inner pointer if the pointee isn't Unpin

so there is no significant difference between

struct A<'a, T>
// If we add `where T: Unpin` this `Pin` doesn't do anything
{
    ptr: Pin<&'a mut T>,
}

and

struct A<'a, T> {
    ptr: &'a  mut T,
}

assuming the field ptr cannot be accessed outside this module

I think that I need the user to ensure the types are pinned to prevent incoherent "safe" operations, such as std::mem::swap on nodes in different containers, which would not update the parent pointer.

Yes, but only if you expose to the user the references that may not be swapped.

Ah, I see what you mean. All it takes is one more layer of composition, and you can prevent those references from ever being touched by the user.

struct Thing {
    container: Pin<Box<Container>>,
}

Then delegate with the methods. Trivial.

Do you think that Pin has any value internally for API developers; to be mindful the values shouldn't be moved, or would you personally just use plain Box<T>?

Well I would probably not use Box<T> because those are not allowed to alias. I would use raw pointers, or maybe raw pointers wrapped in a struct.

Just because it's mutably borrowed doesn't mean it'll respect simultaneous mutation via a raw pointer.

The question, ultimately, is whether your code (once complete) has a path where an access via the raw pointer is made while the struct being accessed is also mutably borrowed. If so, it's basically UB (and in practice results in code where the compiler won't expect updates via the latter path.)

With Rust's lack of an ABI, the compiler can do whatever it wants. In particular, (I think that) there's no guarantee that, for instance, m(&mut self) even compiles into something that passes a pointer in the ABI.

For example, let's say if you have a struct that's mutably borrowed and then immutably borrowed to invoke a mi(&self) method. Despite you're having written &self the method won't actually receive a pointer to the object, it'll receive a copy of the state of the object at time of invocation.

Here's a godbolt. For

pub struct P {
    x: i32,
    y: i32,
    z: i32,
}

the method:

    #[inline(never)]
    fn get(&self) -> i32 {
        self.x + self.y + self.z
    }

is compiled to:

example::P::get:
        leal    (%rdi,%rsi), %eax
        addl    %edx, %eax
        retq  

in other words, the fields x, y, and z are passed in registers %rdi, %rsi, and %rdx rather than by passing a pointer to the location in memory where the target struct is located. This occurs despite the object having been mutably borrowed earlier.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.