Independent borrows of fields behind a Deref

help
1

You can't borrow multiple fields at once if they belong to a type that is behind a DerefMut implementation:

pub struct Person {
    name: String,
    age: i32,
}

pub struct Wrapper<'a>(&'a mut Person);

impl<'a> Deref for Wrapper<'a> {
    type Target = Person;

    fn deref(&self) -> &Self::Target {
        self.0
    }
}

impl<'a> DerefMut for Wrapper<'a> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        self.0
    }
}

pub fn test() {
    let mut person = Person {
        name: "John Smith".to_owned(),
        age: 42,
    };
    let mut wrapper = Wrapper(&mut person);
    let name = &mut wrapper.name;
    let age = &mut wrapper.age; // Fails!
    println!("{name}");
}

Which makes sense, since deref_mut takes a mutable reference to self, and you can't have two of those at the same time. I came up with an idea to allow something of the sort anyway (explanations below):

pub struct Person {
    name: String,
    age: i32,
}

mod wrapper {
    use super::*;

    pub struct Name(());

    pub struct Age(());

    #[repr(C)]
    pub struct Wrapper<'a> {
        pub name: Name,
        pub age: Age,
        inner: *mut Person,
        _marker: PhantomData<&'a Person>,
    }

    impl<'a> Wrapper<'a> {
        pub fn new(person: &'a mut Person) -> Wrapper<'a> {
            Self {
                name: Name(()),
                age: Age(()),
                inner: person as *mut _,
                _marker: PhantomData,
            }
        }
    }

    macro_rules! get {
        ($this:ident.$field:ident) => {{
            let person_ptr = *($this as *const Self as *const Wrapper as *const *mut Person);
            &raw mut (*person_ptr).$field
        }};
    }

    impl Deref for Name {
        type Target = String;

        fn deref(&self) -> &Self::Target {
            unsafe { &*get!(self.name) }
        }
    }

    impl DerefMut for Name {
        fn deref_mut(&mut self) -> &mut Self::Target {
            unsafe { &mut *get!(self.name) }
        }
    }

    impl Deref for Age {
        type Target = i32;

        fn deref(&self) -> &Self::Target {
            unsafe { &*get!(self.age) }
        }
    }

    impl DerefMut for Age {
        fn deref_mut(&mut self) -> &mut Self::Target {
            unsafe { &mut *get!(self.age) }
        }
    }
}

The idea is that thanks to the #[repr(C)] slapped on Wrapper and to Name and Age being ZSTs, pointers to a Wrapper's name or age fields should share the same value as pointers to the Wrapper itself. And since Wrapper is just, well, a wrapper over a pointer to Person, we can access that too.

So, cast the reference to Name to a pointer to Wrapper (can be skipped), and then cast that to a pointer to a pointer to a Person. From there, get a pointer to the relevant field. This is all that is done in the various Derefs implementations here.

As far as I can tell, this is safe, and by my own tests, it does work 100% of the time. The issue is, Miri tells me there is some UB going on when I dereference the pointer to the pointer to a Person. I don't really know why, although I had some doubts about the value of references to ZSTs. AFAIK, the compiler could decide tomorrow to not give me the actual pointer to Wrapper anymore in Name::deref_mut and instead give me a dangling (but well-aligned) pointer, as references to ZSTs are fake and all. But it works in practice, so maybe there is some guarantees I don't know about or that aren't documented?

Anyway, here's my question: is this actually safe or is this UB?

2

You’re trying to take an &Name, pointing to an () value, and access memory that lies past the end of that value. Whether that should be allowed (regardless of whether the value has a size of zero or not) is a matter of debate, and Miri does not allow it by default.

The safe, general solution to DerefMut conflicts is to reborrow early:

    let person = &mut *wrapper;
    let name = &mut person.name;
    let age = &mut person.age;
4 Likes
3

Or perhaps even:

    let Person { name, age } = &mut *wrapper;
1 Like
4

Thank you to both. I know my example is contrived, but what if you're not actually interested in giving access to the wrapped struct's fields, but in managing access to data? Perhaps the name is behind a pointer itself, and the age is nested deep in other structs? Then reborrows aren't available. You can always make accessors methods as_*, but then again the user can't call multiple of these at the same time. My "solution" allows providing the user a nice zero-cost API to solve this issue. I know it's probably overkill.

5

Well, your pitch was to "fix" DerefMut, but DerefMut is the wrong tool for managing access. I see what you're getting at though, it's just a different use case than was presented.

You might be interested in this discussion. It's a nuanced topic; for your code, there never need exist a reference to all of Wrapper from which provenance over the pointer value (the "right" to read the inner field) can be determined.

Edit: The below is unsound.

Here's something I came up with. It's not as ergonomic and I don't know if I covered everything.

1 Like
6

I didn't, it's unsound.

7

Along the same lines as my last post, here's some UB, regardless of the header question.

    #[repr(C)] struct SideBySide<T, U>(T, U);
    let mut p2 = Person { name: "hi".into(), age: 13 };
    let w2 = Wrapper::new(&mut p2);
    let mut sbs = SideBySide(w2.name, wrapper);
    let n1 = &mut *sbs.1.name;
    let n2 = &mut *sbs.0;
    println!("{n1} {n2}");

I didn't really need SideBySide, I just had it handy. The real problem is that you can move the Name anywhere and have it treat the following bytes as a *mut Person.

2 Likes
8

In other words, Name is address-sensitive; it must not be moved. Therefore, it cannot be exposed by value or mutable place. There are two ways it could be exposed:

  • By &Name reference. In this case, that would defeat the point because it prevents mutation.
  • By Pin<&mut Name> reference. This will not (yet) be more ergonomic than not doing anything special with Wrapper.
9

That's a really good catch. I came up with this "fix":

pub struct Person {
    name: String,
    age: i32,
}

mod wrapper {
    use super::*;

    pub struct Name(());

    pub struct Age(());

    #[repr(C)]
    pub struct Wrapper {
        pub name: Name,
        pub age: Age,
        inner: Person,
    }

    impl Wrapper {
        pub fn new(person: &mut Person) -> &mut Wrapper {
            unsafe { transmute(person) }
        }
    }

    macro_rules! get {
        ($this:ident.$field:ident) => {{
            let person_ptr = $this as *const Self as *mut Person;
            &raw mut (*person_ptr).$field
        }};
    }

    impl Deref for Name {
        type Target = String;

        fn deref(&self) -> &Self::Target {
            unsafe { &*get!(self.name) }
        }
    }

    impl DerefMut for Name {
        fn deref_mut(&mut self) -> &mut Self::Target {
            unsafe { &mut *get!(self.name) }
        }
    }

    impl Deref for Age {
        type Target = i32;

        fn deref(&self) -> &Self::Target {
            unsafe { &*get!(self.age) }
        }
    }

    impl DerefMut for Age {
        fn deref_mut(&mut self) -> &mut Self::Target {
            unsafe { &mut *get!(self.age) }
        }
    }
}

Now, the user can't ever move Name out of any Wrapper as it is always behind a reference. Sure, they can still swap it between two Wrappers but that changes nothing. Miri still complains though...

Anyway, thank you both for your time.

10

maybe the problem is that DerefMut is not the right tool to use.

DerefMut is a one-to-one projection, and since it is special to the language for the deref coersion, it needs to be like this.

what we want is a one-to-many projection, a.k.a. "view" types:

I don't think view types are coming in the near future though, but at least we can emulate the behavior in library to a certain extent, e.g. the partial-borrow crate allows you to do something like this:

	#[derive(PartialBorrow)]
	struct Person {
		name: String,
		age: i32,
	}

	// a "view" that borrows all fields as mutably
	type Wrapper = partial!(Person mut *);

	let mut person = Person {
		name: String::from("Jack"),
		age: 42,
	};

	// derived `PartialBorrow` includes `AsRef` and `AsMut` implementations
	let wrapper: &mut Wrapper = person.as_mut();
	// fields of `wrapper` are actually proxy types that implement `DerefMut`
	let name: &mut String = &mut wrapper.name;
	let age: &mut i32 = &mut wrapper.age;
1 Like
11

It is considered sound to move out of a &mut _ so long as you replace the value by the time the borrow expires,[1] so I'm afraid that doesn't actually remove the unsoundness.

  1. in that issue, I link to an RFC that has discussion about the non-'static case, which is what replace_with relies on ↩︎

12

It would be interesting to see if or how they addressed the above unsoundness issues (I haven't checked). But I think they're in the same boat as far as provenance goes.

Edit: They admit they have the same problem. There still hasn't been an FCP, but given that Ralf and Niko consider the replace_with pattern sound, I'd say that the partial-borrow crate is the unsound one.

Edit 2: That crate in particular actually came up in the linked RFC thread. This thread is in some sense a rehash of the conversation there.

2 Likes
13

Ah, that's unfortunate. It seems to me that replace_with would be best served by an enhancement of the borrow checker than by a library function, but oh, well. It feels a bit hard having to reason about the possibility of moving out of mutable references.

Thanks for the links.

14

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.