Is it possible to three-way split a `RefMut`?

Hello,

I don't have any use cases for this, but it just occurred to me (while implementing a library with similar functionality) that it's possibly impossible to split a std::cell::RefMut into three.

RefMut uses the RefMut::map_split associated function to split the borrow into two.

struct Foo{
    a: i32,
    b: i32,
    c: i32,
}
let cell = RefCell::new(Foo{ a: 1, b: 2, c: 3 });
let mut r = cell.borrow_mut();

let (ra, rb) = RefMut::map_split(r, |r| (&mut r.a, &mut r.b));

I always assumed that splitting into three would just involve calling map_split twice. But this is not possible, because it would require me to create a reference to a partial Foo, which only contains the fields b and c:

struct Foo{
    a: i32,
    b: i32,
    c: i32,
}
let cell = RefCell::new(Foo{ a: 1, b: 2, c: 3 });
let mut r = cell.borrow_mut();

let (ra, rbc) = RefMut::map_split(r, |r| (&mut r.a, &mut (r.b, r.c)));
//                                                  ^^^^^^^^^^^^^^^
// Of course this does not work, but I can't write `(&mut r.b, &mut r.c)` either!

let (rb, rc) = RefMut::map_split(rbc, |rbc| (&mut rbc.0, &mut rbc.1));

Do any workarounds exist for this, or is this a known limitation?

1 Like

I don’t see any way to do this either. Of course, data like a slice can be split easily in two steps, but the general case not so much.

At least, it should be possible to write this functionality in terms of map_split using unsafe code. Something like:

fn map_split3<T: ?Sized, U: ?Sized, V: ?Sized, W: ?Sized>(
    x: RefMut<'_, T>,
    f: impl FnOnce(&mut T) -> (&mut U, &mut V, &mut W),
) -> (RefMut<'_, U>, RefMut<'_, V>, RefMut<'_, W>) {
    let mut r2_ptr: Option<*mut V> = None;
    let mut r3_ptr: Option<*mut W> = None;
    let (ref1, ref23_stub) = RefMut::map_split(x, |r| {
        let (r1, r2, r3) = f(r);
        r2_ptr = Some(r2);
        r3_ptr = Some(r3);
        (r1, &mut [(); 0])
    });
    let (ref2, ref3) = RefMut::map_split(ref23_stub, |_| {
        let r2_ptr = r2_ptr.take().unwrap();
        let r3_ptr = r3_ptr.take().unwrap();
        unsafe {
            (&mut *r2_ptr, &mut *r3_ptr)
        }
    });
    (ref1, ref2, ref3)
}
3 Likes

I don't think there is for RefCell and RefMut, but there's a crate called partial-borrow which provide a general solution for this kind of problem. you can use the partial macro to specify the permission (mut, const, or none) for each field.

example:

#[derive(PartialBorrow, Default)]
pub struct Foo {
    a: i32,
    b: i32,
    c: i32,
}

let mut foo = Foo::default();

let (partial_a, remaining): (&mut partial!(Foo mut a, ! *), _) = Foo::split_off_mut(&mut foo);
let (partial_b, remaining): (&mut partial!(Foo mut b, ! *), _) = Foo::split_off_mut(&mut remaining);
let (partial_c, _): (&mut partial!(Foo mut c, ! *), _) = Foo::split_off_mut(&mut remaining);

// note the deref operator below
// it's not needed when calling methods though, because of auto deref coercion 
*partial.a = 42; // good
*partial.b = 42; // compile error

the interesting traits are SplitInto and SplitOff. it is slightly different from RefMut::map_split() though, the return value is a procedurally generated proxy type with the specified access right for the fields, not a direct reference to the fields, but it does implement Deref or DerefMut depending on the partial borrow type.

2 Likes

Ok, then. I was looking for this because I'm working on a crate for generic mutability, and I was trying to implement split for my reference type. I wanted to take a look at how std solved this, but it seems like it just didn't.

I could be lazy and do the same (if std got away with it, I should be able to as well), but what I'm probably going to end up doing anyway is macro-implementing a trait for tuples of all sizes up to 32 or something, and hope that no one is trying to split their borrow 33-way.

If anyone has a better idea for solving this, please let me know.

1 Like

Although I don't think it will help with what I'm trying to solve, it looks very interesting and I will definitely look into it sometime in the future!

Your plan is probably the right one, but it is possible to extend @steffahn's approach via an HList approach to cover arbitrarily-way splits:

#[derive(Debug)]
struct Example {
    a: String,
    b: Vec<u32>,
    c: usize,
}

impl Example {
    fn split_refmut(
        this: RefMut<'_, Self>,
    ) -> (RefMut<'_, String>, RefMut<'_, Vec<u32>>, RefMut<'_, usize>) {
        let split = SplitProxy::new(this, |this, fields| {
            fields
                .append(&mut this.a)
                .append(&mut this.b)
                .append(&mut this.c)
        });
        let (a, split) = split.pop();
        let (b, split) = split.pop();
        let (c, _) = split.pop();
        (a, b, c)
    }
}

fn main() {
    let x = RefCell::new(Example {
        a: String::from("Hello"),
        b: vec![],
        c: 42
    });
    
    {
        let (mut a,mut b,mut c) = Example::split_refmut(x.borrow_mut());
        a.make_ascii_uppercase();
        *b = (0..10).collect();
        *c /= 2;
    }
    
    dbg!(x.borrow());
}

Implementation (Playground)

I feel like custom implementations of HEnqueue could implement enqueue unsoundly here. This probably needs some sealed traits.


On second thought, let me double check whether or not there’s actually any issue. Edit: Nevermind, it seems to be tight by handing out only the FieldList<'b, ()> starting point.

In any case, this does have the downside of being significantly more complex to use than a simple macro-implemented approach supporting a lot of different tuple lengths.

2 Likes

Now I'm attempting to implement the HList version, and on the implementation side it seems to be a bit simpler than the macro version (which I had already got working)...

What I'm more concerned about is ease of use, especially that the majority of splits will be two-way, and that is spelled longer and less readably with the HList version:

(&mut a, &mut b)
//vs
(&mut a, (&mut b, ()))

I guess I could also go with the tuple version for the 2-case (and maybe the 3-case), but also allow HLists if they use a special tuple struct:

(&mut a, &mut b)
//or
HList(&mut a, HList(&mut b, ()))

...but that might be overly complicated, given that this is not the main feature of the crate.

This can be mitigated a little bit with an appropriate helper macro, but it'll never be as clean as direct implementations for tuples. Also, syntax design is hard. Here's my first attempt:

macro_rules! split_ref_mut {
    (|$arg:ident| {
        $(let $f:ident = $val:expr;)*
    }($argval:expr)) => {
        let split = SplitProxy::new($argval, |$arg, fields| {
            fields $(
                .append($val)
            )*
        });
        $(
            let ($f, split) = split.pop();
        )*
        ::std::mem::drop(split);
    }
}

// ===================

impl Example {
    fn split_refmut(
        this: RefMut<'_, Self>,
    ) -> (RefMut<'_, String>, RefMut<'_, Vec<u32>>, RefMut<'_, usize>) {
        split_ref_mut!{|this| {
            let a = &mut this.a;
            let b = &mut this.b;
            let c = &mut this.c;
        }(this)};
        (a, b, c)
    }
}
1 Like

Regarding direct implementation with tuples as a baseline of ergonomics, I think even a single function that just handles the 2 cases of (&mut T) -> (&mut U, &mut V) and (&mut T) -> (&mut U, &mut V, &mut W), without needing to be separate functions map_split and map_split3, and without requiring extra type annotations from the caller, might be nontrivial. (I haven’t tried too hard yet.)

It works if I create a private trait that handles each case. (Just like the macro version, but without the macros)

Fortunately, I can access the internals of my type, so I don't have to hack around forwarding to another split implementation. Btw, all I'm doing in my split is: materialize a NonNull as &/&mut -> pass to function -> convert returned &/&muts to NonNulls (and wrap them in my struct).

Yeah; that's part of why I ended up with a sort of builder pattern. When I tried to use a tuple/hlist of &muts as the function output, I needed a weird HRTB that I don't know how to express; something like

where for<'a>
    F: FnOnce(&'a mut T)->_,
    F::Output: TupleOfMutsTrait<'a>

No, the macro version uses this signature and works fine:

fn split<'a, M: Mutability, T, U, UM, UIM, FM, FIM>(ptr: GenRef<'a, M, T>, fn_mut: FM, fn_immut: FIM) -> U
    where 
        T: 'a + ?Sized,
        UM: TupleMutIntoGenRef<'a, M, Output = U>,
        UIM: TupleImmutIntoGenRef<'a, M, Output = U>,
        FM:  FnOnce(&'a mut T) -> UM,
        FIM: FnOnce(&'a T) -> UIM;

(yeah, I have to create two such traits. actually, it's three.)

That signature looks suspicious to me— RefMut::map_split uses an HRTB to decouple the self lifetime from the lifetimes that are available in the closure. I can't think of a way to use this difference to break something ATM, but they presumably did that for a reason.

Hmm... well, I use a lot of unsafe in this project, so it might be possible that I screwed up the lifetimes without noticing...

Where? I can't see it...?

pub fn map_split<U: ?Sized, V: ?Sized, F>(
    mut orig: RefMut<'b, T>,
    f: F,
) -> (RefMut<'b, U>, RefMut<'b, V>)
where
    F: FnOnce(&mut T) -> (&mut U, &mut V)

This is syntax sugar for F: for<'a> FnOnce(&'a mut T) -> (&'a mut U, &'a mut V)

2 Likes

Oh, I see. It was too elided for me to notice.

I think I see the problem now. 'a originally comes from the &'a RefCell<...> that was borrow_mut()ed. If you let the closure see &'a mut Something, it can smuggle that out through a capture and retain an &'a mut after the last RefMut<'a> is dropped, ending the runtime noalias protections.

Because the original borrow was a shared reference, the borrow checker will then allow another borrow_mut() call on the same RefCell. The cell's refcount is zero because all of the guards have been dropped, so the runtime mechanism also lets the call go through. This creates a second, concurrent &mut to the same place which is UB.

1 Like

Oh, ok. Does that mean I'm safe to use my version, because I'm not dealing with any sort of interior mutability?