Is Rc a good way to work around partial borrow of self?

I apologize for the long-winded question, but this is an issues that has come up often on the forum and I am curious if my workaround is sound or if I am doing something I should not be.

Sometimes you have a complicated structure with multiple methods that take a reference to self. Often you will have some public outer method that calls one or more private inner methods to mutate self, often in the context of a loop. For example:

impl MyStruct {
    pub fn outer_method(&mut self) {
        for thing in &self.things {
            self.inner(thing);
        }
    }
    fn inner_method(&mut self, thing: Thing) {
        // ...
    }
}

error[E0502]: cannot borrow *self as mutable because it is also borrowed as immutable

Please see the Rust Playground for a complete, minimal example.

The Rust borrow checker understands that we can have multiple borrows to disjoint fields in a structure, so we could have safely mutated self directly inside the for loop, for example:

    pub fn outer_method(&mut self) {
        for thing in &self.things {
            self.bar = thing + self.foo;
        }
    }

The problem arises when we call self.inner_method(), which borrows self mutably. There is no way for the borrow checker to know whether self.inner_method() is going to try and borrow self.things again or not. But if our inner method does something long and complicated, then we would really like to split that code out into a separate method to make the outer method more readable!

RFC 1215 proposes a partial borrow syntax that could solve this, but insofar as it's been around since 2015 I'm not holding by breath.

There are multiple other forum posts asking about this problem:

I am wondering what the community things about using Rc as a general-purpose workaround in these situations.

struct MyStruct {
    things: Rc<Vec<Thing>>,
    // ...
}
impl MyStruct {
    pub fn outer_method(&mut self) {
        let things = self.things.clone();
        for thing in self.things.iter() {
            self.inner(thing);
        }
    }
    fn inner_method(&mut self, thing: Thing) {
        // ...
    }
}

See the complete example on Rust Playground. To me this seems ergonomic, requires little modification to the structure or method design, and has very little performance penalty as the Rc is reasonably lightweight and only the Rc (not the entire vector) is cloned.

Can the community think of any pitfalls I am not seeing or suggest a more idiomatic alternative?

What about this?

pub fn outer_method(&mut self) {
    let vec = std::mem::take(&mut self.things);
    for thing in &vec {
        self.inner(thing);
    }
    self.things = vec;
}

I have done that before as well. My understanding is that idiomatically std::mem::take() assumes ownership, which is fine when you really intend to take the object out of the structure. The problem arises when your intention is merely to borrow the object. If you use take instead of borrowing than you must remember to put the object back when you are finished with it!

I agree take is a reasonable alternative to Rc, but can you think of any obvious advantages over Rc?

Well there are a few:

  1. With Rc your type can no longer be send across threads. If you're writing a library, this would be annoying for your users.
  2. It makes the vector immutable. This is sometimes fine.

And if you think that the mismatch of intentions compared to borrowing the object is a problem, then there's also a mismatch of intentions here — Rcs are normally used in very different situations.

Alternatively, why does inner_method need a reference to all of self? Can you pass the fields it actually needs individually?

In the unlikely case inner_method really needs to operate on self.things as well as thing, then you probably do need something like Rc or Cell or whatever, and that’s fine.

My interpretation of all the threads I’ve ever seen on these proposals is that “the cures are no better than the diseases.” When you need partial self borrows, restructuring the code to not need it is always better if you can do it (in every language, not just Rust!), and if you can’t then you’re stuck with the awkward choice of either paying a runtime cost (like you’re doing, which is often totally fine) or making unusually subtle semver commitments (which would ironically become a footgun if we made it any easier!). So IMO none of the proposals appear to be a net win in practice, and it seems likely nothing could be.

(but it’s still very good that we had all those threads, because it was not at all obvious this would be the outcome; now we know to focus on other potentially solvable problems instead)

1 Like

alice, you are quite right that Rc cannot be passed across threads. I suppose that if I intend for my structure to be Send then I should use an Arc in the same way, although that would have even a little more overhead. I have always seen this as a bit of an annoyance with Rust -- wishing that that the author of another crate had used Arc. Edit: Rc vs Arc discussed in this post.

I actually tried your std::mem::take() solution first, but the possibility of introducing subtle errors by forgetting to put the taken object back drove me nuts! If you have any suggestions for a way to guard the take() so that the taken object is automatically put back at the end of a scope, or so that we get a panic if we forgot to put it back at the end of the scope, that would be amazing!

lxrec, I agree that in many of the examples cited the whole issue could be avoided by a small design change. I respectfully disagree and actually find it a little bit condescending when people tell me that partial borrows are always a symptom of poor design. The whole point of having an object oriented language is to encapsulate groups of variables with the same lifetime and purpose. The idea that one should always have to destructure objects runs directly contrary to this. This kind of code quickly gets to be unreadable by humans, even through it conveys to the compiler exactly what we are borrowing and when:

pub struct MyStruct {
    foo: Foo,
    bar: Bar,
    baz: Baz,
}
impl MyStruct {
    pub fn do_something(&mut self) {
        inner1(self.foo, self.bar);
        inner2(self.foo, self.baz);
        inner3(self.bar, self.baz);
    }
}
fn inner1(foo: &mut Foo, bar: &bar) { /* ... */ }
fn inner2(foo: &Foo, baz: &mut Baz) { /* ... */ }
fn inner3(bar: &mut Bar, baz: &mut Baz) { /* ... */ }

When my code is semantically operating on the the object as a whole, it seems to me that it would be much more readable to call a self.inner1() that takes self as &mut. I am looking for an idiomatic way to do this with the least runtime cost and the least possibility of unintended consequences.

My preference is to break the struct into multiple substructures and put the problematic mutable borrows on onto a separate substructure.

struct OtherStruct {
    ...
}
struct MyStruct {
    things: Vec<Thing>,
    other: OtherStruct,
}
impl OtherStruct {
    fn inner_method(&mut self, thing: Thing) {
        // ...
    }
}
impl MyStruct {
    pub fn outer_method(&mut self) {
        let things = self.things.clone();
        for thing in self.things.iter() {
            self.other.inner(thing);
        }
    }
}

This can feel awkward at first and requires identifying the subsets of fields that may need to be borrowed separately. But in the end it allows the borrow checker to work with you rather than against you, by helping you make the code better reflect what modifies what.

For the rare case where you cannot refactor your code into refactoring the struct with composition and sub-structs, and if forgetting to put the things back is an issue, you can define your own helper that does this for you:

struct MyStruct {
    things: Vec<Thing>,
    // ...
}

impl MyStruct {
    pub
    fn outer_method (self: &'_ mut Self)
    {
        self.with_things(|this, things| {
            for thing in things.iter() {
                this.inner_method(thing);
            }
        })
    }

    fn inner_method (self: &'_ mut Self, thing: &'_ Thing)
    {}
    
    fn with_things<R> (
        self: &'_ mut Self,
        f: impl FnOnce(&'_ mut Self, &'_ mut Vec<Thing>) -> R,
    ) -> R
    {
        let mut things = ::core::mem::replace(&mut self.things, vec![]);
        let ret = f(self, &mut things);
        self.things = things;
        ret
    }
}
1 Like

Thank you to everyone for their thoughtful comments! I like Yandros's solution building on alice's suggestion. Added an extra check that panics if the closure modifies self.things while it is taken on the Playground with a snippet below. The advantage over Rc appears to be thread-safety and marginally less overhead at the cost of having to write an extra method.

// private helped method
fn with_terms<R>(
    &mut self,
    f: impl FnOnce(&mut Self, &mut Vec<i32>) -> R
) -> R {
    let mut terms = std::mem::take(&mut self.terms);
    let ret = f(self, &mut terms);
    if self.terms != Vec::default() {
        panic!("self.terms was modified during call to with_terms()");
    }
    self.terms = terms;
    ret
}
// public-facing method
pub fn add(&mut self) {
    self.with_terms( |this, terms| {
        for term in terms.iter() {
            // call some other private method
            this.add_to_sum(*term);
        }
    })
}

Still curious if others have specific criticisms of the above or their own pet solution to this problem!

Looks good! I think the check looks a bit awkward ­— I'd probably do !self.terms.is_empty()

About Rc vs. Arc, you could just wrap the struct in a macro and have it accept a single argument, which would either be my_macro!(Rc) or my my_macro!(Arc). Then you just have to use the macro in your normal module and in a sync sub-module or something like that. Alternatively, you could let the macro consume a second argument which would be the struct name. Then you can have both versions with different names in the same module.

There's another way to take advantage of this, where you can create a "projection reference" struct, with all or a subset of the fields on the original struct, but with the specific mutability needed for the specific method being used.

To do the add_to_sum, the sum needs to be mutable, but the vec can be accessed with a shared reference:

/// Reference projection struct.
pub struct AdderProj<'a> {
    // Vector of numbers to add up, shared ref
    terms: &'a Vec<i32>,
    // Place to store the some of the numbers, mutable ref
    sum: &'a mut i32,
}

Then add_to_sum can be implemented using this projection rather than for a mutable reference to the entire struct:

impl<'a> AdderProj<'a> {
    // Add a single number to the sum.
    // This operates on the projection of adder.
    fn add_to_sum(&mut self, term: i32) {
        *self.sum += term;
    }
}

Then the loop can be written using the projection rather than the full mutable reference:

    /// Private function to project references, where the terms
    /// are immutable for iteration, but the sum is mutable
    fn as_proj(&mut self) -> AdderProj {
        AdderProj {
            terms: &self.terms,
            sum: &mut self.sum,
        }
    }
    
    /// Perform the addition.
    pub fn add(&mut self) {
        // Project self with mutability only for sum
        let mut proj = self.as_proj();
        // Iterate over numbers to sum.  This requires we borrow terms.
        for term in proj.terms {
            proj.add_to_sum(*term);
        }
    }

If the internal methods are not quite so internal, they can proxy the call onto the projection without duplicating the logic itself:

    // Add a single number to the sum.
    // This delegates to the projection of adder.
    pub fn add_to_sum(&mut self, term: i32) {
        self.as_proj().add_to_sum(term);
    }

I think that this is a pretty general way to work around a partial borrow, since the idea is to make the partial borrow explicit.

Full example playground

5 Likes

Douglas that is surely witchcraft and I had to read through the code several times for it to make sense! As I understand it, when we call self.as_proj() we borrow two disjoint fields from self at the same time: the vector terms immutably and the sum mutably.

The following doesn't work because we borrow self.terms immutably and then we try to borrow the entirety of self (including terms) mutably. It's illegal to borrow something (in this case terms) mutably while it is borrowed immutably.

fn add_to_sum(&mut self, term: i32) { /* ... */ }
for term in &self.terms {   // borrows self.terms immutably
    self.add_to_sum(*term); // tries to borrow all of self
}
// error

But the following works because we borrow terms immutably through the projection. Then, through the projection, we tell the borrow checker that we are going to borrow self.terms immutably again (which is legal) and then borrow only self.sum mutably.

let mut proj = self.as_proj();
for term in proj.terms {    // borrows terms immutably
    proj.add_to_sum(*term); // borrows terms immutably and sum mutably
}

If for didactic purposes we redefine the projection to make terms mutable the borrow checker will rightly complain that we are borrowing terms mutably while it is borrowed immutably:

pub struct AdderProj<'a> {
    terms: &'a mut Vec<i32>, // error, can't have mut
    sum: &'a mut i32,
}

I did not realize we could use reference structs to project in this way, but it is immensely clever. Aside from having to define (possibly more than one) projection struct (which can be hidden away in a private module), I don't see any disadvantages to this solution whatsover. There are no runtime checks, there is no allocation overhead, and there is no threading penalty.

1 Like

Disclaimer: ruminations of a total Rust neophyte

struct Adder {
    terms: Vec<i32>,
    sum: i32,
}

impl Adder {
    pub fn new(terms: Vec<i32>) -> Self {
        Self {
            terms,
            sum: 0,
        }
    }

    pub fn add(&mut self) {
        // localize mutability to this method alone.
        for term in self.terms.iter() {
            // inner methods calculate new values from immutable values
            let new_sum = self.add_term(term);

            //  which this method then uses to mutate the
            //  structure's value when it is good and ready
            self.sum = new_sum;
        }
    }

    // No `&mut self` needed - it simply calculates a new value
    // from the structure's immutable value
    fn add_term(&self, term: &i32) -> i32{
        self.sum + term
    }

    pub fn get_sum(&self) -> i32 {
        self.sum
    }
}

fn main() {
    let mut adder = Adder::new(vec![1, 2, 3]);
    adder.add();
    println!("The sum is 6: {}", adder.get_sum());
    adder.add();
    println!("The sum is 12: {}", adder.get_sum());
}

The whole point of having an object oriented language is to encapsulate groups of variables with the same lifetime and purpose.

Where did you get the idea that Rust is an object-oriented language?

The Rust Book only promises:

we’ll explore certain characteristics that are commonly considered object oriented and how those characteristics translate to idiomatic Rust.

Objects tend to be bags of mutable state (immutable objects are possible but not that common) - i.e. most OOP languages tend to wield mutability as a blunt instrument with sometimes undesired consequences.

The borrow checker aims to wield mutability (or more accurately exclusive access) as a much more deliberate, focused and precise measure.

Rust is an imperative language but it borrows heavily on ideas found in functional languages.

Rust seems a lot more value-oriented than the traditional place-oriented OOP languages.

Rust is primarily an expression language.

Expressions produce values - traditional imperative languages use statements to execute processing steps and guide flow of control.

Functional languages favor immutability so in practice existing values aren't mutated but instead new values with the desired changes are created.

In Rust producing new values doesn't require mutability and if the values are small enough, it shouldn't impose too much overhead to get them to where mutability is opportunistically available. So hypothetically value-oriented designs which use focused, controlled mutability as an optimization should be easier to get by the borrow-checker.

This can feel awkward at first and requires identifying the subsets of fields that may need to be borrowed separately.

Taking a value-based approach will tend to guide you in this direction anyway - values that change together will tend to be aggregated together as result values from functions (or methods) acting on purely immutable (shared access) arguments.

So right now I'm viewing structs as values that aggregate subordinate values in an optimal fashion to support the required capabilities and satisfy the constraints of the borrow checker - rather than conventional class-based objects.

Scala suffered from the bifurcation between the "a better Java" and the "I wish I could use Haskell" practitioners. It would be a shame if something similar would happen to Rust between the "Object-Oriented" and "Functional" camps.

  • Mindlessly creating new values can needlessly kill performance. Targeted mutability is key.
  • Trying to mimic OO constructs in Rust could lead to the accretion of unnecessary complexity (I'm reminded of the tactics that were employed to straight jacket prototypal ES5 into class-based OO). In this situation "this looks unfamiliar to me" shouldn't be confused with "this is objectively less readable".

That said, I'm always on the lookout for authoritative "you may be tempted to do this - when you really should be doing this instead" design and style guides (which may end up contradicting some of the notions that I have expressed above).

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.