Help with understanding RefCell<T> and interior mutability pattern

Hi, I am currently learning Rust from the official book, and I am having a hard time trying to understand how the interior mutability pattern works using RefCell in chapter 15.5 of the book.

Can someone explain to me in simple words so that I get a better understanding..

This is an example I wrote...

use std::cell::RefCell;

fn main() {
   let val1 = RefCell::new(Vec::new());
   
   let mut borrow_1 = val1.borrow_mut(); // The borrow_mut method returns a RefMut<T> smart pointer
   
   borrow_1.push(10);
   
   // val1.borrow();    // RUNTIME ERROR ! thread 'main' panicked at 'already mutably borrowed: BorrowError'
   
  // let mut borrow_2 = val1.borrow_mut();
   
  // borrow_2.push(12); // RUNTIME ERROR ! thread 'main' panicked at 'already borrowed: BorrowMutError'
}

My question here is why does borrow_1.push(10); work?

From what I understand borrow_mut() returns a RefMut<T> smart pointer but how can I call push() on it, I need to get the internal vector from the RefMut wrapper before calling push(),

Why does directly calling push() on RufMut<T> work?

What will happen if I do this (*borrow_1).push(10);?

Next, I also tried the Cons List example in the book.

In order, to understand each line of the code I wrote some comments in the code.

Can someone review the comments I have written in this below code and verify if my understanding was correct, especially the HOW IT WORKS part that I wrote, was this correct?

#[derive(Debug)]
enum List {
    Cons(Rc<RefCell<i32>>, Rc<List>), // Our 'List' Enum now holds its data as a Rc<RefCell<i32>> and the rest of the 'List' as a Rc<List>
    Nil,
}

use std::cell::RefCell;
use std::rc::Rc;

fn main() {
    let value = Rc::new(RefCell::new(5)); // Wrap the value '5' in a RefCell<T> and then wrap it in a Rc<T>

    // Here we use Rc::clone() to so both 'a' and 'value' have ownership of the inner 5 value 
    // rather than transferring ownership from 'value' to 'a' or having a borrow from value.
    // Next, we wrap the `List` again in a Rc<T> since we later wish to give ownership of 'a' to 'b' and 'c'
    let a = Rc::new(List::Cons(Rc::clone(&value), Rc::new(List::Nil))); 
    
    // Add a new value '3' and '4' to 'b' and 'c' respectively using `Rc::new(RefCell::new())`
    // Also, adding both 'b' and 'c' as the owners of 'a' by `Rc::clone(&a)`
    let b = List::Cons(Rc::new(RefCell::new(3)), Rc::clone(&a));
    let c = List::Cons(Rc::new(RefCell::new(4)), Rc::clone(&a));
    
    // At this point Rc<T> for '5' is owned by 2 owners 'value' and 'a'
    // and Rc<T> for 'a' is owned by 3 owners 'a', 'b' and 'c'


    // Here calling borrow_mut on 'value', uses the automatic dereferencing to dereference the Rc<T> to the inner RefCell<T>
    // Then borrow_mut method on RefCell<T> returns a RefMut<T> smart pointer
    // Now we use the dereference operator(*) on it and change the inner value.
    
    // HOW IT WORKS:-
    // *value.borrow_mut() -> *(Rc<RefCell<T>>.borrow_mut()) -> 
    // Now automatic dereferencing happens here since borrow_mut() is for RefCell<T> so it automatically deferences Rc<RefCell<T>> to RefCell<T> then ...
    // -> *(RefCell<T>.borrow_mut()) -> *(RefMut<T>) -> 5 (since RefMut implements DerefMut)
    *value.borrow_mut() += 10;

    println!("a after = {:?}", a); // a after = Cons(RefCell { value: 15 }, Nil)
    println!("b after = {:?}", b); // b after = Cons(RefCell { value: 3 }, Cons(RefCell { value: 15 }, Nil))
    println!("c after = {:?}", c); // c after = Cons(RefCell { value: 4 }, Cons(RefCell { value: 15 }, Nil))
    
}

Thanks to anyone who takes the time to read through all this, any advice will be helpful to me. :slight_smile:

The Rust Reference explains this better than I could:

So, in your case the compiler first looks for a push method on RefMut<Vec<T>>. Because there isn’t one, it automatically dereferences the value and looks for it on Vec<T>, which succeeds.

The only difference between borrow_1.push() and (*borrow_1).push() is that the latter will bypass any push method native to RefMut. Because it doesn’t exist, they are equivalent.


Edit: The comments in your last code sample look about right.

1 Like

Thanks for the explanation.

I understood that Rust is automatically dereferencing the RefMut<Vec<T>> to Vec<T> when it does not find push() on RefMut.

When I do (*borrow_1).push() does this mean I am doing the automatic dereferencing here for rust?

This is what I understand, can you confirm if this is correct:

RefMut<T> implements DerefMut trait so *borrow_1 gives me the underlying Vec<T> over which I can then call push().

1 Like

Yes, this is correct.

1 Like

I'd also recommend Crust of Rust: Smart Pointers and Interior Mutability and/or Rust Stream: The Guard Pattern and Interior Mutability -- they both provide excellent explanations and are more example driven than books.

1 Like

Thanks for the links, I will try to watch thhe videos whenever I get time.

I have also written about interior mutability, specifically Cell.

2 Likes

To expand a bit on this:

The "guard" pattern

Let's look at the following code:

let refcell = RefCell::new(vec![]);
{
    let mut vec_mut = refcell.borrow_mut();
 // let vec_mut: &mut Vec<_>  ??
}

Basically, we have some "magical" wrapper that allows to get &mut references to the interior (the vec![]) even though the "exterior" (the RefCell) was not marked mut.

So, if we were the ones designing the RefCell API, we would like to write:

/// We would *like* to have the following signature:
fn borrow_mut (self: &'_ RefCell<Vec<T>>)
  -> &'_ mut Vec<T>

Sadly, such a signature is not1 possible, since we need to have a way to express to the code that our borrow_mut() is limited in time / within a span of code (in my code snippet above, that would be the { ... } block).

And since we are talking about runtime properties, this means that we need, at some point within the code, to express that we "release" our borrow_mut() borrow / that we relinquish it, so that the initial RefCell is borrowable again.

That is, back to our code snippet, we want to have:

let refcell = RefCell::new(vec![]);
{
    let ... vec_mut ... borrow_mut();
} // <- insert a call to `refcell.__internal_release_mut_borrow__()`

And that's the reason we cannot return a &'_ mut Vec<T>: such a type is a "primitive" type (just a bare (mut) reference), so there is no way to attach some code (.e.g., the "call" to release_borrow_mut()) when that &mut borrow ends.

The way to achieve this is to, instead, return some custom-made wrapper type, that wraps that &mut reference, but which, by virtue of being a new type, can have "destructor / drop() glue" attached, which thus provides us of a place where we can call release_mut_borrow():

//! Pseudo-code:

struct MutRefWrapper<'refcell, T> {
    original_refcell: &'refcell RefCell<Vec<T>>,
    mut_borrow: &'it_is_complicated mut Vec<T>,
}

impl "this is a `&mut Vec<T>`" for MutRefWrapper<'_, T> {
    use-ing self.mut_borrow...
}

impl<T> Drop for MutRefWrapper<'_, T> {
    fn drop (self: &'_ mut Self)
    {
        self.original_refcell.__internal_release_mut_borrow__();
    }
}

which allows us to now provide the following safe API:

impl<T> RefCell<Vec<T>> {
    fn borrow_mut (self: &'_ RefCell<Vec<T>>)
      -> MutRefWrapper<'_, T>
    { ... }
}

I think that up until this point, what I've said should feel "natural", as long as we hand-wave the implication details :smile:. In other words, the "idea" of the signature of borrow_mut() is rather:

fn borrow_mut (self: &'_ RefCell<Vec<T>>)
  -> /* something that */ impl Drop + BehavesLike<&'_ mut Vec<T>>

Again, I hope I have conveyed the wave-handed idea of the intention behind the design of such an API.

At this point, there are only two things left to clarify, before we are no longer wave-handed:

  • the impl Drop part,

    • Regarding this, it turns out that Drop is a special trait w.r.t. the compiler, so we do not need to be explicit about this. The moment we have some custom-made type, be it an explicit MutRefWrapper... or an opaque impl ... type, they both are free to have an impl Drop and things will just work, even if it is not apparent within the API.
  • the impl BehavesLike<&'_ mut ...> part.

    Regarding this one, this is where the "auto-deref" of "smart pointers" comes into play.
    That is, the auto-deref mechanism was designed precisely to be able to define wrapper types, with special oddities about them (such as having Drop glue), while behaving like a reference. That's the role of the Deref{,Mut}:

    impl Deref   <Target = X> β‰ˆ &    X
    impl DerefMut<Target = X> β‰ˆ &mut X
    

But granted, the real definition of Deref{,Mut}, as their names indicate, is that there needs to be a *-dereference for them to kick in:

  • x: impl Deref<Target = X> β‡’ &* x: &'_ X
  • x: impl DerefMut<Target = X> β‡’ &mut* x: &'_ mut X

And where all the magic happens, is that, indeed, the dot operator . is able to perform these kind of dereferences if needed:

{
    let mut vec_mut =  val1.borrow_mut();
    // `vec_mut: impl DerefMut<Target = Vec<T>>`
    // β‡’ `&mut *vec_mut: &'_ mut Vec<T>`
    (&mut *vec_mut).push(...);
    // and thanks to `.`-operator auto-deref, we can just write:
    vec_mut.push(...);

} // <- `drop(vec_mut)` β‡’ `Drop::drop(...)` β‡’ `val1.release...()`

// val1 can be borrowed again :)

TL,DR

fn borrow_mut (self: &'_ RefCell<T>)
  -> impl '_ + DerefMut<Target = T>
//-> Pseudo<&'_ mut T> where Pseudo<...> may impl Drop too.

This pattern is called the "guard" pattern, since this returned wrapper value "guards" the accesses to the interior value, only performing a clean up of it when are done using the interior value, thus guaranteeing a proper usage.

Aside: an alternative API

Note that a way to implement a similar API, but with much less magic, would be to rely on a callback-style rather than relying on DerefMut + Drop magic:

fn with_borrow_mut<R> (
    self: &'_ RefCell<T>,
    with: impl FnOnce(&'_ mut T) -> R,
) -> R
{
    self.__internal_try_acquire_mut_borrow().unwrap();
    let inner_mut: &mut T = unsafe {
        self.__internal_get_mut_unchecked__()
    };
    let ret = with(inner_mut); // <- run caller-provided logic on a `&mut T`.
    self.__internal_release_mut_borrow(); // <- run the clean-up logic
    ret
}

Which would have been usable as:

val1.with_borrow_mut(|vec_mut: &'_ mut Vec<T>| {
    vec_mut.push(...);
}); // <- closure ends here, the above cleanup happens then.

This comes with the heavy ergonomic cost of requiring a callback, and thus with a much less readable function signature, but has the advantage of not requiring DerefMut or Drop sugar :slightly_smiling_face:

  • (although if we want a guaranteed cleanup in case of panic within the user-provided closure, we would need a helper Drop-able guard, or to temporarily catch the panic to perform the cleanup before resuming it).

1 Unless we condemn the RefCell to be borrow_mut-ed forever.

2 Likes

This made me think that when I started with Rust, I discarded interior mutability as some ugly kludge that that a well-behaved programmer would never want. Little did I know back then! :slight_smile:

What really helped me was to realise that mut/non-mut is not the same as const/non-const in other languages. Once I started thinking about mut as exclusive access and non-mut as shared access (as opposed to read-write vs read-only access), interior mutability started to make a lot more sense -- it allowed me to get exclusive access to something to which I only had a shared access, for example an element in a collection or a field in a structure. It also helped me to see that this is similar to Mutex which is a guard that gives me exclusive access to something which is otherwise shared unguarded.

1 Like

In fact, Mutex is just a generalization of the RefCell, which does not panic on concurrent access, but rather allows one to wait (blocking the current thread, until another thread finishes its part of work).