RefCell alternative that is Send?

use std::cell::RefCell;
use std::rc::Rc;

#[derive(Clone)]
pub enum Packet<'a, T: Clone> {
    Owned(Vec<T>),
    Slice(&'a [T]),
    Cell(Rc<RefCell<&'a mut [T]>>),
}

fn main() {
    let m = Packet::Owned(vec![]);
    std::thread::spawn(||{
        let m = m;
    });
}

This examply won't compile because RefCell is !Send. I think this is the only thing preventing Packet from being Send, at least for the 'a = 'static case. It could store the raw references inside of it but it's done this way so it can be cloned.

So, is there an alternative for RefCell that would make the example above Send?

This is the entire example if someone wants to know more: Rust Playground

The thread-safe counterpart of Rc<RefCell<T>> is Arc<Mutex<T>>.

2 Likes

I think Mutex is too much, what about a Packet that is just Send? Couldn't there exist Arc<&'a mut [Y]> since Arc is Clone?

By the way, what is Rc<RefCell<>> exactly? I understand that Rc is reference count and that RefCell makes it possible to borrow only once at the same time on the same thread. But why it's used in this example? I just needed a type that can be cloned and is Send

Sure, but Arc<U> only lets you get a &U, and you can't get &mut U from &&mut U, so you wouldn't be gaining anything over just Arc<[T]>.

Arc also can't make anything Send or Sync that isn't already [1, 2].

Rc<U> gives you single-threaded shared ownership of a value of type T, but you can't get &mut U from Rc<U>, only &U. RefCell gives you interior mutability: with some caveats, you can pass from &RefCell<U> to &mut U. So Rc<RefCell<U>> implements shared ownership of a value that you can also mutate (again, with caveats).

To get shared ownership (implying cheap Clone) plus mutability in a multi-threaded context, you need Arc<Mutex<U>>. Note that Mutex<U> also cannot be Send unless U is.

2 Likes

Your premise is wrong:

In fact, RefCell<T> is Send, at least when T: Send. You have a RefCell<&'a mut [T]>, which means that to make the RefCell Send all you need is to add a T: Send bound...

... But that actually doesn't matter at all, because you're not sending a RefCell<&'a mut [T]>, you're sending an Rc<RefCell<&'a mut [T]>>, and Rc<_> is simply never Send no matter what. So you're looking at the wrong type: it's not (directly) the RefCell that causes problems here, it's the Rc. This is also what the error message tells you explicitly:

error[E0277]: `Rc<RefCell<&mut [_]>>` cannot be sent between threads safely
               ^^^-----------------^

So you should be looking at Rc, not RefCell to fix your Send problems. And the Send equivalent of Rc is Arc.

Now what's going to happen if you write Arc<RefCell<&mut [T]>>? Well, you'll get a different error, because RefCell is not Sync. And it's Sync, not Send, that makes the difference between RefCell and RwLock. But you have to read the error messages and fix the Rc first before the compiler will tell you about that secondary problem.

2 Likes

makes sense, but why would Sync be needed if I just want to send the entire thing?

May I follow up on this question?

What exactly is the design trade-off underlying RefCell (and interior mutability as a whole)? Is it that one is willing to pay the price of dynamically tracking mutability vs. the potential compiler optimization gains from being allowed to assume there are no mutable aliases (e.g. restrict)? Or is there a deeper reason of why Rc<RefCell<T>> must track who has currently borrowed the interior cell? Given RefCell's single-threaded nature, is there a memory safety reason for the .borrow()/.borrow_mut() calls?

I was reading the When to choose interior mutability section in the documentation, which makes it sound as though interior mutability is uncommon. In my own experiments with Rust so far, I didn't find it that uncommon - my thought process was that any sharing (multiple pointers/back pointers, or being stored in multiple hashtables, etc. etc.) requires an Rc<T>, and if T is a struct with mutable fields, an Rc<RefCell<T>>. This then quickly leads down a path of .borrow(), .borrow_mut(), and .try_borrow_mut() which I found initially difficult to debug. For instance, when either borrow* call panics you don't get a backtrace of where the offending first borrow occurred.

If you just want to send a RefCell from one thread to another, you can do that. But you can't do it with an Rc, because there might be other Rcs in the current thread pointing to the same data and then you could have unsynchronized access from two threads at once. You'll have to try_unwrap the Rc to get the RefCell out. Or just remove the Rc, if you don't need shared ownership.

We all know the popular statement "shared mutable state is bad". I've not seen a programmer who totally disagree with it yet. But many of them still says "but sometimes it's too convenient" though it makes things harder later when the project got larger. From my experience the Rust help on this problem much by making shared mutable state a lot harder and inconvenient to use.

During the learning phase it's common to replicate program structures you're used to from languages shared mutable state is convenient. You may not even noticed it's shared mutable since it's too convenient but now you know it (and you've learned something more than Rust!). Next thing you can do would be to restructure the program to avoid shared mutable state.

2 Likes

Let's break this down. First, shared, then mutable.

My motivation for needing sharing is the need to look up information via 2 different keys. Could you explain how to do this using the proper Rust idioms without using Rc. Else, if I use Rc, I have shared state.

Second, mutable. I group information that is related into a struct in which individual fields are being updated. The only alternative to this I know is to create a new struct with copies of all fields + the changed field. Is this what you have in mind?
Basically replacing:

let mut a = A{x:4,y:6,z:7};
a.x = 5;

with

let a = A{x:4,y:6,z:7};
let a = A{x:5,y:a.y,z:a.z};

to avoid the mutability? Does Rust provide zero-cost abstractions for that?

You should not break it down. Only shared mutable state is blamed, not shared state nor mutable state.

I'm afraid I'm still at a loss as to what I'm missing/what you're trying to convey. The traditional (as in all other procedural languages) approach I am using is to have a struct with say half a dozen fields. References to the struct must be stored (as a value) in 2 hash tables. The struct's fields are updated depending on external events, which requires mutability.

How do I avoid the issue by getting rid of either sharing or mutability?

Assuming you have struct Person with lots of fields, and need to search them with both SSN and name(also assume names never conflict). I guess you're trying to have HashMap<Ssn, Rc<RefCell<Person>>> and HashMap<Name, Rc<RefCell<Person>>> to search and modify person data. I'd rather to have HashMap<Ssn, Person> and HashMap<Name, Ssn> instead. With Rc<RefCell<T>> you can make mistake that seemly unrelated method call tries to .borrow_mut() the person while you're already holding the mutable access of it, which make it panic. But without the shared mutability such mistake is statically checked and make it compile error instead of runtime panic.

So you're proposing to pay the runtime overhead of double indirection (2 hash table lookups?) to avoid the sharing?

This is indeed what's happening. Specifically, I have a 'currentPerson' (to stay in your example) that is already mutably borrowed. Sometimes, an operation applies to the current person, and sometimes it applies to another. I find the person in the hashtable, try to borrow it mutably, and if that fails, I know it must be the current person and fall back to it.

let personref = self.ssn2person.get(&ssn).unwrap().clone(); 
if let Ok(mut person) = personref.try_borrow_mut() {
   handle_person(&mut person, ...);
} else if let Some(cperson) = currentPerson {
   handle_person(cperson, ...);
} else {
   panic!("must provide either currentPerson or borrowing must succeed");
};

this code path is sometimes called with currentPerson = Some(&person) and sometimes with currentPerson = None. Not very elegant.

So let's say I were prepared to pay the price for the double indirection to avoid the sharing introduced by having 2 references in 2 tables. What then?

As soon as I store a struct in a hashtable (which transfers ownership), absent Rc, I can access it only by borrowing the value stored in it. I couldn't use the currentPerson reference since this would again create sharing/violation of the borrow rules. How do you propose I handle that? By cloning the key and repeating the hashtable lookup wherever I would otherwise use a direct reference to the struct? Butler Lampson is credited with saying that every problem can be solved by adding another layer of indirection; is this really the Rust way?

Indeed. It pays double lookup to remove aliasing.

For this case, consider to not store the current person in the hashtable. This would add additional branch to the lookup function to check current ssn first.

struct State {
    // boxed the person to minimize move cost, possibly premature optimization
    persons: HashMap<Ssn, Box<Person>>,
    ssns: HashMap<Name, Ssn>,
    current_ssn: Ssn,
    current_person: Box<Person>,
}

Also suggested by the discourse popup, it seems we're somewhat hijacking the thread at this stage. Feel free to open another thread and mention me if you have more questions.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.