Rc<RefCell<T>> and RefMut<T> in a single struct (for FFI)

[gnosek]

I have a type that's basically BTreeMap<K, Rc<RefCell<V>> that's exposed over FFI with methods like:

• create_entry(&mut self) -> Rc<RefCell<V>> (create a new entry, not inserted to the map)
• get_entry(&mut self, key: K) -> Option<Rc<RefCell<V>>> (look up an entry in the map, return a rc-copy)
• insert(&mut self, key: K, entry: Rc<RefCell<V>>) (insert an entry created by create_entry under a key)

I also have some methods that read/write the Rc<RefCell<V>> values, which call .borrow()/.borrow_mut() on the RefCell.

So, from the consumer's (who may not even be Rust) perspective, the flow is e.g.:

let entry = ffi_magic_get_entry(...); // Rc::clone happens here
entry.modify_somehow(); // RefCell::borrow_mut() happens here
entry.modify_again(); // another RefCell::borrow_mut
entry.just_read(); // RefCell::borrow
ffi_magic_release_entry(entry); // the Rc clone gets dropped here

(the entry cannot be stored away, it's conceptually a borrow of the actual Rc<RefCell<V>> from the map; if I didn't want to make the code multithreaded, I could probably drop the outer Rc)

This all works fine, but. The code is obviously single-threaded now and I'd like to make it multithreaded at some point. If I just replace Rc->Arc and RefCell->RwLock, I'd expect it to mostly work fine as well, except that the locking will be too fine-grained then. I'm not going to hold a lock over the whole time the entry exists, so separate threads can interleave access to the entry. I.e. the flow will be:

let entry = ffi_magic_get_entry(...); // Arc::clone happens here
entry.modify_somehow(); // RwLock::write happens here
entry.modify_again(); // another RwLock::write
entry.just_read(); // RwLock::read
ffi_magic_release_entry(entry); // the Arc clone gets dropped here

while I'd want it to be:

let entry = ffi_magic_get_entry(...); // Arc::clone and Mutex::lock happens here
entry.modify_somehow(); // already locked
entry.modify_again(); // already locked
entry.just_read(); // already locked
ffi_magic_release_entry(entry); // the Arc clone and lock gets dropped here

This means I'd need to store Arc<Mutex<V>> and a MutexGuard<V> referring to the same mutex in a single struct (because I can only hand out a single raw pointer to FFI as my entry type). Going back to the single threaded code, it maps to a struct with Rc<RefCell<V>> and RefMut<V>.

Now, this obviously won't Just Work if I .borrow_mut() from the sibling field, so I'm looking for alternatives. It almost feels like making the borrowed field a RefMut<'static, V> with a violent transmute is okay (the RefMut is valid as long as the struct containing it is, and due to Rc-wrapping, the RefCell has a stable address) but that just feels dirty. The actual RefMut won't be exposed anywhere outside a single module (except as a member in an opaque struct), so I can Just Not Do some things like storing the RefMut outside the struct.

Is there a way to accomplish this that's not as wildly unsafe? Or is the transmute actually fine? If I had a reference to store, I'd probably just make it a raw pointer, but it's not quite as easy with RefMut

[kpreid]

@gnosek

Is there a way to accomplish this that's not as wildly unsafe?

You can use an interior mutable type that provides a guard that uses Arc natively; for example, parking_lot and tokio. Both of these are overkill (they're thread safe and tokio's is async-compatible) in some way, but perhaps there's some crate that plugs a RefCell-like non-thread-safe borrow counter into lock_api::RawRwLock, at which point you can use lock_api::ArcRwLockReadGuard to get a type that does exactly what you need. (I know that this is a possibility but I've never explored lock_api for actual use.)

For more complex situations, ouroboros does the unsafe lifetime extension thing but wraps it up in a way you don't have to write any unsafe code yourself. However, that sort of thing has a rocky history of unsoundness bugs being found, so using it shouldn't be taken as 100% confidence that the result is sound.

[gnosek]

Thanks! ArcRwLockReadGuard looks exactly the kind of thing I'll need to make the thing thread safe. I'll still want the single threaded equivalent to:

1. make the transition smoother (rather than rip out and replace everything, first switch to the new API and then s/Rc/Arc/ as a separate step) and
2. have the option to use the single-threaded version, even though I don't expect it to be any faster (technically, the FFI API I'm coding against explicitly specifies non-thread-safety, so making it thread-safe is an extension).

I'll definitely take a look at the lock_api source and worst case I'll s/Arc/Rc/ my way into a single threaded implementation

EDIT: Going hybrid with an equivalent of Arc<RefCell> seems much simpler (no need to reimplement a bunch of lock_api internals, just a new impl of RawRwLock) and probably good enough.

EDIT 2: lo and behold: refcell_lock_api - Rust (no idea if it works yet but this is exactly what I need)