Still being a Rust newbie, I'm working on a music notation project with lots of different structs are holding vecs of references to other items. The project has grown fairly large, there's a lot of interior mutability going on. And as the program grows, there are more and more of those Vec<Rc<RefCell<Note>>>s with the corresponding boilerplate to deal with...
So, I'm thinking, what are the alternatives to using this interior mutability approach?
Partly because of Cahterine West's talk about ECS systems (https://www.youtube.com/watch?v=aKLntZcp27M&ab_channel=Rust), I'm starting to sensing a bit of "<Rc<RefCell>> smell"...
So, what if these entities - instead of keeping lists of references to the notes (Vec<Rc<RefCell<Note>>>) - instead should keep lists of Note IDs? And the notes themselves, when created, were stored in some kind of global static mutable HashMap?
(Still being new to Rust, I don't really have a clue to set up such an architecture. A global hashmap obviously can't be static and mutable at the same time, as an example.)
So, how do you guys deal with larger scale apps with lot's of object cross references going on? Is interior mutability still the way to go? Or is the list-of-indexes-to-global-map described above a way to go? Other approaches? Entity Component Systems? GhostCells?
If you want to store user data and access/manipulate it globally, then you want a database. Storing all of the arbitrary user-generated content in a giant map is unacceptable regardless of interior mutability.
This is false. static mut is a thing (albeit unsafe), and non-mut statics can be mutated via interior mutability.
I didn't succeed with interior mutability for a static hashMap, but using the once_cell::sync::Lazy; from once_cell crate seems to work fine.
Below is an first try that seems to do what I'm after. Does this seem sound, and idiomatic? Will it scale?
#![allow(dead_code)]
#![allow(unused_variables)]
use once_cell::sync::Lazy;
use std::sync::atomic::Ordering;
use std::{
collections::HashMap,
sync::{atomic::AtomicUsize, Mutex},
};
type ID = usize;
pub static ID_COUNTER: AtomicUsize = AtomicUsize::new(0);
pub static NOTES_MAP: Lazy<Mutex<HashMap<ID, Note>>> = Lazy::new(|| {
let m = HashMap::new();
Mutex::new(m)
});
//=======================================================
#[derive(Debug)]
pub struct Note {
pub id: ID,
pub value: f32, // arbitrary example data
}
impl Note {
// create a note, insert it into the map, and return its id
pub fn create(value: f32) -> ID {
let id: ID = ID_COUNTER.fetch_add(1, Ordering::Relaxed);
let note = Self { id, value };
NOTES_MAP.lock().unwrap().insert(id, note);
id
}
}
//=======================================================
// a collection of note ids as Vec<ID> - replacement for Vec<Rc<RefCell<Note>>>
#[derive(Debug)]
pub struct NotesCollection {
pub ids: Vec<ID>,
pub sum_value: f32,
}
impl NotesCollection {
// create collection from vector of note ids
pub fn new(ids: Vec<ID>) -> Self {
let map = NOTES_MAP.lock().unwrap();
let value = ids.iter().map(|id| map.get(id).unwrap().value).sum();
Self { ids, sum_value: value }
}
// function to test mutability of the referred notes, and of the collection itself
pub fn set_note_value_and_recalc_sum_value(&mut self, note_id: ID, val: f32) {
let mut map = NOTES_MAP.lock().unwrap();
if let Some(note) = map.get_mut(¬e_id) {
note.value = val;
self.sum_value = self.ids.iter().map(|id| map.get(id).unwrap().value).sum();
}
}
}
//=======================================================
fn main() {
let mut notes = NotesCollection::new(vec![Note::create(10.0), Note::create(20.0)]);
dbg!(notes.sum_value); // 30.0
let first_note_id = *notes.ids.first().unwrap();
notes.set_note_value_and_recalc_sum_value(first_note_id, 20.0);
dbg!(notes.sum_value); // 40.0
}
Why do you need mutability? At what point(s) do you need to mutate your state, and when you do is there a reason why a method consuming self to produce a new, owned value of Self wouldn't work?
Large databases are obviously tricky with this, opting for 'eventual consistency' or something like that. But it seems like your parallel calculations are based mostly on processing external data, and your sticking point is at trying to have multiple threads simultaneously be able to update a collection and/or modify one or more values within it. At the very least your goal to get to a single Arc/Mutex wrapped data structure is probably the right direction, though I definitely consider using globals as a code smell.
The need for mutability comes from that core objects, like Notes in the example, need to carry properties that can only be calculated in "passes", where key values for one pass can't be known until the previous "pass" is calculated.
As mentioned above, I'm a bit inspired by this: https://www.youtube.com/watch?v=aKLntZcp27M&t=1790s&ab_channel=Rust, and Catherine's suggestion to try Vecs with indexes instead of interior mutability. Maybe this can be accomplished without a global map to store the items..?
There is the im crate, which implements several immutable versions of standard library collections. While its collections are ostensibly immutable, they do have the ordinary mutable methods you'd expect (e.g. Vector's apppend method takes &mut self). But really I'd just recommend reading the crate's overview as that has some interesting notes on mutability in Rust (and what it does and doesn't get you).
More to the point, if a pass over the stored data is required to know the next state, that specific point has to take place as a sequential operation. Starting with just your Note struct above, you could separate any complex and parallel parts out in an &self method, and call that from the consuming or mutating next state method, roughly:
I'm no expert on FP, but if you can separate out the process of changing state from the calculations required to get that next state, I feel that's where you get true "fearless concurrency" in Rust.
If a global map isn't a way to go, let's drop that!
Back to the basic question:
The solution that I have to day has a lot of entities with Vec<Rc<RefCell<Items>>> in them. This means a lot of boilerplate, up to the point that I'm starting to considering alternatives. Are there any established patterns that could be used?
Cahterine's talk suggests so, and there are also some examples of using ECS outside game development:
Does any of you have experience with ECS outside games and/or GhostCell solutions? Or is interior mutability still the way to go (despite the wordy boilerplate)?
There's no one simple answer for this. There won't be a "just use MagicCell" answer. It needs a different way of thinking about application architecture, which is a big "it depends" topic.
However, there are some small things you can start with:
Try not to mutate things in place, and avoid having state inside most small things. Prefer a more functional approach that returns new values. Instead of
let mut car = Car::new(); for _ in 0..4 { car.add_wheel(w) }
use
let car = Car::builder().with_wheels([w; 4]).build();
Prefer iterators and collect() instead of vec.push() (but if you have some loop that is too awkward for iterators, don't fight iterators, push is fine).
Don't expect to have parent references, or globally available singletons. Use function arguments a lot. Pass context down to functions that need it, explicitly. That's basically dependency injection, but in a minimal way, not the Java's magic DI framework style. Instead of note.insert_yourself_to_database_and_log() have db.insert(note).map_err(logger).
For UIs, compare the functional-style egui to traditional refcounted and mutable gtk.
ECS is specific to games or similarly very dynamic systems. If you're not making something big and complex, then a few Arc<Mutex> objects are fine.
OnceCell is a useful compromise to have some mutability for lazy init, but still use bare references to the data. There's also ArcSwap for things that you mutate rarely (e.g. global-ish config of a program).
Been looking a bit further in this, and I realise that one of the mistakes I've made so far is overusing Rc<RefCell<Item>>, and underusing Cell<SpecificItemProperty>.
Hardly ever do I need to mutate anything but specific properties of Item, and as I can not see that there should be any problem with the "cost" of those properties having to implement Copy, the use of Cell should be fine.
Here's a note to further me (and to others interested):
Michael F Bryan's article "Common mistakes and bad practices in Rust" adresses the problems of overusing Rc<RefCell<Stuff>> when trying to implement solutions from other OO-langugages.
The example in the article is also referenced in this video from Let's get Rusty.
