Iterating over multiple mut from single owner


#1

Hi,

I am writting plugin manager in rust, to see how it is possible to split rust based application into smaller decoupled modules.

At this point everything that I’ve done, is made in rust safe way, however now I have dilemma.

I have HashMap of containers, where each container is just a trait, that could return some container item by key.

I am implementing kind of lifecycle over set of container items, it means that according to criteria I have to build a list of container items from different containers, and iterate through this list couple time to send different events to the item.

Also I’ve borrowed mutable reference to the hash-map, and at this point rust guaranteed that no other thread or subroutine can read or mutate hash map, container and container items.

The issue that rust borrow checker, doesn’t allows me to take two mutable references to container that are stored in same hash-map, since I have only one mutable reference to hash-map. Same issue with references to multiple mut container items from single mut container.

Questions that I have:

  1. I need some simple solution for that. I don’t want go go to any over-engineering, but I still want to have this mutability. Please help me with any suggestion, how to solve this issue?
  2. In C++ which unsafe by default, seems it is not an issue if you know what you are doing. Should I use rust unsafe mut pointers for this case, or should I keep pushing safe way to solve this task?
  3. If it still not possible to write every algorithm in rust in safe way. Are there any good guideline for real enterprise and production systems where no time to think, which will describe when to use safe and unsafe code, which would be understandable for newcomers on the project, who came from Java, Go or C++?

#2
  • You can always cheat mutability with RefCell or Mutex.

  • You can always cheat ownership with Rc or Arc.

  • As a rule of a thumb, never put references in structs that live longer than one function call. If some data structure is a temporary view of another struct that you create and then immediately drop, then references are OK, but if a struct lives across function calls, it has to own its content. Otherwise it will be a horror to use. If two structs can refer to a plugin, use Rc/Arc.

  • References are not pointers. They are shared or exclusive locks on the data. If you have &mut Plugin, that’s an exclusive lock on the plugin and nobody else anywhere can use it. It’s common in C++ to refer to an object by a pointer to it. In Rust objects don’t have stable addresses (except temporarily while being locked by a live reference), so this doesn’t make sense. Use some other ID/name or Rc/Arc pointer.

  • HashMaps are a tough case for the borrow checker. To work around limitations in common cases there’s entry API. Sometimes you can remove immediately instead of get_mut. Sometimes you just need to do lookup twice (e.g. in your case instead of holding mutable reference to a plugin, hold a clone of the Arc key instead).

  • It takes practice to get hang of what borrow checker likes and doesn’t like. When you “think in C++” you will keep running into this all the time. Once you learn to “think in Rust” you won’t.


#3

Can’t comment on code but search brings up;
https://crates.io/crates/multi_mut


#4

Hi, thank you for your answers.

I’ve checked source codes of multi_mut, and this is what I am saying the only way to get two or more mut references to child items from single parent is to use pointers, how it is done in multi_mut.

I like the way how multi_mut is implemented, it provides safe API, while internally it uses unsafe pointers. If I have mutable reference to HashMap, I can use list of unique keys to get iterator over mutable references to hash-map values. Moreover this Iterator will be RAII Guard that will borrow mut reference to hash-map, and will protect from double aliasing to that hash-map and its children.

Regarding the RefCell and RC, I was considering this at first, however have couple arguments against this.

  1. I have single mut ref to hash map, it means that I expect that I have an exclusive access to hash-map and all children owned by hash map, so I should be able to write any algorithm to work with hash-map I need. If I will rewrite hash-map values type to Rc RefCell instead, yes I will be able to get multiple RefMut to iterate, however mut ref to hash map will not provide any guarantee for exclusive use for me. It will not solve all issues, since RefCell could throw panic exception anytime, it is safer than SEGFAULT but still.
  2. The second argument is that this is my libraries, and I can change typing, however my goal still to learn rust best practices, and speaking about real systems it is more common that hash-map type already defined and used by many other code modules, and changing data model type will require more complex refactoring or even constrained due to backward compatibility support. So here should be the way how to write any algorithm for any application model and any data structures.

Regarding my questions, it seems that answers are:

Regarding 1st question - I have to use external crate multi_mut with which provides safe API for my case, or use unsafe pointers. Seems that raw pointer can be used to work with any data structure and any application model.

Regarding 2 and 3 questions - when I should stop using raw pointers, it is not clear. Any answers regarding that?