Reference to a collection of mutable references

Hello,

I'm so confused that it's hard for me to search for an answer here - and I'm sure there are plenty already because this seems "simple".

I want to do the following: I want to have a struct that holds a value. I want to have a collection of instances of that struct. I also want to be able to Impl a method that from that struct acts on every member on that collection (possibly mutating it).

pub type Collection<T> = Vec<StateWrapper<T>>;

pub struct StateWrapper<T> {
    field: T,
    collection: Option<Collection<T>>,
}

fn main() {

        let mut a = StateWrapper { field: 1, collection: None };
        let mut b = StateWrapper { field: 2, collection: None };

        let mut collection: Collection<i8> = Collection::new();

        collection.push(a);
        collection.push(b);

        a.collection = Some(collection); // This will not work as I want readable reference here
        b.collection = Some(collection); // but I need let mut collection to be able to .push() 
}

What I want is to be able to add new StateWrapper instances to collection, and I want to be able to do something like:

impl<T> SateWrapper<T> {
   fn touch_all(&self) {
      for sw in self.collection.unwrap() {
         sw.do_whatever_probably_mutable()
      }
  }
}

I'm sure I'm making a silly mistake somewhere - but I can't make this work. Maybe this should be modeled some other way?

It's a little unclear what you need. Please bear with me while I ask questions and provide vague advice.

I want to have a struct that holds a value. I want to have a collection of instances of that struct. I also want to be able to Impl a method that from that struct acts on every member on that collection (possibly mutating it).

This all makes sense. But note that compared to your title — a “collection of instances” is a very different thing from a “collection of mutable references”. It is rarely a good idea to make a collection of mutable references, but also, your code hasn't done that. It is currently free of mutable references. (let mut does not create any kind of reference.)

If you can describe more of the problem you want to solve (not just how you currently think you should do it in Rust) then we might be able to give better advice.

        a.collection = Some(collection);
        b.collection = Some(collection);

This code will always fail, because collection has been moved in the first line and therefore cannot be used in the second line. But, what do you want instead? Do you want each of a and b to have its own collection? In that case, add #[derive(Clone)] to StateWrapper and:

        a.collection = Some(collection.clone());
        b.collection = Some(collection);

Or do you want a and b to share a collection they can both modify? That is much more complex and involves more design choices, because Rust's default and preference is to not have mutation of shared values.

In order to make this code work, here are the changes you need to make:

  • This mutates the contents of the StateWrapper, so it must take &mut self instead of &self.
  • self.collection.unwrap() attempts to move out of the field self.collection (because unwrap takes its receiver by-value), which isn't what you want. The general solution is to call .as_ref() or .as_mut() first, which allows going from &Option<T> (reference to an Option) to Option<&T> (optional reference), or the same but for mutable references.

This should compile (given appropriate contents of the loop, of course):

impl<T> StateWrapper<T> {
   fn touch_all(&mut self) {                        // added &mut
      for sw in self.collection.as_mut().unwrap() { // added .as_mut()
         sw.do_whatever_probably_mutable();
      }
  }
}
3 Likes

Thank you - this is very useful :bowing_man:

Or do you want a and b to share a collection they can both modify? That is much more complex and involves more design choices, because Rust's default and preference is to not have mutation of shared values.

Yes! I want them to share the collection but be able to modify the elements (so not the collection itself). For example let's say the touch_all method is comparing the value of field of the self with all other StateWrappers in the collection and does something to them based on that value (like +1 or -1 on the field).

For that to be possible I probably don't need self.collection to be mutable but the items it holds need to be mutable.
Also because after a and b is added to the collection it can still be modified by adding c, d, ... at later point - so somewhere I need to have a mutable reference to it (but probably not in the StateWrapper).

And I'm struggling with this sequence:


let mut collection = ..... // so I can add a and b it needs to be mutable, right?
//...
let a = .... collection: collection // here I would like to pass readable reference but I have mut above
collection.push(a);
//...
// here I might want to add more elements so collection needs to be mutable again
collection.push(c);

This can't be so tricky, I have to be doing something wrong :thinking:

Nothing wrong, you are just clearly trying to bring code from some other language into Rust. It's not impossible, but would definitely be painful.

How would you code guarantee that you wouldn't modify self via that collection?

The whole Rust philosophy is built around that shared mutable access to objects is evil and shouldn't exist. You can either have few observers or one modifier. If someone may look on your data then it must be frozen and unmodifiable, if someone can modify it then there should be no one else who can be affected by such modifications.

Shared modifiable state is not allowed.

Your design is fundamentally incompatible with that approach thus you probably want to use Arc or Rc (which would still enforce that rule, but at runtime, not compiler-time like references do).

But for better advice one really need to know just why you are trying to create this convoluted and circular pile of pointers.

Very often one can do better than just brute-force port code from some other language with the use of Arc/Rc if the actual problem is known.

1 Like

Okay, this is a very, very specific requirement as far as Rust is concerned. It means that:

  • Your Collection must provide interior mutability to allow modifying the elements. This can be done using, for example, RefCell. (Without interior mutability, you cannot say things like “modify the elements but not the collection” because there is no distinction between those two things — a mutable reference always allows replacing the entire thing it refers to.)
  • You must use Rc to allow sharing the collection between two “parents”.

(Those are the non-thread-safe versions. For thread-safety, substitute Mutex and Arc — but think twice before combining threads and interior mutability, as now you have to think about timing of actions of multiple threads.)

It will probably make sense to make Collection a struct rather than a type alias, and give it suitable methods.

However, before embarking on this, can you say why you want this? It might be that there is a better option. For example, in some cases it makes sense to, instead of having nested data structures and interior mutability, to use keys/indices — that is, you have a single table of all Collections in your application's state, and your StateWrappers store keys that identify the collections rather than owning them. Or there might be some completely different representation that makes the most sense. Tell us about your use case, instead of what code you currently imagine will help implement it.

3 Likes

This took me some time to digest. And since I don't want RefCell, Rc or any thread-safety variants at this point - it seems clear I need to design it some other way.

My expectation was that Rust will be more like structs + methods.. but it seems to be more data + functions.. maybe?

Back to my problem. What I'm trying to do is to have a bunch of Server objects that form Clusters. Down the line each of those servers will have some kind of loop (thread). But for now I want to model some of this stuff in one process, just focusing on the interactions between the servers.

The flow I want to have is this:

  • a Server joins the cluster (so I need it to join the collection and change some state on it)
  • a Server in cluster sends a message to all other servers in the cluster and possibly changes some of their state.
  • servers can join/leave the cluster at any time

So... instead of having a Server with a reference to a Cluster... Should I have something like server.cluster_id where ID identifies the cluster in some global collection? But then the server still needs to somehow send the message to all other servers in the cluster... so it has the ID but it needs the reference to the global collection of clusters as well... What am I missing here?

This is just a fundamentally hard problem, and all of the “easy” solutions have some major drawbacks, either in terms of system performance, architectural restrictions, or both. In the most general case, you’ll end up with something like Paxos or Raft.

The simplest approach is probably to use one channel per server as a message queue, and keep a copy of all the senders in a shared collection for when you need to broadcast to the entire cluster.