Problem with borrowing mutable references

Hi all.
recently I have started working with Rust to implement some code that I had implemented with C++, then Go.

I understand the concept of security that Rust provides but it seems to me that it is so restrictive that kill productivity over time since one needs to change and restructure the code many times to the point that the natural intuitive implementation is completely gone.

I just give a simple example. I have a several layers in neural network and they communicate with each other by passing messages.

Inside main I just process two messages by the first layer and take the messages produced by the first layer and pass them to the second layer.
Rust does not allow me as apparently I am creating two mutable reference to l1 while processing first and second message.

This is just a very naive example and I do not know how one can fix it. One can definitely move the values but this may not be a good idea in data science because there are always two phases forwrad: data is processed and backward: gradient is computed.

Any idea how one can really fix this ?

// simple message containing information
struct Message{
     content: f64,
     channel: i64,
}

// a simple linear layer
struct Linear{
   msg_in: Message,
   list_msg_out: &[Message],
}

impl Linear{
   fn new()->Linear{
       ....
   }

   fn process(&mut self, &msg_in: Message)->&[Message]
       //do some processing and store the results in output array
       return self.list_msg_out;
   }
} 

fn main(){
     // create two layer network: just a dummy example
    let mut l1 = Linear();
    let mut l2 = Linear();

    let msg_in = Message{....};
 
    // process the message with the first layer
    let mut list_msg_out1 = l1.process(&msg_in);

    // pass the results to the second layer: here just process the first message from the list
    l2.process(list_msg_out1[0])

    // now process another message
    let msg_in_next = Message{...}
    let list_msg_out1 = l1.process(&msg_in_next)

At a guess you're using &[Message] when you should have an owning Vec<Message>, but it's hard to tell from the given example. I couldn't reproduce a borrow error, but I also had to make a lot of adjustments before the code compiled.

1 Like

I apologize for not publishing the whole code which is a little bit complicated.

yes internally I represent the list_msg_out as a vector and pass a slice of it as &[Message] to the next layer. The problem is, as I mentioned, when I do this, I cannot use the linear object l1 more than once as Rust complains that I am using a mutable reference twice, which is quite annoying.

Please see the following example code, which is not exactly what I have but quite similar:

#[derive(Debug)]
struct Message{
    content: i64,
}

#[derive(Debug)]
struct Processor<'a>{
    added_value: i64,
    msg_ref: &'a mut Message,
}

impl<'a> Processor<'a>{
    fn new(added_value: i64, msg_ref: &'a mut Message)->Processor<'a>{
        let processor = Processor{added_value: added_value, msg_ref: msg_ref};
    
        return processor;
    }

    fn forward(&'a mut self, msg_ref: &'a mut Message){
        self.msg_ref = msg_ref;
    }

    fn process(&mut self){
        self.msg_ref.content += self.added_value;
    }

    fn backward(&mut self) -> &mut Message{
        return self.msg_ref;
    }
}

fn main(){
    // create a message
    let mut msg = Message{content:10};
    println!("message before process: {:?}.", msg);

    // processor
    let mut dummy_msg = Message{content:0};
    let mut processor= Processor::new(10, &mut dummy_msg);

    processor.forward(&mut msg);
    processor.process();
    let msg_ref = processor.backward();

    println!("message after process: {:?}.", msg_ref);

    println!("dummy message is {:?}", dummy_msg);
}

I give another example: suppose we have an struct that contains a reference. Try to write a function new that returns an instant of that struct, good luck :slight_smile: It is almost impossible.
How to initialize this reference:

  • create a Message object inside new1() and use it for temporary initialization. This does not work since that object gets destroyed at the end of new() and this reference becomes dangling.
  • Pass a dummy message to new2() so that this reference is initialized temporarily. This also does not work since that object gets sinked inside new2 and the returned reference gets dangling.
  • Return the passed dummy object together with the desired object. This also does not work since if the dummy is not used later after returning from function, that also gets destroyed.

I am not a novice programmer but things like this almost kills the development because normal simple things are impossible in Rust.

struct Message{
   content: f64
}

struct Processor{
    msg_ref: &mut Message // message ref used for storing and processing the messages
}

impl Processor{
    fn new1()->Processor{
        let mut msg = Message{content: 0.23};
        Processor{msg_ref: &mut msg}
    }

    fn new2(dummy: Message)->Processor{
        Processor{msg_ref: &mut dummy)
    }

    fn new3(dummy: Message)-> (Processor, Message){
        (Processor{msg_ref: &mut dummy), dummy)
   }
}

This should probably not have an 'a annotation on &mut self, which is a common anti-pattern. On mobile, so I’ll leave it to others to explain why.

4 Likes

That does indeed fix the example given (playground).

The reason why lifetimes on &mut self is almost always not what you want is that lifetime annotations are supplied to tie together different lifetimes in a function signature. If different lifetimes are wanted for every input reference, and there is either only one input reference or a &self or &mut self reference that should have the same lifetime as an output reference, then the lifetime elision rules take care of those cases and explicit lifetimes need not be written.

In fn forward(&'a mut self, msg_ref: &'a mut Message), the lifetime of the &mut self reference is tied to the lifetime of the msg_ref that the processor contains, meaning the type of the first parameter to forward() is of type &'a mut Processor<'a>. With shared references (&) Rust can be quite forgiving of this odd pattern since it can pop in any compatible lifetime, but unique references (&mut) are invariant, which means the lifetimes they are instantiated with have to match exactly (the nomicon has a lengthy explanation as to why, in case you're interested). Because &mut is invariant, it means the lifetime for &'a mut self must be the same as the lifetime of the contained messages, which are used later in main(), so there is no option but to borrow self for the whole of main(). Without the lifetime annotaion on &mut self, it gets its own separate lifetime, which only borrows for the call to forward() then allows other things to borrow self.

5 Likes

Here are two ways that work:

struct ContainsRef<'a> {
    reference: &'a str
}

impl<'a> ContainsRef<'a> {
    fn new_static() -> ContainsRef<'static> {
        ContainsRef {
            reference: "hi"
        }
    }
    
    fn new_with(s: &'a str) -> ContainsRef<'a> {
        ContainsRef {
            reference: s
        }
    }
}

and an alternative if you're working with a "nullable" reference

struct ContainsRef<'a> {
    reference: Option<&'a str>
}

impl<'a> ContainsRef<'a> {
    fn new() -> ContainsRef<'a> {
        ContainsRef {
            reference: None
        }
    }
}
2 Likes

Thank you so much, it was really helpful especially the last solution with Option is a very good choice.

Thanks a lot for the reply.
After your edits, the program runs but when I just repeat the same procedure by asking to process the same message again, it does not compile.

I would be grateful if you take a look at this link:

References in Rust are typically for short-lived borrows, and not long living data structures. And a core tenant of Rust is that memory reachable by a &mut cannot be aliased -- once something that moves, changes, or even gives the ability to examine msg is created independent of an existing &mut msg, the &mut msg can no longer be valid.

So the example can never work, as e.g. the second call to process.forward(&mut msg) results in two &mut to the same object being alive within forward. [1]

Perhaps you can separate out the long-lived data from the short-lived borrows.


  1. The println also observes msg and thus invalidates the &mut msg and thus the Processor. ↩︎

2 Likes

Thank you so much for restructuring the code.
I think I need to practice more to get used to this way of writing code with Rust, which seems to be a little bit different than other languages.

Thanks again.