Help about multiple mutable borrows

#1

Hi,

I'm new to Rust and I'm trying to write a miniflow data struct used in open vswitch. I'm stuck at this error due to multiple mutable borrows. Any suggestion or things to read are welcome. Thanks!

use std::usize;
struct flowmap {
    bits: [u64; FLOWMAP_UNITS],
}

pub const FLOWMAP_UNITS:usize = 2;
pub const MAP_T_BITS:usize = 64;
pub const FLOW_U64S:usize = 84;

impl flowmap {
    pub fn new() -> flowmap {
        flowmap { 
            bits: [0; FLOWMAP_UNITS],
        }
    }
}

struct miniflow {
    pub map: flowmap,   // a bitmap
    pub values: [u64; FLOW_U64S],
}

impl miniflow {
    pub fn new() -> miniflow {
        miniflow {
            map: flowmap::new(), 
            values: [0; FLOW_U64S],
        }   
    }   
}

/* Context for pushing data to a miniflow. */
struct mf_ctx<'a> {
    map: flowmap,
    data: &'a mut [u64], 
    //end: &'a mut [u64], // cause multiple mutable borrow: XD
}

impl<'a> mf_ctx<'a> {
    pub fn from_mf(map_: flowmap, data_: &'a mut [u64]) -> mf_ctx<'a> {
        mf_ctx {
            map: map_, 
            data: data_,
        }
    }   

    pub fn miniflow_push_uint64_(&'a mut self, ofs: usize, value: u64) {
        self.data[0] = value;
        self.data = &mut self.data[1..]; // move self.data to next u64  
    }   
}

fn main() {
    println!("Hello, world!");

    let mut mf: miniflow = miniflow::new(); 
    let mut mfx = &mut mf_ctx::from_mf(mf.map, &mut mf.values);
    
    let ofs = 0;
    mfx.miniflow_push_uint64_(ofs, 1234);
    mfx.miniflow_push_uint64_(ofs + 1, 1234);
}

I got error saying below, but I though I'm not creating another mutable borrower? I'm reusing the same one 'mfx', so I don't understand why this is an error...

error[E0499]: cannot borrow `*mfx` as mutable more than once at a time
   --> src/main.rs:104:5
    |
103 |     mfx.miniflow_push_uint64_(ofs, 1234);
    |     --- first mutable borrow occurs here
104 |     mfx.miniflow_push_uint64_(ofs + 1, 1234);
    |     ^^^
    |     |
    |     second mutable borrow occurs here
    |     first borrow later used here
#2

Ah that's an unfortunate pitfall. You need to change &'a mut self to &mut self.

The reason it fails is that when you specify the lifetime 'a explicitly like that, you're saying that this function borrows the value for as long as that lifetime lives, but since this lifetime refers to the lifetime of the data inside itself, the borrow lasts as long as the value itself does.

One case where you'd want to use the 'a lifetime explicitly is if you want to have a function that replaces the value in the data field inside the struct. Of course, to put a new reference into that field, that reference needs to be using the 'a lifetime explicitly, as you would otherwise not be able to store it somewhere that exists after the function returns.

3 Likes
#3

Hi Alice,

Thanks for your quick reply!
I try it but then hit another error which looks more confusing ...
so remove the 'a

    pub fn miniflow_push_uint64_(&mut self, ofs: usize, value: u64) {
        self.data[0] = value;
        self.data = &mut self.data[1..]; // move self.data to next u64  
    }

and

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
  --> src/main.rs:57:26
   |
57 |         self.data = &mut self.data[1..]; // move self.data to next u64  
   |                          ^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 55:5...
  --> src/main.rs:55:5
   |
55 | /     pub fn miniflow_push_uint64_(&mut self, ofs: usize, value: u64) {
56 | |         self.data[0] = value;
57 | |         self.data = &mut self.data[1..]; // move self.data to next u64  
58 | |     }   
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src/main.rs:57:26
   |
57 |         self.data = &mut self.data[1..]; // move self.data to next u64  
   |                          ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 47:6...
  --> src/main.rs:47:6
   |
47 | impl<'a> mf_ctx<'a> {
   |      ^^
note: ...so that reference does not outlive borrowed content
  --> src/main.rs:57:21
   |
57 |         self.data = &mut self.data[1..]; // move self.data to next u64  
   |                     ^^^^^^^^^^^^^^^^^^^
#4

I think the easiest approach is to store an IterMut. You can call next on it which returns a mutable reference to the first element and increments the internal counter by one.

While this sounds like something that should be able to be done in an easier way, I can't find it.

#5

It turns out you can do this.

1 Like
#6

Thanks a lot for your help!

1 Like
#7

What if one needs to return a immutable reference in addition to that? Like

struct Foo<'a>(&'a mut [u8]);

impl<'a> Foo<'a> {
    fn foo<'b>(&'b mut self) -> &[u8] {
        &self.0[0..1]
    }
}

Oh, it compiles, but I am not sure why it fails on https://github.com/dtolnay/request-for-implementation/issues/7#issuecomment-563387564, I guess maybe because the lifetime was inferred incorrectly?

#8

The code you posted has the elided lifetime like this:

struct Foo<'a>(&'a mut [u8]);

impl<'a> Foo<'a> {
    fn foo<'b>(&'b mut self) -> &'b [u8] {
        &self.0[0..1]
    }
}

playground

but the code on your github corresponds to this:

struct Foo<'a>(&'a mut [u8]);

impl<'a> Foo<'a> {
    fn foo<'b>(&'b mut self) -> &'a [u8] {
        &self.0[0..1]
    }
}

playground

This fails with pretty much the same error. Normally when reborrowing a mutable reference, you will make the mutable reference unusable while the reborrow exists in order to avoid aliasing references, but by linking the reference to 'a you unlink it from self, so you are free to use self during the lifetime of the returned reference. For example, this compiles just fine: (but is horribly undefined behaviour)

struct Foo<'a>(&'a mut [u8]);
impl<'a> Foo<'a> {
    fn foo<'b>(&'b mut self) -> &'a [u8] {
        // transmute the lifetimes
        unsafe { std::mem::transmute(&self.0[0..1]) }
    }
}
fn main() {
    let mut arr = [0, 1, 2, 3];
    let mut foo = Foo(&mut arr);
    
    let ref1 = foo.foo();
    let ref2 = foo.foo();
    
    println!("{:?}", ref1);
    println!("{:?}", ref2);
}

playground

An example of code that uses this kind of lifetime dancing is IterMut, which approximately looks like this:

struct IterMut<'a, T: 'a> {
    // details ommitted, it actually uses raw pointers
    inner: &'a mut [T],
}
impl<'a, T: 'a> Iterator for IterMut<'a, T> {
    type Item = &'a mut T;
    fn next<b>(&'b mut self) -> Option<&'a mut T> {
        // omitted
    }
}

This design means you can call next many times on the iterator and keep the returned mutable references around. In this case it's safe because the returned references are non-overlapping.

Alternatively if the inner reference is not mutable, you are free to return the immutable reference in a non-linked way, because immutable references are allowed to alias.

// compiles!
struct Foo<'a>(&'a [u8]);

impl<'a> Foo<'a> {
    fn foo<'b>(&'b mut self) -> &'a [u8] {
        &self.0[0..1]
    }
}

playground

PS. Please make a new topic in the future, optionally linking to an old one if you feel it may be relevant.

1 Like
#9

Wow, I did not know that one can transmute lifetime. I was thinking that something in std::mem works for this but I did not know it is transmute.

That was the original version, I added mut to &'a [u8] since the in situ parsing requires taking in a mutable slice.

Thanks a lot for the response, I will make a new topic in the future, I added here because I think it may be similar.

#10

Don't actually use transmute for this. It's undefined behavior and your code will horribly break if you use unsafe incorrectly like I did in that example.

3 Likes
#11

Just never play with unsafe {} until you feel you mastered the basic language features and have descent test infra with sanitizer setups on your playground. Not pedantically tested unsafe {} code not only make misbehavior on your project, but also harm the entire Rust ecosystem. Usually, if not always, you can solve your problem with only safe Rust.

3 Likes
#12

Is there any possibility to just transmute the lifetime? I did transmute and it worked, I am not 100% sure but I think it should be safe since the type is the same and the index will probably be correct. If possible, can you please take a look at https://github.com/pickfire/json/commit/6bd0238645f6b880dc6770a5c127c5d340335734#diff-f400f58d5e78694b93db5d4fe5f2a246R675 if it is safe? So far, I don't think the other methods are feasible for this yet.

That is definitely, I try to avoid unsafe {} whenever possible but I am not aware of any methods to do those without unsafe in a possible manner without making huge changes to the API.

#13

Transmuting the lifetime is 100% definitely incorrect. Sure, the types may match, but that's not where the problems come from. Normally when writing Rust programs, the compiler prevents you from doing certain invalid things with some checks, and by using unsafe you can bypass these checks, but this doesn't mean doing the invalid things suddenly become valid. It's just that the compiler trusts you to not do those things.

When you violate these rules, you invoke what's called undefined behaviour. This means that the Rust compiler is allowed to generate literally whatever code it wants, and you can't complain if it does something weird such as inserting a security vulnerability or segfaulting. In the example I posted it did something vaguely reasonable, but this is just a coincidence.

In this case the rule you're invalidating is that a mutable reference must be unique. It is invalid for there to be any other pointers or references anywhere in the codebase with access to the same part of memory, and the compiler assumes this is the case when optimizing your code.

The example code I posted was to illustrate why the compiler doesn't allow the code. As you can see if it was allowed, you could generate immutable references that overlap with the mutable reference without borrowing from it, and this is instantly 100% undefined behaviour.

To reiterate: The fact that your code worked when transmuting is a coincidence, and the compiler is free to emit incorrect code in the future.

3 Likes
#14

To be extra clear about UB. If you invoke UB, the compiler is allowed to emit any code. It may leak provate information, segfault, or more insidiously appear to work correctly for a while, then randomly crash much later. unsafe allows you to write the most subtle and horrible to debug bugs there are if you are not careful.

Before you write any unsafe code, please read the nomicon

1 Like