Rationale behind exclusive references

help
1

When I first learnt about exclusive references "&mut T", they made sense because of "data races".

However, I didn't question when this would happen (hadn't learnt about parallel and concurrent code at that time), and simply remembered it.

But...what is the reason for &mut T when we running non-parallel or concurrent code, where there aren't data races?

In this case, I supposed there can not be any data races, at least not in the sense I conceptualise them, so the three clauses the book cites (excepting the second one):

  • Two or more pointers access the same data at the same time.
  • At least one of the pointers is being used to write to the data.
  • There’s no mechanism being used to synchronize access to the data.

apply mostly to parallel or concurrent code, but not in all other cases.

One possible reason for &mut T is that it simply possibly makes code easier to follow.

Another reason is in cases where we may have 2 pointers and one of them may cause the data to be moved to a different place in memory (I don't think this is called data race, right?). For this case, AI gave a good example:

let mut v = vec![1, 2, 3];

let first = &v[0];
v.push(4);

println!("{}", first);

But this is still a subset of cases, as it wouldn't affect fixed sized things.

I was wondering of any other important reasons you would add?

2

Because shared mutable state is evil.

The example the AI give is canonical and is common in C/C++ in various forms. But the real reason is, "shared xor mutable" enable local reasoning. Both for the computer and for humans, local reasoning is way easier than global reasoning, and it's just way harder to reason locally when everything can suddenly change under your belt. That's also the reason Rust is so amenable to formal verification.

Also, this enables additional optimizations.

3

That's interesting! I wish it were written there (I could've missed it though).

shared xor mutable

I did not get that though, but I assume "local vs global" means tracking variables nearby vs spread all over the program.

I will do some searching.

4

The ur example of why you want them even for trivial inline code is:

for item in & collection {
  if stale(item) {
    // uses a &mut ref to modify the collection
    collection.remove(item);
  }
}

In most languages this will cause at best you to skip the item after each removed item, and in most native languages you're likely to try to remove garbage after the last removed item, possibly crash when trying to read the item in an empty collection, or even to try to remove random unrelated data as an item.

JavaScript pulls off heroics here to make their collection iteration handle this, but that comes at a performance cost, of course (what doesn't in JS?)

It turns out if you want both performance and safety, you want to be able to write code that doesn't need to continually check preconditions, so you need to allow statically declaring what your expectations are. This includes knowing that while you have a &mut ref, you can safely assume nobody else can have modified it (except via specific, loud escape hatches like Cell)

5

Yes, I agree, but it falls pretty much in the same category than the AI example.

It does not seem so pervasive as to conclude the whole language abides by that rule, if that makes sense? It could just prevent those cases, for example.

6

Standard answer:

Or in summary, (from the article)

Aliasing with mutability in a sufficiently complex, single-threaded program is effectively the same thing as accessing data shared across multiple threads without a lock

Of course, there's also &Cell<i32> if you want "it's shared but it's fine since I'm not using it across threads" -- and I'll worry about re-entrancy problems and such myself.

7

It's the example because it's indicative of realistic code that produces the problem without any need for global analysis.

The point isn't that it stops the caller from writing that, but that it allows the called collection code to stop the caller writing code like that so it doesn't need to worry about it and can just write the simple, obvious, fast code and still be safe.

Whether you should then be able to take a reference to a fixed size array or any other specific exception is really just either needing to look harder for an example that shows how that's bad for some other called code that could cause safety issues or a language design question of how confusing you are ok with the rules being.

In short, Rust wants very simple global rules whenever you're calling external code, so that's where you're most likely to run into borrow checker errors. Local borrow checking can and has been relaxed as they improve the number of cases that is can understand to be safe, but on the other hand it's not going to bother adding special cases when it's easy enough to rewrite like when you're taking separate shared and mutable refs to a local fixed array. There's no reason to do that, just get a mutable Iterator or use indexes which are definitely going to have the bounds check optimized out.

8

It is also important because of enums: Memory Safety's Hardest Problem

This post uses Zig, but it is translatable to rust and would not compile, because mutable references are exclusive.

9

That's exactly this section in the link above, BTW: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/#it-causes-memory-unsafety.

10

Oh sorry i didn't know this was explored in some other blogpost. I only checked that this link hadn't been posted yet.

11

No worries; showing that this isn't just a Rust thing is valuable too.

12

It's aliasing that causes headache:

struct Example {
    a: u32,
    b: u32,
}

fn cross_copy(dest: &mut Example, src: &Example) {
    dest.a = src.b;
    dest.b = src.a;
}

fn main() {
    let x = Example { a:1, b:2 };
    
    cross_copy(&mut x, &x); // Assume, Rust would allow this
}

Now imagine this with references to members in a nested data structure, distributed across different functions/methods.

It is interesting, that virtually all other languages, including the popular "safe" garbage-collected ones like C#, Java, JavaScript, Python, etc. all suffer from this problem.

In my experience, preventing (dangerous) aliasing via exclusive references is the main benefit of Rust resulting in much more robust programs and is mostly underrated and not discussed and showed enough.

13

It's “a subset of cases” in a sense that not all 100% of bugs can be traced to such “logical data races”, but only 90% of them.

Almost every time you look on some stupid bug you may trace it down to the discrepancy between two different things that must be keps synchronized (something is updated on screen, but not in database, something is shown to one player, but not the other, something is not updated after times and a so on).

And in case where bugs are not related to such synchronization issues you may easily ensure that &mut is unique, so that means we are covering 99% of tricky code.

No, the main benefit that Rust brought was the fact that it managed to solve that problem while retaining mutability in its natural form.

The fact that shared mutability is the root of [almost] all evil was known for decades. It's so well-known that the whole branch of programming languages was designed around avoidance of mutability. And these languages were praised, for decades, for all these things that Rust does: “if it compiles then it runs“, etc.

But it's just hard for a lot of people to design programs where mutability is a weird corner case with a special syntax and special rules. Our programs exist to change the world, after all, that the mutability right in the design spec!

And Rust is unique because it popularized another, alternate solution for the exact same problem. The one that people find easier to follow.

14

I've not read the linked posts yet. But so far, my interpretation from the answers is that the other reasons beyond data races are:

  • Invalid pointer access (example at OP),
  • Mutating data that we are also reading. Both struct Example and Vec examples seem to have issues coming from this problem.
    • This makes some sense to me, it'd be like changing a book while we read it (though 1 &mut T can still do this).
15

While it's just the converse of the reasons you've given, I think it's important to emphasize that it allows code to be simpler and faster because it just doesn't need to be worried about those problems.

Like all interfaces, a restriction on the caller is equally a relaxation on the callee, and vice versa.

16

The reasons I gave commute (I am talking about them overlapping in the lifetimes span, as 2 different "permissions" requested), so the converse is just as valid.

But I may not have expressed it correctly, or maybe I didn't understand your comment.'

PS: Maybe you meant that it'd be escalating from read permission to read+write?

17

Yeah, I'm saying you're not wrong, but if you look at them the other way around it's just letting you write simpler, more correct code with only local reasoning.

This is implied by listing the things you doing have to worry about, but I think it's helpful to make it explicit when the question is

Rust could have done what JS etc do and very carefully ensure the validity of iterators to preserve safety and only have exclusive references enforced across threads, and make most of the same claims it currently does, but the exclusivity rule that worked for preventing data races also meant they didn't have to worry about revalidation in the same thread.

18

You mean changing a mutable reference will will impact other references, and updating them is too complex?

19

Yes

20

It prevents some logic bug even in single thread

For example the logic bug

use std::ptr;

fn decide(altitude: *const f32, sensor: *mut f32, new_reading: f32) -> &'static str {
    unsafe {
    let alt_at_decision = ptr::read(altitude);
    ptr::write(sensor, new_reading);
    let alt_at_execution = ptr::read(altitude);

    println!("altitude when decision made  : {:.0} ft", alt_at_decision);
    println!("altitude when command sent   : {:.0} ft", alt_at_execution);

    if alt_at_decision < 10_000.0 {
        "CLIMB"
    } else {
        "CRUISE"
    }
    }
}

fn apply_correction(altitude: *mut f32, hpa: f32) {
    unsafe {
        *altitude += (1013.25 - hpa) * 30.0;
        println!("after correction: {:.0} ft", *altitude);
    }
}

fn print_instrument(altitude: *const f32, label: &str) {
    unsafe {
        println!("{}: {:.0} ft", label, *altitude);
    }
}

fn main() {
    let mut altitude: f32 = 9_800.0;
    let ptr_read  = &altitude as *const f32;
    let ptr_write = &mut altitude as *mut f32;

    println!("pre flight instrument check :");
    print_instrument(ptr_read, "current altitude");
    apply_correction(ptr_write, 1008.0);
    print_instrument(ptr_read, "corrected altitude");

    println!("\nautopilot with aliased pointers :");
    println!("altitude before sensor update : {:.0} ft", altitude);

    let new_sensor_reading: f32 = 11_500.0;
    let command = decide(ptr_read, ptr_write, new_sensor_reading);

    println!("command issued : {}", command);
    println!("altitude in memory : {:.0} ft", altitude);
    println!("correct command should be : {}", if altitude < 10_000.0 { "CLIMB" } else { "CRUISE" });
}

The output :

pre flight instrument check :
current altitude: 9800 ft
after correction: 9958 ft
corrected altitude: 9958 ft

autopilot with aliased pointers :
altitude before sensor update : 9958 ft
altitude when decision made  : 9958 ft
altitude when command sent   : 11500 ft
command issued : CLIMB
altitude in memory : 11500 ft
correct command should be : CRUISE

The bug : the autopilot reads altitude at 9.950 ft, below the 10.000 ft cruise threshold, and decides to CLIMB. But the sensor update fires mid decision and overwrites memory with 11.500 ft. The aircraft is now already above cruise altitude, but the command sent to the engines is still CLIMB. The decision was made against stale data, but executed against new data, the system never reconciles the two. In a real autopilot this is the class of bug that causes controlled flight into terrain or an uncontrolled climb through restricted airspace. Having bug like this is caught at compile time is easier than finding the bug manually at test time

The same bug can happen in discout calculation, bank transaction, patient medicine dose calculation, etc