Does Interior Mutability Contradict Rust's Borrowing Rules and Safety Philosophy?

Sometimes a type needs to be mutated while having multiple aliases. In Rust, this is achieved using a pattern called interior mutability.

Does this approach bend the borrowing rule, which states that no mutable reference is allowed when shared references exist?

With interior mutability, borrow checking is shifted from compile-time to runtime. If borrow checking still occurs, why do we move it from compile-time to runtime at all?

Does the existence of interior mutability go against the philosophy of Rust's borrow checking and memory safety — especially the benefits discussed here?

2 Likes

Interior mutability doesn't necessarily shift borrow-checking from compile-time to runtime. That's what happens when you use RefCell, but other kinds of interior mutability don't have that drawback. For example, Cell or AtomicI32 can be modified through a shared reference, and these modifications do not have a runtime error-case.

You may like this article:

11 Likes

No, all sound interior mutability patterns will respect that rule.

Because compile-time borrow checking is limited. There will always be situations where a program satisfies the borrow checker rules at runtime but the compiler can't verify that at compile-time (this is actually formally proven, see Rice's theorem).

By using interior mutability you lose some of Rust's advantages, but it is still memory safe.

3 Likes

No.

Seems to me that often it is impossible to do borrow checking at compile time. For example a structure might need to be updated by two threads or updated by one thread and read by another. Clearly enforcing the borrow checker rules on that structure at compile time would make it impossible to write such threads. Unless we employed unsafe I guess.

So we give the struct to an Arc or whatever, clone() copies of that Arc and distribute them to our threads. Now the guts of the Arc can enforce the read/write access at runtime and ensure that they do not happen at the same time. Essentially the Arc enforces the mutability rules. It wraps all that unsafe code for us.

I don't much like the term "interior mutability". What has happened here is that we have given ownership of our struct to that Arc or whatever and we then ask that new owner to mutate it.

Anyway, I'm pretty sure you cannot make memory use errors with such interior mutability. Barring bugs in their implementation.

1 Like

It's not correct to say that mutable references cannot coexist with immutable ones. If that were the case, Rust would be borderline unusable, because you'd be only be able to borrow from unborrowed local variables.

The purpose of the borrow checker is to protect against memory safety violations. Most importantly, a piece of data must not be simultaneously modified by two different threads, and the data must not be invalidated while something else is trying to access it. The solution is to create a sort of stack of borrows on each byte in memory. Borrows on the lover levels of the stack cannot be used until the ones on the higher levels are popped (or rather, trying to use a borrow at a lower level pops off all higher ones, and you get a borrow checker error if you try to use any of the popped borrows). Pushing a new borrow on that stack is called "reborrowing": you can only create a new borrow starting from the previous top one (since you can't access anything at the lower levels, including the original place of the data).

Consider the following example:

// We create a new place, containing a value `0u32`. 
// The stack of borrows is `[x]`.
let mut p1 = 0u32;
// Now we push a new mutable borrow of the place `p1`
// onto the stack. 
// The stack is `[p1, p2]`.
let p2 = &mut p1;
// `foo` is defined below. How can we use `p2` twice 
// when `&mut T` is not `Copy` or `Clone`, and it should
// have been moved into `foo`? The answer is reborrowing!
// The compiler implicitly creates a new reborrow of `p2` 
// for each function call.
// The stack of borrows inside this function call 
// is `[p1, p2, &mut *p2]`.
foo(p2);
// And here it is also `[p1, p2, &mut *p2]`! The old reborrow
// was popped after the previous return from `foo`.
foo(p2);

fn foo(p3: &mut u32) {
    // We create a mutable reborrow, just because we can.
    // The stack is `[p1, p2, p3, &mut *p3]`.
    let p4 = &mut *p3;
    // Here we push an immutable reborrow of `p4` onto the
    // stack. The place cannot be mutated while this reborrow
    // is active. 
    // The stack is `[p1, p2, p3, p4, &*p4]`.
    let p5 = &* p4;
    // We use `p4` to read the value.
    pritnln!("{p5}");
    // We want to mutate the value. This requires us to pop off
    // the last immutable reborrow.
    // Since we do the mutation via `p3`, we must also pop off
    // the mutable borrow `p4`.
    // The stack is now `[p1, p2, p3]`.
    *p3 = 5;
    // If we try to use `p4` or `p5`, we get a borrow checker
    // error, because they were already popped.
    // dbg!(p4, p5);  <-- compile error
}

Note that we didn't really discuss mutability in the above. The important part was the borrow stack.

So how does mutability and UnsafeCell fit into this picture? Normally the compiler assumes that the pointee of &T cannot be modified in any way. This is useful both for compiler optimizations and for programmer reasoning. It is also important for thread safety: read-only values can be freely shared between threads, because concurrent reads cannot violate memory safety if there are no writes.

UnsafeCell allows us to mark a region of memory as "this place may be modified even if borrowed with a &". This means that reads and writes now require special synchronization. But note that it doesn't violate the borrowing rules described above. You still cannot have multiple active mutable borrows of the same region. Trying to create multiple live mutable borrows, even if they are not used to concurrently access the memory, violates compiler's assumptions and thus violates memory safety.

Actually, the model described above still doesn't really describe what's really happening with the borrows. First, it doesn't apply to the runtime semantics. For that you need to consult the Stacked Borrows and Tree Borrows models. The thing described above is just a conservative approximation used in the static analysis.

Second, even the static model is incomplete. There are cases where multiple mutable borrows can coexist. For a long time it was a kind of hack in the compiler used to support async functions, but now there is an official way to opt-out of &mut uniqueness using the UnsafePinned type (not yet stable), just as you can opt-out of immutability for &T using UnsafeCell.

In general, you need to remember that the borrow checker isn't a source of truth, but a conservative approximation to the true rules for pointer aliasing. This makes it tractable to implement, verify and reason about, but there are plenty of cases which cannot be fitted in that mold and require unsafe code. Some of the borrow checker assumptions are sufficiently deeply ingrained in the compiler that you need special syntax to opt out of those.

4 Likes

I'm banging my head against the wall trying to wrap my mind around this!

So without interior mutability, the concurrency primitives would be impossible because of Rust's borrowing rules. Rust uses types like UnsafeCell (which is the core type behind interior mutability) to bypass these borrowing rules safety, enabling mutation without violating Rust's memory safety guarantees.

Is it correct?

The benefits listed in the other thread were of 2 general categories

  • safety from “concurrent access”-related issues
    • which could be data races in multi-threaded contexts
    • but also issues like avoiding double-frees, iterator-invalidation, etc…
  • compiler optimizations from known-non-aliasing pointers

The philosophy of Rust on the first one is that safety is always important and we should make safe Rust as capable as safely possible. This is why most interior-mutability APIs provide safe APIs.

The unsafe ones, e.g. UnsafeCell are clearly marked as unsafe, and most commonly used only in places where safe alternatives had unacceptable drawbacks, or where new safe APIs are defined. In fact, UnsafeCell is somewhat of a special built-in type that must be used internally for any interior-mutable type in Rust.


Regarding the second point, the benefit of compiler optimizations relating to aliasing in Rust is pretty impressive, since it turns out to be a sort-of effortless side-benefit you get automatically (whereas e.g. the restrict keyword in C is not used very often, and requires careful analysis by the programmers that they aren’t using it wrong). In fact, Rust code creates so many instances of such no-aliasing markers that multiple LLVM bugs had to be addressed over some months (or was it years) before we could make full use of it; as far as I understand that’s because Rust uses this “noalias” feature in LLVM so extensively that corner cases were being hit and noticed which stuff like manually annotated C code just never practically reached.

Still, the philosophy here would be about correctness first and foremost; for safe Rust code, the compiler should only ever do optimizations (let LLVM do optimizations) that are correct:

IIRC in the context of interior mutability, for references to interior-mutable types, certain aliasing-related (or immutability-related) optimizations will be turned off; which is also one of the reasons why using the special UnsafeCell type (or wrappers thereof) is mandatory.

Still, this seems to be following the principle of “pay only for what you use”, so you only pay for any lost optimization opportunities when you actually explicitly need&use the shared mutability.

It is a similar story with atomics; usage of which also prevents certain compiler optimizations, in order not to do incorrect optimizations. Here, the story of Rust is comparable to other languages, as far as “pay only for what you use” goes, because atomics are opt-in all the same. A core difference however is that in safe Rust also still ensures safety, so you can’t simply forget to use atomics and run into data races by naively mutating a shared value with ordinary non-atomic operations.

7 Likes

Yes.

I guess the point is that the Rust borrowing rules are ALWAYS in force. The only question is whether they are enforced at compile time when using simple references &/&mut or at run time using Arc<> or whatever.

2 Likes

Also note that interior mutability doesn’t necessarily look like run-time borrow checking at all. For example, channels provide communication between threads that is (for most channel types) about transferring ownership of values, never borrowing them. But there must be interior mutability to manage shared state inside the channel implementation.

7 Likes

The terminology about references is IMHO a bit misleading, because existence of interior mutability means that & isn't always immutable, and &mut isn't the only way to mutate things. They are better thought of as & being shared, and &mut as exclusive.

Interior mutability does allow having shared-mutable data. It would be pretty bad if that was simply allowed by default (like it is in many languages), but Rust has marked the unchecked UnsafeCell as unsafe, and requires all other wrapper types around it (like Mutex or RefCell) to ensure it cannot cause data races from safe code. And because you have to use an explicit wrapper type for shared mutability, you can't do it by accident, so shared mutability gets used intentionally, hopefully with the extra care it requires.

3 Likes

The key insight for me to stop being confused in the same way I think you are was to learn that UnsafeCell is not a normal type. It is special-cased by the compiler in significant ways. It is a so called "lang item", which means it is treated specially by the compiler.

You can think of UnsafeCell as being special syntax disguised as a regular type. When the compiler encounters it it knows that it can't assume that shared references will not mutate data.

There are many examples of types that the compiler treats specially. Box is another example of a lang item that the compiler treats specially.

2 Likes