Why can't `self.lambda` be `FnMut` when `self` is borrowed immutably?

help
4

The short answer is that from an outer &-shared reference, you cannot get an inner &mut-exclusive reference, unless you use:

  • either a dynamic/at-runtime synchronisation primitive, such as a Mutex;
  • or unsafe to assert that you are, yourself, somehow avoiding any unsoundness that would otherwise ensue.

Mainly, you cannot just go from & to &mut, not even via a transmute().

And yet:

  • you have access to a self: &Analyzer<F> argument in the .pipeline() method;
    • i.e., ".pipeline() can be called concurrently";
  • the postprocess: F field requires &mut to be used, as per the very definition of FnMut.
    • i.e., ".postprocess cannot be called concurrently; it must be called sequentially".

These are just the mere type definitions involved here, you really cannot contest or refute any of this. Do notice, however, how I have not used the word mutation even once in all of the above, but rather, talked of &mut/FnMut exclusivity / sequentiality / it not being concurrency-friendly.

But to humour you, let's try to see what happens if we force Rust's hand to "look the other way" and let us do that. For the sake of the example, since non-parallel concurrency is a bit more convoluted to exploit, I'll take advantage of the fact that your data structure is safe to share across threads (i.e., Sync), whenever F is, and use your concurrent .pipeline() API to thus run it in parallel.

By doing so, and by forcing the hand to get ahold of an &mut postprocess: &mut impl FnMut handle within .pipeline(), I will showcase how then, your API can be used without any unsafe code on the caller side, and yet running into a data race, i.e., Undefined Behavior :warning:.

#![feature(sync_unsafe_cell)]

pub struct Analyzer<F> {
    postprocess: ::core::cell::SyncUnsafeCell<F>,
}

impl<F: FnMut(i32) -> i32> Analyzer<F> {
    fn process(&self, n: i32) -> i32 {
        n + 1
    }
    pub fn pipeline(&self, n: i32) -> i32 {
        let n = self.process(n);
        // 1. let's assume we were allowed to get `&mut postprocess`
        // e.g., here, using (unsound) `unsafe` to make Rust look
        // other way.
        let postprocess: &mut F = unsafe { &mut *self.postprocess.get() };
        postprocess(n)
    }
}

fn main() {
    let mut total = 0;
    let postprocess = |n| {
        for _ in 0..n {
            total += ::std::hint::black_box(1);
        }
        0
    };
    let analyzer = Analyzer {
        postprocess: ::core::cell::SyncUnsafeCell::new(postprocess),
    };
    // 2. then, the following code compiles fine:
    ::std::thread::scope(|s| {
        _ = s.spawn(|| analyzer.pipeline(4999));
        _ = s.spawn(|| analyzer.pipeline(4999));
    });
    dbg!(total);
    assert_eq!(total, 2 * (4999 + 1)); // Fails!??
}

Even though UB can be triggered without it being observable in a normal run, I have tweaked the code so that we do observe a symptom of it on the version of the compiler the code is run, wherein a total: usize value which is supposed to be incremented 5000 times, twice, does not yield the expected deterministic output of 10000, but rather, some non-deterministic value below it.

  • If you run the Playground you'll get:

    [src/main.rs:37:5] total = 6033
thread 'main' panicked at src/main.rs:38:5:
assertion `left == right` failed
  left: 6033
 right: 10000

To be convinced of this being UB, the right tool for the task is a special interpreter of Rust code, called Miri, which can detect most (but not all!) cases of UB.

On the top-right UI of the Playground, under the Tools menu, you will find the option to run Miri:

error: Undefined Behavior: Data race detected between (1) non-atomic write on thread unnamed-1 and (2) non-atomic read on thread unnamed-2 at alloc1071. (2) just happened here

  --> src/main.rs:29:17
   |
29 |                 *total += ::std::hint::black_box(1);
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |                 Data race detected between (1) non-atomic write
   |                 on thread `unnamed-1` and (2) non-atomic read
   |                 on thread `unnamed-2` at alloc1071. (2) just
   |                 happened here
   |
help: and (1) occurred earlier here
  --> src/main.rs:29:17
   |
29 |                 *total += ::std::hint::black_box(1);
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  • Demo (1.81 nightly)
2 Likes