Mutate from multiple threads without interior mutability?

I saw some people do something like this one:

let file = Arc::new(Mutex::new(File::create("foo.txt").unwrap()));

Here they use Mutex just because Arc require some value that's immutable (or mutate some value, through interior mutability). But what is the point of this ?

Its something like this: There is a public toilet and there are 4 peoples in line, But only-one people can use it at any point of time, A people take 2m, so they all finished in (4 peoples x 2m) = 8m.

Forget parallelism, This is not even concurrent!

I can't tell about this one, But something mutate some value also could be thread same. But need to work with multiple ownerships (like passing values to many threads and safe to mutate, without any sync primitive or locking system, like Mutex), So How to do ?

Side Note: I create a static mut (because they are static and doesn't require any atomic ref counter to send value in threads), and mutate it every time with ugly unsafe block(though its safe to mutate, if used correctly), which worked well! But not practical.

:

They don't! &File (shared reference to a File) implements Read and Write, so you can easily "mutate" a File concurrently without interior mutability:

let file = /* get a `File` somehow */;
let file_ref = &file;
crossbeam::scope(|s| {
    s.spawn(|_| {
        let mut buf = [0; 100];
        let _ = file_ref.read_exact(&mut buf);
    });
    s.spawn(|_| {
        let mut buf = [1, 2, 3, 4, 5];
        let _ = file_ref.write_all(&buf);
    });
});

[edit: those should be move closures and the declaration of file_ref should read let mut file_ref = &file;, thanks @parasyte]

Or to put it another way, File has built-in interior mutability, implemented by the OS and not by the standard library.

1 Like

static mut is almost certainly not what you want because it trivially triggers UB by allowing a safe API to hand out multiple mutable borrows that live for 'static. I've written about this before: Mutable statics have scary superpowers! Do not use them

3 Likes

My mistake, It (std::fs::File) only required mut Self when used with use std::io::*, And file need to work with std::io utility module, So we can say that file require mutability.

Right, and I would also point out that if that's the desired behavior, well, you don't need threads at all: you could just use a mutable integer and a for loop. So I ask you the same question: "what is the point of this?" It is true that Mutex may be used as part of a concurrent algorithm for reasons other than memory safety, like algorithmic correctness - is that what you mean?

I'm not sure I understand the question. Any mutable state shared between threads needs some kind of synchronization strategy - whether that's a Mutex or atomics or something special and complicated you do yourself using unsafe (concurrent data structures often have bespoke synchronization patterns). But you can't just ignore it; data races are UB.

It is a virtual certainty that static mut is not what you want. If you are able and willing to share your code, someone here can probably suggest a better way to approach the problem.

4 Likes

Those .read_exact() and .write_all() in the example code are methods of traits declared within the std::io module. You don't need &mut File for them(Read, Write), as the kernel already handles concurrency for file system access.

2 Likes

I'm not sure I understand the point of impl Read for &File (and Write) in the context of threads, because all of the useful methods take &mut self. The code in Mutate from multiple threads without interior mutability? - #2 by cole-miller does not even compile:

error[E0596]: cannot borrow `file_ref` as mutable, as it is not declared as mutable
  --> src\main.rs:10:21
   |
6  |     let file_ref = &file;
   |         -------- help: consider changing this to be mutable: `mut file_ref`
...
10 |             let _ = file_ref.read_exact(&mut buf);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable

error[E0596]: cannot borrow `file_ref` as mutable, as it is not declared as mutable
  --> src\main.rs:14:21
   |
6  |     let file_ref = &file;
   |         -------- help: consider changing this to be mutable: `mut file_ref`
...
14 |             let _ = file_ref.write_all(&buf);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable

error[E0524]: two closures require unique access to `file_ref` at the same time
  --> src\main.rs:12:17
   |
8  |         s.spawn(|_| {
   |                 --- first closure is constructed here
9  |             let mut buf = [0; 100];
10 |             let _ = file_ref.read_exact(&mut buf);
   |                     -------- first borrow occurs due to use of `file_ref` in closure
11 |         });
12 |         s.spawn(|_| {
   |           ----- ^^^ second closure is constructed here
   |           |
   |           first borrow later used by call
13 |             let buf = [1, 2, 3, 4, 5];
14 |             let _ = file_ref.write_all(&buf);
   |                     -------- second borrow occurs due to use of `file_ref` in closure

Ah, probably need to change those to move closures, my bad. The point of impl Read for &File is that the signature of <&File as Read>::read is then (&mut &File, &mut [u8]) -> std::io::Result<usize>, so you only need a &mut &File, which you can get from &File by borrowing.

3 Likes

That's the trick. :slight_smile: Also file_ref needs to be mutable, as mentioned in the error.

You understand it correctly, and that is, If a file only could be accessed once at any point of time. No matter how many threads accessed that file, It doesn't make any sense...

I'm not sure I understand the question

Mutuality also could be safe.

Imagine a restaurant has N (lets say 4) numbers of waiters, And one manager, Every waiters has unique IDs (0..N), The Manager take waiter ID number, and give them some task, they do there task and submit it with there ID to the Manager,

The roll of the manager is simple, he manage workload, give tasks to waiters, and uniquely maintain there work report using there ID.

A waiter also can quite the job, lets say, Waiter 3 left, So new waiter could take his ID,

There (waiter as Thread) task is unique, The manager has to take all the responsibility to make there task unique.

I found two way:

  1. create a static mut of manager (so that waiter could came and take there task)
  2. pass manager instance to the waiters, So manager instance need many ownership, we could use Arc but without something like Mutex or RwLock

A common approach to this sort of problem is to pass messages through a channel instead of giving each worker a direct reference to the manager. In this setup, the manager's work is serialized on its own thread, but doesn't necessarily block any worker: Each channel can have a buffer of future jobs to do until the manager gets around to processing a response.

(cf. crossbeam_channel and std::sync::mpsc)


Also, take a look at rayon, which is designed to distribute array computations across multiple threads.


Your example code is holding the lock for c longer than it needs to. Explicitly releasing the lock after you've updated the value reduces the time back down to 2:

        threads.push(thread::spawn(move || {
            let my_c = {
                let mut c = c.lock().unwrap();
                *c += 1;
                *c
            };
            do_some_work();
        }));

Just use UnsafeCell if you make sure there's no mutable alias. It's still interior mutability, but I'm quite sure by without interior mutability, you just means without overhead.

1 Like

Assuming your question is why Arc (usually1) requires interior mutability in order to mutate its contents, and not why Mutex in particular is used here instead of other thread-safe interior mutability primitives (like atomics) based on your title. This is easy to answer in Rust:

  • The type of so-called "mutable reference", something like &mut T, is actually more accurately a "unique" reference. Having &mut T access to a value means that there cannot be any other (live) reference (&T or &mut T) to the same value at the same time. Once you embrace this definition (and realize the slight misnomer “mutable reference”), you might already get a feeling for why Arc<T> doesn’t give you access with &mut T to its contents; because Arcs can be shared, but &mut T must be having unique access.
  • “Interior mutability” usually refers to any kind of API that offers mutation (this does not necessarily involve creating or handing out “mutable references”) through so-called “immutable references”, i.e. &T, more accurately called “shared” references. So anything that mutates (some “field” of) a value e.g. through a &self method is considered “interior mutability”. This means, the word “interior mutability” in Rust effectively just refers to “mutating through some form of shared access”.
  • An Arc can be replicated; that’s its whole point. Cloning an Arc is an operation that takes &Arc<…>. If you are ever going to do any mutation of some value through an Arc, you’ll automatically be able to do mutation through a shared reference, since even if the API for mutating the contents did require the Arc<…> to be owned, or passed by “mutable reference” &mut Arc<…>; when you start with &Arc<…> you could always clone it first and you’d still be able to mutate your value.
  • So by definition mutating the contents of an Arc can only work with “interior mutability”.
  • Also your premise to “mutate from multiple threads without interior mutability” is impossible, by the same token: “mutating through a shared reference/pointer” is by definition “interior mutability” in Rust, and “mutat[ing] from multiple threads” obviously requires some form of shared reference/pointer to the thing you’re mutating.

1 I am ignoring API such as Arc::get_mut or Arc::make_mut here, since they only mutate the contents of the Arc when you haven’t cloned the Arc yet [correction:] no other clones of the same Arc exist; which – while arguably being a way to mutate the contents of an Arc without interior mutability – is a boring special case. The abovementioned argument of “you could clone the Arc first, so it’s always interior mutability” doesn’t apply either, because cloning the Arc has the effect that those two methods no longer allow you to mutate anything inside of the original Arc.

6 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.