It is `*mut UnsafePinned<[u8]>` sound for shared mutating memory (in process or inter process)?

help
1

Shared memory is often useful in IPC or other lock-free data structures. But it is hard to use shared memory soundly without breaking the memory safety and aliasing rules.

Image we have a SeqLockWriter<T: zerocopy::IntoBytes>and SeqLockReader<T: zerocopy::TryFromBytes>[1] [2], we need to hold some form of &T or *T inside SeqLockReader<T> and some form of &mut T or *mut T inside SeqLockWriter simultaneously, because the memory region may be mutated by others, wrappers at at lease UnsafeCell-level is required, but the aliasing rules are still broken (when dereferencing the pointer).
So as the UnsafePinned make its way into nightly, can we soundly implement SeqLockWriter and SeqLockReader with it:

struct SeqLockWriter<T: zerocopy::IntoBytes> {
    seq: *const AtomicUsize,
    data: *mut UnsafePinned<T>,
}

struct SeqLockReader<T: zerocopy::TryFromBytes> {
    seq: *const AtomicUsize,
    data: *const UnsafePinned<T>,
}

, when the writer and reader are in the same process, or they are in different processes but the memory region of T is shared?

Assuming read-write race can be fixed by AtomicPerByte RFC or atomic-memcpy crate.

Related:
How unsafe is mmap? - #20 by newpavlov
Abstraction of shared memory
Ralf Jung's answer about UnsafeCell

  1. SeqLock is a reader-writer consistency mechanism with
    lockless readers (read-only retry loops), and no writer starvation. ↩︎

  2. zerocopy::IntoBytes are required to opt-out the padding bytes which is unsound to read from. ↩︎

2

Cc @RalfJung on this one I feel.

Shared memory IPC is generally a tricky question with a lot of yet to be determined semantics, as you seem to already have found out.

Seqlocks are also problematic in the Rust AM, it is not something that can really be expressed currently (atomic memcpy is the relevant keyword to search for for that)

3

I am quite confused by the entire question.^^

Inter-process concurrency is basically identical to intra-process concurrency so we can focus on that. You must use atomic accesses for all racy accesses; plain reads and writes are always assumed to not race (and we have UB otherwise). Whether you can soundly have these types depends on what they actually do so it's impossible to answer that question without seeing the code. However, as @Vorpal mentioned, seqlocks are unfortunately currently not possible to soundly implement in Rust (or C++). This is clearly something we should fix, and there's an RFC that would achieve this.

Why do you think you need UnsafePinned? We have plenty of existing concurrency primitives that are doing just fine without UnsafePinned. An API that provides an &mut and and the same time any other reference is clearly unsound, and UnsafePinned does not fix that, as mentioned by the docs:

Many functions that work generically on &mut T assume that the memory that stores T is uniquely owned (such as mem::swap). In other words, while having aliasing with &mut Wrapper is not immediate Undefined Behavior, it is still unsound to expose such a mutable reference to code you do not control! Techniques such as pinning via Pin are needed to ensure soundness.

Please restate your question with a lot more detail and explain what exactly the unsoundness is you are worried about, with a fully concrete example.

4

Sorry about that/
I might got into some x-y problem and oversimplified the example as I ripped it from a larger private project.
The core part of the problem is, can we implement a address-free SeqLock soundly based on AtomicPerByte RFC? If we do, how to represent data on reader and writer?
Here is the more concrete question: Is this`SeqLock` based on atomic memcpy sound?

5

I don't know what you mean by "address-free". But generally, seqlocks should be possible with per-byte atomic memcpy, yes -- that's the point. I don't think UnsafePinned would be involved, it'd look mostly like a normal mutex/rwlock:

struct SeqLock<T> {
  seq: AtomicUsize,
  data: UnsafeCell<T>,
}

The reader/writer guards can just have a &SeqLock, just like the RwLock guards do.

6

Ah, I think I see the problem -- or rather, a SeqLock would simply not have reader/writer guards. It doesn't function like a normal lock. It would function more like crossbeam's AtomicCell.

If you mean an seqlock that provides writers with in-place access to the data... yes that does seem tricky indeed. (Readers cannot get in-place access, they will always need to make a copy.)

7

There are more contexts in Is this `SeqLock` based on atomic memcpy sound? .Address-free refers to suitable for communication between processes using shared memory, and the reader and writer are in different threads (or say, processes with shared memory) so sharing a &SeqLock is not possible.

8

Oh writers with in-place access to the data is a interesting point to focus on. I'd add it as question #5 at the new post.

9

Why? You just place a SeqLock in the shared memory, and then every process creates its own &SeqLock. They see the same value at different addresses but that's no problem, is it?

10

:face_with_peeking_eye: Oh indeed I missed that point.

11

So in conclusion, only UnsafeCell is required for SeqLock or SeqLockWriter/SeqLockReader pair as

  1. We can get &UnsafeCell<T> out of &SeqLock<T>, and then get *mut T via UnsafeCell::get`
  2. We can do a byte-wise atomic store and load for &AtomicPerByte<T> (per RFC) or for *mut T and *const T (per atomic-memcpy crate), with no &mut and & aliasing occured.

Is my perspective correct?

12

Yes, for an API where writers make a full copy (as in crossbeam's AtomicCell, and as in your sketch), that sounds right. Writers that hand out an &mut to safe code don't work. Not sure if that's something seqlocks even want though?

More like &UnsafeCell<T>, if you have &T for a general T it must be immutable so there's never a reason to use atomics.

1 Like
13

Corrected it as &AtomicPerByte<T>, which is very likely to be UnsafeCell<MaybeUninit<T>>.

14

:+1: