Pointer to usize transmute lead to unexpected errors

levs57 · July 27, 2024, 9:06pm

Hi, I'm trying to do pointer arithmetic, and apparently because raw pointers are not Send, I am transmuting them to usize and back to send them into multiple threads.

However, for some reason I have stumbled onto a bug. Sadly, I can not reproduce it in a minimal example (this seems to be edge case of some sort), but when I'm transmuting *const *const T into usize, and then reverse the operation inside of a (parallel) iterator, it gives nonsensical error message (attempt to access vector of length 729, but index is 0).

I have replaced the transmutes with pointer casts (as usize and as *const *const T) and it seems to work now, but I wonder

What could cause such bug? I've read a bit about pointer provenance, but didn't understand it well enough to judge if it could be the reason.
What is the "correct" way of doing multi-threaded pointer arithmetic, considering that const * and mut * are not Send?

cuviper · July 27, 2024, 9:27pm

You can wrap the pointer in a newtype that does implement Send, like:

github.com

rayon-rs/rayon/blob/7b3ef64f3527f30446c9f91ac2e33df2f639cf3a/src/lib.rs#L129-L160


      
          /// We need to transmit raw pointers across threads. It is possible to do this
          /// without any unsafe code by converting pointers to usize or to AtomicPtr<T>
          /// then back to a raw pointer for use. We prefer this approach because code
          /// that uses this type is more explicit.
          ///
          /// Unsafe code is still required to dereference the pointer, so this type is
          /// not unsound on its own, although it does partly lift the unconditional
          /// !Send and !Sync on raw pointers. As always, dereference with care.
          struct SendPtr<T>(*mut T);
          
          // SAFETY: !Send for raw pointers is not for safety, just as a lint
          unsafe impl<T: Send> Send for SendPtr<T> {}
          
          // SAFETY: !Sync for raw pointers is not for safety, just as a lint
          unsafe impl<T: Send> Sync for SendPtr<T> {}
          
          impl<T> SendPtr<T> {
              // Helper to avoid disjoint captures of `send_ptr.0`
              fn get(self) -> *mut T {
                  self.0

This file has been truncated. show original

levs57 · July 27, 2024, 10:58pm

Yeah, that seems a bit more convenient than usize conversions. Thanks.

paramagnetic · July 28, 2024, 6:31am

Yeah, you should never transmute pointers. Transmute means "re-interpret bit-by-bit". That operation is basically completely nonsensical outside of a very limited set of "pure value" types (such as primitives and direct, owning aggregates thereof).

When you start bringing indirection into the picture, all hell breaks loose. In particular, people often confuse the following two scenarios:

transmuting a value of type T to a value of type U. This can, for example, be realized by obtaining a pointer of type *const T, then casting (not transmuting!) the pointer to *const U, finally reading it to obtain a value of type U. This is usually what people want. It is performed by casting the pointer, because that is an operation that preserves provenance, and it's sometimes the only correct option. E.g. transmuting a fat pointer to a thin pointer is simply not possible because they have different sizes; you need the compiler to understand the conversion operation in order to throw away the correct half of the pointer-metadata pair.
directly transmuting (not casting!) a pointer of type *const T itself into a pointer of type *const U. This is basically almost never correct and almost never what people want.

system · October 26, 2024, 6:32am

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Why isn't ptr::addr const? help	14	499	January 5, 2023
Problematic integer-to-pointer transmute help	2	588	November 28, 2023
Why const T, mut T : !Send + !Sync by default help	2	1067	December 1, 2021
Why can't use transmute for int-to-pointer conversion? help	6	173	February 21, 2025
Reference cast to raw pointer help	3	3801	March 17, 2021

Pointer to usize transmute lead to unexpected errors

Related topics