It feels wrong to be asking this because I'm sure Rust has a good reason for not allowing me to do it, but is there a way to move a struct containing a non-Send object to another thread?
In this case I am receiving some data on a TcpStream, writing it to shared memory using the shared_memory crate, and then I want to pass that shared memory off to a different thread for further processing.
The root of the problem is that shared_memory::SharedMemRaw at its core contains *mut std::os::raw::c_void
I know that the sending thread is done with the shared memory before trying to send to the receiving thread over mpsc::channel, but Rust doesn't know this.
A struct that is non-send is not safe to send between threads. You should re-think your design. That being said, shared_memory is inherently unsafe. What you probably want to do is put your shared memory reference/buffer inside an Arc<Mutex<YourThingThatIsn'tSendSync>>. Then the struct will be Send because and Arc<Mutex<...>>> is send.
Not sure if you mean wrap my whole struct in Arc<Mutex<...>> or just the field of the struct that is not Send. I updated the playground example to do both of them, and it looks like Rust sees through that.
Unless I misunderstood?
EDIT: Trace from playground
error[E0277]: the trait bound `*mut std::os::raw::c_void: std::marker::Send` is not satisfied
--> src/lib.rs:12:18
|
12 | let thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut std::os::raw::c_void` cannot be sent between threads safely
|
= help: the trait `std::marker::Send` is not implemented for `*mut std::os::raw::c_void`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Mutex<*mut std::os::raw::c_void>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::Mutex<*mut std::os::raw::c_void>>`
= note: required because it appears within the type `NoSendStruct`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Mutex<NoSendStruct>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::Mutex<NoSendStruct>>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::mpsc::Receiver<std::sync::Arc<std::sync::Mutex<NoSendStruct>>>`
= note: required because it appears within the type `[closure@src/lib.rs:12:32: 17:6 receive:std::sync::mpsc::Receiver<std::sync::Arc<std::sync::Mutex<NoSendStruct>>>]`
= note: required by `std::thread::spawn`
You need to wrap only the field, but, then the whole struct must have "Send" implemented for it. So, by putting the struct field in an Arc<Mutex<>> guard you've made it safe to send/sync the contents of that field from the perspective of the struct, but, the compiler doesn't know you've made it safe until you do:
unsafe impl Send for YourStruct;
That lets Rust know it is now safe to send your struct across thread boundaries.
In general, it’s not safe to move a value that’s !Send to another thread. It may, for example, be using some thread-local state or it may be a smart pointer type like Rc that shares state across multiple Rc instances and isn’t prepared to handle those updates atomically/threadsafely.
Rust automatically determines whether a type is Send and/or Sync. Anything that has a raw ptr inside, which is what you have here, is considered !Send and !Sync. This isn’t because it automatically means the type is unsafe in Send/Sync terms, but rather more like a lint for the code author: they need to determine the safety themselves, and then if they’re safe, manually impl Send and/or Sync, as appropriate, for the type.
In the case of the shared_memory crate, it’s possible that the author simply didn’t take this into account - it’s easy to miss the auto-derived aspect of these traits unless you explicitly look/test for it. It’s also possible that they’re !Send for a good reason. In either case, you may want to ask about this in that project’s issue tracker.
That said, if you know that a type that’s !Send is actually sendable, then you can wrap it in a newtype and manually impl Send for the newtype.
The Rustonomicon about manually implementing Send was especially helpful. This comment:
However raw pointers are, strictly speaking, marked as thread-unsafe as more of a lint . Doing anything useful with a raw pointer requires dereferencing it, which is already unsafe.
Here's a playground that illustrates what I ended up doing.
And a snippet of my code:
use shared_memory::SharedMemRaw;
pub struct SharedMemoryManager {
shm: SharedMemRaw,
...
}
/// Extensive doc indicating why this was done, how/why it's unsafe,
/// and how to take special care to not leave the pointer behind when
/// sending between threads
unsafe impl Send for SharedMemoryManager {}
I opted not to wrap SharedMemRaw in Arc<Mutex<>> within SharedMemoryManager because an instance of the struct will only be present in one thread at a time.
That's false, Mutex is only Send if its contents are Send.
The name Send is a bit misleading, since it only covers one aspect: moving things into another thread. But accessing something from two different threads is basically the same thing, and that's what Send allows. The additional ability that Sync gives you is that you can access it from those threads at the same time.
Likewise, I think that Arc is only Send if its contents are Send. This is Rust saw through my attempt to get around things by wrapping the !Send field in Arc<Mutex<>>
Wrapping something and then impl Send for it does not make it safe to send across threads. When you impl Send, you are just telling the compiler that you have ensured it is safe to send. Adding the impl Send doesn't make it safe if it isn't. Having a pointer shared between threads that isn't guarded is not safe. It is unsound. Relying on, "I know it will only ever be present in one thread", is not the same as being "safe". By adding the "impl Send", you are saying it is safe to send across threads. I think to truly be sound it must be protected by a guard like Arc<Mutex<>> otherwise you are relying upon "good behavior" of the user/programmer rather than guaranteeing it won't be misused.
I understand that the wrapper does not make it safe, but it can't ever be completely safe, right?
There is a raw pointer at the core of it. I agree that wrapping it with Arc<Mutex<>> would make it safer, but something could still have that raw pointer even though it's wrapped in Arc<Mutex<>>.
Based on the Rustonomicon statement that raw pointers are not safe to share between threads as more of a lint, and the fact that wrapping in Arc<Mutex<>> doesn't make it completely safe, I decided to forego the Arc<Mutex<>> wrapper.
It can be safe if you expose a safe API. It's all about what you enable users to do with it.
I mean, here's something that's obviously safe to send across threads.
// You can't do anything with this!
struct Useless(*mut i32);
impl Useless {
fn new() -> Self { std::ptr::null_mut() }
}
// This is clearly safe because nothing can be done
// with a Useless
unsafe impl Send for Useless {}
// Likewise, nothing can be done with &Useless
unsafe impl Sync for Useless {}
this happens to be correct at the moment for your program, but a trivial refactoring might undo that, without you noticing.
Maybe by yourself, half a year from now? Maybe by a different maintainer who has to fix it after the program mysteriously starts misbehaving after an OS upgrade? You're right back into mysterious-segfault-bug terrain!
unsafe impl Send is only meant for cases where you have explained the safe conditions to the compiler, in the form of validation code and/or assertions. Do that, the compiler's (and this forum!) have your back
"more of a lint", because with raw pointers, there's really not much you can sensibly say about them at all.
In your case, the problem seems to be that there might be multiple copies of SharedMemoryRaw. Is there a way that you can prevent this?
A strategy might be to only initialise the SharedMemRaw inside the SharedMemoryManager::new(), and/or initialising it as a lazy_static, to ensure it's only called once during the entire lifetime of the program.
If you can statically prove that there is no way to get that second copy; it can probably be safe.
