Unique reference doesn't require Sync, but shared reference does

Hi, I found out some strange quirk while refactoring my some code, and I feel like I don't fully understand why it works the way it does. I thought someone here might know the answer.

Versions:

# stable rust
$ rustc --version 
rustc 1.39.0 (4560ea788 2019-11-04)

My only dependency is futures = "0.3.1"

So this piece of code doesn't compile:

use futures::future::BoxFuture;
use futures::task::{Spawn, SpawnExt};

struct MyNum(pub u32);

trait MyTrait {
    fn method(&self, my_num: MyNum) -> BoxFuture<'_, MyNum>;
}

struct MyStruct<MT> {
    my_trait: MT,
}

#[allow(unused)]
impl<MT> MyStruct<MT> 
where
    MT: MyTrait + Send + Clone + 'static,
{
    async fn my_func<S>(&mut self, my_num: MyNum, spawner: &S)
        where
            S: Spawn,
    {
        let mut c_my_trait = self.my_trait.clone();
        let handle = spawner.spawn_with_handle(async move {
            let _ = c_my_trait.method(my_num).await;
        }).unwrap();
        handle.await
    }
}

This is the compilation error:

$ cargo run
   Compiling futures_check_sync v0.1.0 (/home/real/temp/futures_check_sync)
error[E0277]: `MT` cannot be shared between threads safely
  --> src/main.rs:24:30
   |
24 |         let handle = spawner.spawn_with_handle(async move {
   |                              ^^^^^^^^^^^^^^^^^ `MT` cannot be shared between threads safely
   |
   = help: the trait `std::marker::Sync` is not implemented for `MT`
   = help: consider adding a `where MT: std::marker::Sync` bound
   = note: required because of the requirements on the impl of `std::marker::Send` for `&MT`
   = note: required because it appears within the type `for<'r, 's, 't0> {&'r MT, MT, MyNum, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 's)>>, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 't0)>>, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:24:59: 26:10 c_my_trait:MT, my_num:MyNum for<'r, 's, 't0> {&'r MT, MT, MyNum, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 's)>>, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 't0)>>, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:24:59: 26:10 c_my_trait:MT, my_num:MyNum for<'r, 's, 't0> {&'r MT, MT, MyNum, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 's)>>, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 't0)>>, ()}]>`
   = note: required because it appears within the type `impl core::future::future::Future`

However, if I change &self to &mut self at MyTrait::method(), it surprisingly compiles. Just for reference, this is how MyTrait looks after the modification:

trait MyTrait {
    fn method(&mut self, my_num: MyNum) -> BoxFuture<'_, MyNum>;
}

What is the reason for this? What is not safe about the first version?
Any help is appreciated!

Basically the issue is that a shared reference is only send if the underlying type is sync. On the other hand mutable references only require the underlying type to be send.

use std::marker::PhantomData;
use std::rc::Rc;

fn assert_send<T: Send>(_t: T) { }

struct NotSync {
    inner: PhantomData<Rc<()>>,
}
unsafe impl Send for NotSync {}

fn main() {
    
    let mut ns = NotSync { inner: PhantomData };
    // This is ok:
    assert_send(&mut ns);
    
    // This is not ok:
    assert_send(&ns);
    
}

playground

2 Likes

@alice: Thank you for the sharp explanation!

Out of curiosity I tried to play a bit with the simplified example you sent.
I changed NotSync to NotSend, and ended up with a dual example:

use std::marker::PhantomData;
use std::rc::Rc;

fn assert_send<T: Send>(_t: T) { }

struct NotSend {
    inner: PhantomData<Rc<()>>,
}

unsafe impl Sync for NotSend {}

fn main() {
    
    let mut ns = NotSend { inner: PhantomData };
    // This is ok:
    assert_send(&ns);
    
    // This is not ok:
    assert_send(&mut ns);
}

There is something subtle about the duality of the two examples that I don't quite understand, but I feel like I might be one step closer to Send/Sync enlightenment.

Do you have any idea why assert_send(&mut ns); is not safe in this case?

If &mut T is Send, then you can use std::mem::swap to send values of type T across threads.

Note that types that are Sync but not Send are very rare.

2 Likes

One example is a MutexGuard


Note: Sync means "safe to share actoss threads"
Send means "safe to give another thread unique access" (and by proxy, give another thread ownership because of std::mem::swap)

In your case this involves the code generated by the compiler when unsugaring the async move future:

  • async move { // <-- captured env here: `{ c_my_trait: MT, my_num: MyNum }`
        let _ =
            c_my_trait
                .method(my_num)
                .await // <-- captured env here: `+= { &c_my_trait: &MT, temp: BoxFuture<'_, MyNum>}`
                       // `&c_my_trait` captured because `&self`
        ;
        // <- captured env here: just `()`
    }
    
  • I have annotated the three main control flow points here:

    1. Right when starting,

    2. At each .await point (here there is just one),

    3. Return point,

  • Each of these points require capturing some environment, which lead to an anonymous structure being automagically defined by the compiler, having some Future logic by translating the async sugar into .poll() calls based on this structure (this is similar (but more complex) to closures and their captured environment, see @KrishnaSannasi's blog post presenting it: Closures: Magic functions ).

  • You can glance at some quick-and-dirty approximation of this unsugaring in the Playground, which is quite ugly (I haven't factored in the different enum and just used a bunch of Options instead, but that was just to get the Pin-ning guarantees right (at least I hope so)).

Now, the interesting point here, is that because the value returned by .method() keeps borrowing the input self: &MT, Rust decides that it needs to keep a &MT around.

  • See the type mentioned in the error message containing &'r MT

    the type `[static generator@src/main.rs:24:59: 26:10 c_my_trait:MT, my_num:MyNum for<'r, 's, 't0> {&'r MT, MT, MyNum, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 's)>>, std::pin::Pin<std::boxed::Box<(dyn core::future::future::Future<Output = MyNum> + std::marker::Send + 't0)>>, ()}]`
    
  • Personally, unless I have forgotten something and my playground link is wildy unsound, I don't think that capturing &MT in the between yield points is really necessary, but hey, that's what the compiler seems to be doing.

Finally, your whole Future, that is, the structure containing all this captured environment, needs to be Send-able to other threads. This, in turn, requires that each and every member of the structured / captured expression also be Sendable.
So it requires that the reference &MT be Send-able.

And a shared reference to something can be Sent to another thread if and only if this "something" is safe to share across threads, i.e.,

&T : Send ⇔ T : Sync

And since your generic MT isn't bounded by Sync, it could be !Sync which could make your code unsound for those "bad" choices of MT, hence the compile error here. You could make the code fail to compile only when someone used a !Sync MT by adding Sync to the bounds of MT.


Now, when you change .method() to take self: &mut MT, then the captured variable in the anonymous compiler-generated struct becomes &mut MT, and,
because one can trivially go from T to &mut T (and the other way around with ::core::mem::replace) within one thread, then:

&mut T : Send ⇔ T : Send

And you did have a Send bound on your generic MT.


Finally, regarding the intuition behind Sync and Send,

  • A type T is Sync if and only if it is sound to share across threads. This can only not be the case when there is shared mutation that does not use multi-threaded synchronization, e.g., Cell, RefCell.

  • A type T is Send if and only if it is sound to make it cross thread boundaries. In practice, this is the case:

    • when T = &U (or some wrapper around it) and U : Sync,

    • and when it is sound to interact (e.g., .drop()) with the T within a thread different from the one where it was created (imagine a struct interacting dealing thread-local stuff).


For a more detailed insight on sharing vs. mutation and the meaning of Sync, see this blog post of mine.

4 Likes