Suspicious "undefined behavior" report by miri when using `tokio::task::yield_now`

[chuigda]

Today I'm testing my serializer with miri, and miri then reported an undefined behavior. After some work, I got this "minimal reproducible example"

#![forbid(unsafe_code)]

pub struct Foo {
    bar: String
}

impl Foo {
    pub fn new() -> Self {
        Self {
            bar: String::from("baz")
        }
    }

    pub async fn yield_test(&self) {
        eprintln!("{}", self.bar);
        tokio::task::yield_now().await;
        eprintln!("{}", self.bar);
    }
}

async fn application_start() {
    let foo: Foo = Foo::new();
    foo.yield_test().await;
}

fn main() {
    tokio::runtime::Builder::new_current_thread()
        .build()
        .unwrap()
        .block_on(application_start())
}

Here is the (somewhat simplified?) stack trace:

error: Undefined Behavior: trying to reborrow for SharedReadOnly at alloc3209, but parent tag <7882> does not have an appropriate item in the borrow stack
  --> src/main.rs:17:25
   |
17 |         eprintln!("{}", self.bar);
   |                         ^^^^^^^^ trying to reborrow for SharedReadOnly at alloc3209, but parent tag <7882> does not have an appropriate item in the borrow stack
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
   
   = note: inside closure at src/main.rs:17:25
note: inside closure at src/main.rs:23:5
  --> src/main.rs:23:5
   |
23 |     foo.yield_test().await;
   |     ^^^^^^^^^^^^^^^^^^^^^^
   = note: inside `tokio::runtime::Runtime::block_on::<std::future::from_generator::GenFuture<[static generator@src/main.rs:21:30: 24:2]>>` at /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/mod.rs:450:46
note: inside `main` at src/main.rs:27:5
  --> src/main.rs:27:5
   |
27 | /     tokio::runtime::Builder::new_current_thread()
28 | |         .build()
29 | |         .unwrap()
30 | |         .block_on(application_start())
   | |______________________________________^
   = note: inside `std::rt::lang_start::<()>` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:62:5

error: aborting due to previous error

Removing the tokio::task::yield_now will make miri happy, while miri seems to have no problem with other awaits. I'm not sure if this is a bug of miri, or it is related with some Rust's magic internals. So I'm requesting for help here.

[chuigda]

I guess these stuff might be related with the problem

Mutable references vs self-referential structs

Async/Await I: Self-Referential Structs

Support for borrow across yield (self-referential generator)

Here's a toy example of a self-referential generator. let mut generator = || { let x: u32 = 1; let ptr = &x; yield 0; yield *ptr; }; It keeps a borrow across a yield. When the generator is desugared into a state-machine, it needs to store the state that each yield depends on as a struct. In this case, one of the state structs will need to contain a reference to one of its members, thus, self-referential. Such a struct is not safe to move to another location in memory since the …

[alice]

Yes, this is simply because miri doesn't understand the objects generated by async/await. You may also find this relevant, though in your particular case the issue is with async/await syntax itself, not any of Tokio's abstractions.

[chuigda]

Thanks alot for the answer. I think perhaps I have (somewhat) understood: when implementing async/await things, compiler will use some "magic" approach like the UnsafeAliase/UnsafeAliasedCell mentioned above, which are currently not in Rust's model?

[chrefr]

No. When implementing async/await. Compiler will just generate unsound code. You can do the same by hand. The only difference is that the compiler will turn off LLVM's noalias markers so LLVM will not misoptimize this. But according to the Rust Virtual Machine rules (at least as specified currently), this is UB.

[riking]

And this is what we mean when people say "the standard library cheats in its unsafe code". The code generation for async/await is locked to the current version of the compiler, so it can rely on non-language-level compiler guarantees.