The following program fails to compile because the future returned by asdf is not Send. b (&Cell) is not Send because Cell is not Sync, and this borrow is held across an await point, therefore the future is not Send.
use std::cell::Cell;
fn must_send(_: impl Send) {}
async fn asdf() {
let a = Cell::new(1);
let b = &a;
async {}.await;
// let b = &a;
println!("{}", b.get());
}
fn main() {
let _ = must_send(asdf());
}
This makes sense to me if the async function doesn't own the Cell. For example, if you had an async function that accepted a &T where T: !Sync, you cannot poll the future on another thread since that would allow you to access T on multiple threads simultaneously.
However, in this case, the async function has ownership over T. Even if you were to poll the future on another thread, T and &T would "move" together, which I believe means that it would not be possible to simultaneously access T from multiple threads. Therefore, even though &T is not Send, you would still be able to safely hold &T across an await point in a Send future, assuming that the async function owns T.
Is my reasoning correct here? Is there a case where it would not be safe to hold a &T borrow (where T: !Sync) across an await point in a Send future while having ownership of T? If this is in fact safe, is this just one of those cases where it is not worth it for the compiler to incur more complexity to handle such things?
I encountered this with something like the following code:
#[axum::debug_handler]
async fn handle_something(headers: HeaderMap, request: Request) -> Response {
let the_response = || request.uri().to_string().into_response();
// closure the_response borrows `request`, which is not Sync, across await point
// can fix this by doing `let uri = request.uri()` then borrowing `uri` in closure
if !something(&headers) {
return the_response();
}
if !async_something(&headers).await { // <-- await point here
the_response() // <-- closure used here
} else {
"ok".into_response()
}
}
Welcome to the world of async. Whatever intuitions you might have developed up to this point are better off neatly packaged, tied with a ribbon on top, then promptly thrown out of a window.
Any async functionasync fn(A) → B is syntactic sugar for fn(A) → impl Future<Output = B>. The “ownership” you are trying to carry over from the sync side doesn’t apply here.
Any F: impl Future itself isn’t a singular memory location tagged with a few impl's here and there. Rather: it’s a compiler-produced state machine that provides no guarantees whatsoever as to where or when or how, thread/execution/timing-wise, any next state snapshot is to be handled.
Meaning: the compiler has to assume the worst case scenario. As in: each and every await point has to be considered a potential thread hand-off of the state A created by that await point into the state B which follows next.
Since both a: Cell<i32> and b: &Cell<i32>in your scenario are (to be thought of as) initialized in a thread X and processed after the await in the println!(...) by a thread Y: you’ve got yourself a perfectly reasonable violation of the Sync and Send trait bounds.
In your particular example, it’s easy enough to solve. Simply hand out the ownership over a itself instead of the its reference &b. This compiles fine too. For the future reference (pun intended), when dealing with impl Future in any shape or form, think less about the ownership over a singular memory location (sync) and more about the implicit synchronicity over the data captured within/by any particular await point of the Future itself. More often than not, it’ll help.
Should you happen to feel momentarily distraught by all the aforementioned, tough luck. Better yet: let me channel my inner Bethesda’s customer service from the times of Fallout 76:
Hello,
We are sorry you aren’t happy with the state of the async in the current edition of Rust. The memory ownership intuition you were meant to develop when working with single-threaded and/or parallel execution turned to be too expensive to port into our zero-cost concurrency framework, reinvented from scratch for the ultimate benefit to no one in particular.
For a fn(args) -> F where F: Future, Future::poll takes Pin<&mut F> and ctx (for waker, not particularly relevant here), producing Poll<F::Output>. The async function is therefore "resumed" when polled. Both T and &T exist within F at the await point. Sync is applicable when T is accessed across two threads simultaneously, but since Future::poll takes Pin<&mut F>, F cannot be accessed simultaneously from multiple threads and therefore neither can T. This is (as of now at least) the contract of Future.
Sure, so in the worst case, F, having both T and &T, gets polled from a different thread. T is still not accessed from multiple threads simultaneously; effectively T has been Sent along with &T, and this Send does not permit concurrent access.
But Cell is Send, just not Sync. Per the above, I do not believe Sync access is permitted by holding a &T across an await.
Have I misunderstood something? I would like a counterexample where this doesn't work, if you do wish to come up with one.
To be clear, I have already solved the original issue. I've asked this question more out of curiosity than due to any real problem. I do quite like async, even with its flaws and all.
I think this is essentially the same as another wish that comes up occasionally: “I want to be able to send a structure containing Rcs across threads, but this will be fine because the Rcs all move together so only one thread ever modifies the reference counts.”
This is valid to do, but the compiler can’t be shown that it is safe, because there is no way to express, within the Send and Sync trait mechanism, that none of the Rcs escape this boundary. Similarly, in your case, b can validly be sent given how it is created and used, but there’s no way for the type of b, by itself, to implement Send, because it only has this property together with a, not alone.
So, in order for the compiler to allow this case, it would need to derive the Sendness of the async block from some entirely new kind of analysis, not how Send normally works by composition (a type is Send if all its fields’ types are Send). As you say, it is “more complexity to handle” — in an area which is otherwise not “complex”, not like, say, the borrow checker.
As a workaround, you can write a wrapper type with unsafe impl Send. In the case of the Rcs, it would wrap the whole graph; in your case, it would have to wrap either the value of b or the future as a whole, and the future is responsible for not betraying this claim of Send.
And even if we try to expand this mechanism, it's highly non-trivial to have a sound check - one of the Rcs could be stored in a thread-local, for example.
A possibly interesting tangent: this code is basically identical to the example in the unstable std::sync::Exclusive. But, Exclusive (or sync_wrapper, a stable third-party version of the same idea) cannot help you in this situation because the future isn't Send and Exclusive can't make it Send. I think this makes the example kind of misleading, and I filed issue #146245 about it.