I have a place in my program that can only accept Send functions. However, I'd like to be able to submit non-Send functions as well using a wrapper that makes them Send like so:
In MakeThisFnSend I convert functions that return Result<(), Box<dyn Error>> to Fn's that return Result<(), Box<dyn Error + Send>>
let func2: Box<dyn Fn() -> Pin<Box<dyn std::future::Future<
Output = Result<(), Box<dyn Error + Send>>> + Send + 'static>
> + Send + Sync> =
Box::new(move || {
let func_clone = Arc::clone(&arc_func);
Box::pin(async move {
let before: Result<(), Box<dyn Error>> = func_clone().await;
let after: Result<(), Box<dyn Error + Send>> = match before {
Ok(()) => { Ok(()) }
Err(err) => {
Err(Box::new(std::io::Error::new(
std::io::ErrorKind::Other, err.to_string())))
}
};
after
})
});
However, the compiler complains: ^ future created by async block is not Send!
As far as I can research, the compiler sees any non-Send values within the block as making the entire block non-Send, even though the whole purpose of the block is to strip away all non-Send material.
As far as I can tell, this code is perfectly safe and it is just that the compiler cannot see that or is overly strict. Is this the case? If it is, can I force this to work somehow? Is this a use case for unsafe?
Wrapping a closure or future in something which returns Result<(), Box<dyn Error + Send>> is one thing… though, it feels like your example is morally returning Result<(), String>, which is less than ideal.
That doesn’t make the closure or future itself Send, though. Its state might contain !Send data, and you can’t get around that with a wrapper (though for closures which implement Fn and Sync, it suffices to pass a reference to the closure to other threads).
if you have a non-Send field in a struct, the struct cannot be Send (unless you unsafely implement it, and take the responsibility to maintain the safety invariant yourself).
closures and futures are also data types. for closures, the "fields" are the captured environment, and for the futures created by async block or async fn, the "fields" are any variable that are kept alive across .await points. you can't just "strip away" the non-Send content by wrapping a data type, you have to redesign it.
And... are you aware of how futures are basically state machines? When you await the inner future, it has to be part of your state machine, and control gets passed back to the runtime at that await point. If your new future is Send, the runtime can send it to a different thread whenever it has control. If that can happen when the state holds the inner future -- like it does at that await point -- it would be unsound, as the inner future may genuinely not be Send. (Corollary: this is not a valid use case for unsafe.)
You could avoid that by driving the inner future to completion with a local runtime, instead of awaiting it... but that would block the runtime driving your outer future for the duration of the inner future (you would not be cooperating with the outer runtime during that duration). And this will generally not go well.
Alternatives include those discussed in the article, or requiring the returned future be Send further up the logic somewhere (so it's Send in arc_func already).
You seem to think that what makes the closure !Send is the presence of a !Send type in its signature, but that's wrong. It's the types that it captures and (for async blocks) the types that are held across the .await points. This is because closures and async block at the end of the day are structs and those types are their fields.
Seen from another point of view, if a closure cannot be moved/called from another thread what makes you think it's ok to do so if you just change its return type afterwards? It's still not allowed to be moved/called from other threads!
For example, imagine the closure captured a Rc from its own thread and internally incremented/decremented its reference counter. If you move/call it from another thread this could cause a data race on the reference counter, possibly making a thread believe that it's 0 while in reality it should not, and ultimately leading to a use-after-free. Your changing of the return value has no impact on this.
I think my misunderstanding was I thought the compiler would skip generating the state machine/saving the !Send data if the awaited future was immediately ready and effectively make it a contiguous function. Or that the !Send data could be dropped before the await point.
Apparently according to the Async Book, that should be logically possible, but isn't supported.
Either way, I guess the answer is to just have the input be Send.
Regardless of any other consideration, whether a future is immediately ready or not is part of the run-time behavior of the future, not part of the type of the future, and therefore it cannot affect whether or not a program is accepted by the compiler.