Async spawn_blocking, but scoped?

[kornel]

I need to gracefully combine a blocking database interface with an async code. The usual way to spawn blocking code in async context is something like spawn_blocking.

The problem is, spawn_blocking requires 'static, so I can't do:

async fn query(arg: &str) {
    spawn_blocking(|| {
       sync_query(arg)
    }).await
}

because the arg won't live long enough. I know I could work around that with a clone, Arc, etc., but I wonder if there's an async equivalent of rayon::scope() that could launch blocking code on a thread, while also allowing temporary references.

[Dushistov]

@kornel

while also allowing temporary references.

But this basically require blocking call.

async fn query(arg: &str) {
    new_shine_spawn_blocking(|| {
       sync_query(arg)
    })

So basically new_shine_spawn_blocking should return only after sync_query is done,
there is no way around. And this will break all async model.

[alice]

You can use block_in_place, which has no lifetime requirement. Note that this approach has some big pitfalls: anything running concurrently in the same task (i.e. within the same spawned task using e.g. join) will not be able to be polled during the blocking operation.

In general scoped threads or spawns cannot be used inside an async fn without blocking the poll call itself, as a future can be canceled (dropped) at any time, which would invalidate any references the task has into the future.

[Matthias247]

There is some discussion around making it work in Tokio here. Unfortunately it's likely not possible to make it a safe API, since one can leak and forget the code the scope is running in.

[DoumanAsh]

Example relaxing restriction src/global.rs · master · Douman / tokio-global · GitLab

Note that it is safe as long as you can guarantee that you will not out outlive spawned future

This is not local spawn though, I spent a lot of time trying to achieve it, but with current API I think it is not possible