Adding tokio to a callback-driven project

help
1

When I read the tokio docs I see nice greenfields projects where main() creates a future and does tokio::run(). Everything's async, you can call tokio::spawn whenever you like, and stuff magically attaches to the executor.

Does anyone have any examples or articles that would help with adding tokio to a project that doesn't have an executor? As a concrete example, I have a function that fires a callback when it's finished. This could easily be returning an impl Future instead. I already have async code in place that runs when it completes, that I could use to trigger notify() on the task instead of run the callback. But I have no idea where to put the tokio Runtime. The caller of the function isn't a tokio function so it can't spawn() the task.

I suppose the key thing is that there doesn't seem to be an equivalent of iOS dispatch_async that I can use to push async tasks into the tokio thread pool when I'm operating outside that context. How can I move into tokio when I'm not already inside it?

2

You don't need a runtime/executor for the callback-based functions. They have their own way of scheduling work already.

If you need to interoperate with futures, then all you need to do is to implement poll that saves task::current() for calling .notify() later, or saves result from the callback to return it in poll later.

3

You just create Runtime manually instead of using tokio::run,
there is method Runtime::executor after call it you get TaskExecutor that can be used outside of tokio thread pool to schedule tasks.

4

Thanks all, I think I'm beginning to understand!

  • If a library/module offers a future as a return value, it is not its responsibility how the polling gets scheduled. The executor comes from the caller.
  • In the code where I want to initiate async tasks, I create a Runtime as shown in the tokio::runtime API docs
  • Rather than call Runtime::spawn directly, I can call executor() to get a TaskExecutor that offers spawn() and is Clone + Send + Sync. In other words, it's easy to share this around or inject it into places that have a need to schedule futures while having them all use the same thread pool. (This was the main piece of the puzzle I was missing!)
  • Within each future chain, I'm now back to the nice situations like in the tokio docs. I can spawn() and do whatever else I want.
  • If I call an async (future-returning) function from outside of a tokio task context, it can simply stuff the returned Future (along with its processing chain) into TaskExecutor::spawn.

I've put all these ideas successfully into a test program (playground link). The Arcs are kind of tedious but I can't see a way around them.

2 Likes
5

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.