Resources for low-level async programming

help
1

Hi, i'm interested in async programming in rust. I already understand some kind of idea of how does that work under the hood. I tried to search for more concrete examples/explanations with a real code examples, but didn't find anything easy enough for me to understand.

I already checked:

And also tried to follow these rewriting it on rust:

Those gave me general understanding how async works, but where do i continue? It feels REALLY overwhelming to read tokio source code. Do i have any alternatives?

My goal right now is to create tokio-like library with executor and futures in educational purposes

Ps i don't know how to remove yt player. Sorry for bloat

2

I'd check out smaller executors like pollster (just about a hundred LoC in a single file) and futures::executor, before looking at the source code of tokio's runtimes, which are heavily optimized and pretty well abstracted.

3 Likes
3

Thank you very much :pray:

1 Like
4

I found Asynchronous Programming in Rust to be good for my taste. It builds from the bottom up and basically shows how asynchronous task execution can be implemented without the async/await language features which gives you a better understanding what's happening under the hood. It starts somewhat slow by presenting relevant syscalls so if you're already familiar with epoll() and the likes, you can probably skip or skim those parts.

1 Like
5

You can build a pretty trivial (and therefore inefficient) threaded executor using just thread::park and a mutex around a vector of tasks:

// Typed on my phone, completely untested! 

struct Executor {
  workers: Vec<Worker>
}

struct Worker {
  thread: Thread,
  tasks: Arc<Mutex<Vec<Pin<Box<dyn Future>>>>,
}

impl Worker {
  fn new() -> Self {
    let tasks = Default::default();
    let thread = std::thread::spawn({
      let tasks = tasks.clone();
      move || {
        let thread = thread::current();
        let waker = Arc::new(ThreadWaker(thread)).into();
        let mut cx = std::future::Context::from_waker(&waker);
        loop {
          std::thread::park();
          let tasks = tasks.lock().unwrap();
          tasks.retain(|task| task.poll(context).is_pending());
        }
      }
    }).unwrap().thread();
    Self { thread, tasks }
  }
}

struct ThreadWaker(std::thread::Thread);

impl std::future::Wake for ThreadWaker {
    fn wake(self: Arc<Self>) {
        self.0.unpark();
    }
}

You'll need a way to spawn new tasks while it's polling has the list locked - for example a separate list of new tasks the worker drains into the running tasks in the loop, but then your spawn function just needs to Box::pin the future, pick a worker, add the task to it, then unpark the thread.

There's obviously lots of ways to improve this!

1 Like