Crates to use for worker processes

It depends on what you are using the worker for.

Python doesn't really work well when you want to do compute-heavy tasks in multiple threads because the GIL essentially serializes everything, but Rust uses bare OS threads so you can run tasks in the background as much as you want. There are libraries which provide a threadpool abstraction for doing exactly this... If you are using an async framework it's usually enough to us their spawn() function (for async jobs) or spawn_blocking() (for sync jobs) to make sure a task is done on its threadpool.

Alternatively, if you are wanting to use workers to distribute tasks among multiple computers, you can often use clients for normal queue libraries like RabbitMQ or Redis (which has pub-sub capabilities) and follow the same patterns as you would in Python.

1 Like