I have a variable, growing number of tasks that I'd like to run with controlled concurrency on a thread pool. It would be an ideal case for
rayon::scope, but I have some shared data these tasks need to access, and Rayon tasks can't use a
Mutex (Rayon is prone to deadlocking when tasks may depend on each other).
Is there a Mutex-safe thread pool crate for Rust?
The closest that comes to my mind is tokio runtime + async mutexes, but that feels like a heavyweight solution.
I haven't tried this, but wouldn't a ReentrantMutex work?
I don't see how Tokio's async mutexes help? If you lock two mutexes in a way that causes two tasks to deadlock on eachother, you're just out of luck. What kind of deadlock is the main issue in question?
I think @kornel meant to use tokio instead of rayon because it has it's own mutexes and a work stealing pool.
Probably referring to this issue
Ah, I see. This is the same issue as what happens if you hold an non-async std mutex across an
.await in Tokio. The call into rayon's
collect or whatever might schedule an unrelated task on the current thread, blocking the task holding the lock.
So, yes, Tokio and async mutex does solve that, because the task will yield control when it tries to lock and it being already locked.
It seems like the only ways to combat this is to:
- Don't use
join or anything else that gives up control while holding the lock.
- Have more threads than jobs.
- Be able to suspend tasks that wait to get the lock.
- Somehow know in advance whether a task would take a lock?
- Only schedule subtasks on the thread holding the lock.
Where Tokio is solution three.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.