I am looking to write a multi thread program with many threads waiting on a queue using a mutex and condition variable combination.
Program first creates a thread pool, lets each thread wait on its own dedicated queue.
A job scheduler finds the data to process via user input, network etc and takes a lock to the specific worker thread in the threadpool. It then pushes data to the thread pool.
Once pushed, the job scheduler wakes up the thread(s) by calling a signal/ notify.
Upon wakeup, the specific thread takes the lock of the queue, dequeues the work, unlocks and executes the work given.
In C++, this is pretty much quickly written with std::thread, std::mutex, std::condition_variable and std::queue.
I am finding it difficult in rust as to how to approach with the thread function. Most of the examples use closures and it is hard to look at the code and understand it when using them in largescale code when they span beyond 50 lines.
I would like to ask, if there is an example(s) that is similar to that of C++? or a link that provides some educational information about Rust multi threading, locking and data sharing mechanisms. Much appreciated.
There are multiple Rust libraries that provide thread pools. Most of them use closures as their interface because it’s the most flexible way to transfer arbitrary work. You can also create your own set of threads and use a MPMC[1] channel library (like flume or crossbeam-channel) to distribute work to them and thus turn them into a thread pool. I would not recommend writing a thread pool and job queue from scratch unless your goal is specifically to have the experience of doing so. Is your goal to write a thread pool or is it to write a program that uses a thread pool?
My approach to that is to put the main work of a thread into a function (or a method) and call that from the closure that is spawned as per most example one sees.
Typically I'm using Rust channels or crossbeam::channel crossbeam::channel - Rust for queueing things.