In that case, a channel based solution is definitely going to be easier for you.
The kind of design you are currently trying is the only way to do it in C/C++, but is incredibly tricky to get perfect.
If you compare it to an office with humans, the shared-data-structs is like having a big table in the center, where the boss writes something on a random piece of paper, and then hopes the workers notice it. ("communicate by sharing")
Meanwhile, the workers are frantically reading all the papers repeatedly, hoping they didn't miss an update from the boss. (and also getting into fights if they want to read the same piece of paper at the same time)
With channels, you instead get an orderly system: The boss writes a note "please do X for me", and gives it to a selected employee. The employee (who now has his own desk, no sharing required!) handles the request, and writes a note back to the boss: "done! I have attached your answer to this note". ("share by communicating")
In Rust, Channels and message passing is the correct solution, that will not explode your brain (or cause you to fight endlessly with the borrowchecker, which hates shared tables in the middle, and wants every employee to have his/her own desk).
You'll need two types of channels:
- one that goes Worker-to-Controller (the "feedback" channel)
- one for each worker that goes Controller-to-Worker (the "order giving" channel)
Channels are strongly typed, so we will need an enum that defines the possible Message.
each message can contain different sorts of data, which we will use
(rought pseudo code only, sorry.. You could have two enums, one for boss-to-worker, and another for worker-to-boss, but that's just an optimisation)
enum Messages {
// infrastructure part that we need for a working channel solution
WorkerToBoss_HiIamNew_HereIsMyInbox( Sender-of-Worker),
BossToWorker_YouAreDone_Shutdown(),
// messages that replace the "raw" field-accesses you are doing right now
// you will likely need multiple sets of these, one for each different type of order the boss can give
BossToWorker_PleaseGiveMeDataOf(description-of-what-you-want),
WorkerToBoss_AnswerData(the-data-you-want),
BossToWorker_PleaseSetDataTo(description-of-what-to-set, new-value),
}
This enum of messages is where the "message passing" name comes from.
We will use "mpsc" channels ("multiple producer, single consumer") from Rust std library (you can also use the excellent crossbeam lib, which is almost holy for multithreading in the rust world)
The steps are then as follows:
- Main Thread: create "feedback" channel; you get a Sender and a Receiver back.
the Receiver is unique ("single consumer"), the Sender can be .cloned infinitely ("multiple producer").
- create the "Controller" thread. This thread gets the feedback-Receiver (
move it in)
the main thread keeps the Sender
- main thread creates the workers. For each worker:
- give the worker a clone of the Controller-Sender (so they can reply to their boss)
- the first action each worker does is create a new Channel (their personal "order-channel".
they keep their receiver (so they can hear what the boss wants).
their second action is to introduce themselves to the boss (WorkerToBoss_HereIsMyInbox) with a message containing their personal sender)
- after that, the worker loops 'forever' doing
match on its order-channel receiver.
- if it gets a "tell me something" message, it sends the answer as a message on its feedback-sender.
- if it gets a "set data to X" message, it sets it (maybe with a feedback message "ok done", if you need it.
- if it receives the "you are done" message, it breaks its forever-loop, cleans up, and exits. (controlled shutdown)
- the Controller-thread has a
inboxes: Vec<Sender-of-Worker>: this is your thread pool.
it then listens on its feedback-receiver, and matches on the incoming stream of messages. You unpack each of the "here is my inbox" messages, and you add the order-Sender to the inboxes list.
Tricky part: You must know when to stop listening (e.g. after receiving 10 here-is-my-inbox-messages, if you create 10 workers)
5.Our infrastructure setup is done. Now comes your actual work. the Controller thread does whatever controlling it wants, sending orders to workers, and listening for answers. The details here depend on what it is you are doing, but I can respect that you are not allowed to discuss specifics.
I hope this helps you get started. Searching for keywords like "crossbeam example" "message passing in rust", "multithreading with channels" or "rust share by communicating" will help.
This pattern is also very common in Go-lang, so maybe you can find some good tutorials there too. (the pattern is the same, the code-parts just have different names).
Good luck!
I will check crossbeam and try your suggestions.