Concurrent data structure for midi synth

I'm programming a midi synthesizer using egui.

By the libraries I use, I already have three threads running, one for receiving and parsing midi (crate: midir) signals, one for the audio output (crate: cpal) and the main thread which is controlled by egui/eframe.

For context:
Both cpal and midir expect a callback closure for the processing of the respective data on creation of the connection to audio output or midi input device, respectively.
I managed to get a version of the code working without egui, by hard coding the synth settings and moving the synth to the closure for the audio output. As for the midi messages, I simply parse the messages and do some additional filtering before sending them through a channel, the receiver of which I also move to the audio callback.

The play(&mut self, samples: usize) -> Vec<f32> function on the synthesizer implementation needs to be mutable, because I need to update its state, such as a Vec of active notes, the phase of oscillators and similar. Obviously I need mutable access from the egui side too, since I want to change settings of the synth like waveforms and similar.
I realized that the parts each thread needs to have mutable access to, are separate. The audio thread only needs to update the state that is associated with what I'd call acute state (so what's needed to actively generate sound), while the egui side only needs mutable access to the more permanent side (essentially what I would serialize to reproduce the same experience while playing the synth). And both threads need read only access to the other part of the synthesizer. To display what notes are played, for example on eguis side, and to know which waveform to use on the audio side.

I have made the Arc<Mutex<_>> / Arc<RwLock<_>> mistake before, which seems even more terrible when I have a time sensitive thread running.

This problem is additionally made hard to reason about because I have no control over when code is executed in any of the threads.

Are there any design patterns or crates I could look into to get a concurrent data structure that is split up in such a way?

Edit: I should add that while the audio side should always be fast to update the state, I don't care if the changes made to the synth settings don't take effect immediately.

If two parts of the code only ever need to access two disjoint subsets of self, then the solution is simple: split up the type into two disjoint types and pass separate mutable references to the respective functions.

However, this suggests that this is not the case, i.e., you need to access both parts from both threads. In this case, it doesn't matter that the access to the "other" part is read-only. Simultaneous access is already a race condition if there is even a single concurrent writer; it is not the case that only two or more concurrent writers need to be synchronized. A writer needs to be synchronized with any and all concurrent access, including readers.

Which means that you will need to perform synchronization.

I can't imagine how this is a problem in practice. Locking an uncontended mutex is fast, at least fast enough to not cause lag in audio processing. Even the best-quality audio sampling dictates something on the order of 44k samples per second, i.e., 23µs per sample. Assuming a very modest 1 GHz clock speed, that's 23 thousand clock cycles. If a lock can't be acquired in that much time on commodity hardware, then there's something seriously wrong with its implementation.

Anyway, if you are highly worried about locks, you should probably use something more specialized in terms of data structure and code architecture. For example, consider communicating via message passing through a queue. Search the standard library documentation for mpsc to get started.

1 Like

I would recommend you move UI to a thread separate from synthesis and communicate through channels.

1 Like

After trying with Arc<Mutex<>> I came to the conclusion you were correct this is fast enough
thanks :slight_smile:

But in the case where the mutex is locked by another thread, in order to update settings, you still want to avoid causing a glitch by being unable to supply samples, which (depending on the signal being created and the particular moment of interruption) may be a highly unpleasant or at least distracting “click”.

The standard wisdom for implementing real-time audio processing is that the audio thread(s) should use only lock-free means of communication with other threads (such as channels).

Even the best-quality audio sampling dictates something on the order of 44k samples per second, i.e., 23µs per sample.

This, on the other hand, is a more conservative analysis than necessary: audio sample data is often transferred in chunks (e.g. 512 samples at once) and stored momentarily in buffers, so that the CPU can spend time doing one thing to many samples (SIMD, cache locality, etc) and to allow for preemption of your thread to not be an instant disaster by having some buffer capacity. The buffer size is chosen as a tradeoff between reliability (large buffer) and latency (small buffer).

Therefore, the spare time available is roughly the sample period times the buffer size, rather than just the sample period.

(Even more nitpicking and largely tangential: audio processing is sometimes done at higher sample rates than 44.1 kHz. It may not be necessary for a final recording due to the limits of human hearing, but additional sample rate makes it easier to implement artifact-free synthesis and effects. 48 kHz data is very common, and 192 kHz is the typical maximum supported rate, in my experience.)

1 Like

That's true, although I find this to have a negligible probability. OP's description sounds like the simultaneous locking can only occur when the UI is updated, and I can't fathom a user who can press buttons fast enough to cause lock contention.

Yes, I also made that suggestion above.

Yes, I was trying to come up with an absolute worst-case calculation.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.