Event Style Multithreaded Architecture

I'm looking at building a system which acquires data from external sources (in this first example, an I2C temperature chip - but could be high speed ADCs) and I want to be able to stream this data to gRPC clients.

The hard thing is there could be 0, 1 or n gRPC clients connected at any time.

In other languages I would look at some kind of event registration system where the acquiring thread provides an event that the gRPC clients could register for when the client starts the request.

However I don't see anything out of the box for Rust. I've found a couple of possibilities:

  • Observer pattern implementations - I think I could get this to work but it is complicated by being across threads and doesn't really represent what I would like it to (more of an event/data stream)
  • mpmc channel implementations in crossbeam/crossfire but these close down if there are no receivers. I could keep a dummy reader loop open to keep them alive.
  • I did come across a ring buffer solution which I can probably adapt to work.

Am I missing other options that could allow for this event driven style of architecture in Rust?

What's the rate of data acquisition?

If it's low enough that you can afford function-call-per-data-item, you could work with async and tokio::sync::broadcast, which doesn't require there to always be receivers.

If it's fast enough that you need to minimize per-item CPU overhead, then this problem domain is very much like a DSP/SDR application and maybe FutureSDR could provide the right framework? I haven't yet tried it myself.

If you can't find a channel implementation that already suits your needs, then building on top of a ring buffer sounds like the right place to start.

Is bus what you would want?
https://lib.rs/crates/bus

Thanks to both of you - you've given me lots of ideas.

@kpreid They are often DSP applications but I can normally work on larger data frames so that the CPU overhead of calls isn't a big issue. FutureSDR looks very interesting though and I will have to try and find time to have a play.

@8573 this looks like exactly the sort of thing I need - I'll try that out for my first attempt and see how far it takes me.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.