So, I'm currently dipping my toes in audio visualization, and discovering the fun challenges of Real Time Done Right where one tries very hard to fully ban unbounded-time work (system memory allocator, blocking I/O and synchronization...) from the timing-sensitive threads.
These constraints lead to less straightforward code that is more likely to have weird and hard-to-reproduce bugs, so I think excellent diagnostic information is crucial. Hence I'd like to have some logging in my RT threads. But I obviously can't just drop in a general-purpose logger there, because those do very RT-unsafe things. What could maybe work, however, is to use an asynchronous facade that feeds a dedicated logging thread in an RT-safe way.
Here's my vision of an ideal asynchronous logger for this purpose:
- Threads that send logs won't ever block, do I/O, or allocate memory. All they do is to serialize their logs into a lock-free MPSC queue, which is asynchronously processed by the logging thread.
- A queue that neither blocks nor allocates must have a finite capacity and drop messages when it's full. Personally, I'd like to give new messages priority over old ones.
- Log queue overrun is a serious issue that should be detected and reported by the logging thread.
So far, my research on crates.io has not turned up any suitable crate. Some like
slog-async get really close, but even that one allocates memory all over the place. Of course, one solution would be to write an RT-safe memory allocator (using fixed-size memory blocks that are allocated during application startup, managing those the arena way, and panicking if they get all used up), and inject it as the global allocator of my application. But if I can avoid this level of guru hackery and messing around with magical global state, I would rather do so.
So, is anyone aware of an existing effort towards RT-safe logging? Or should I write a crate for that myself?