A listener for slices of bytes

Hello,

Say I have a super barebones service written in C that does something on the platform it's running on, and then gets a few kilobytes of binary data repeatedly (binary log "lines").

Assuming the lifecycle of the barebones C service is managed by the platform, what would be the best way to add as little as possible to this C service to send out those bytes to a (Rust) listener running on another server.
The Rust listener will then process those slices of bytes.
It needn't acknowledge the receipt of those bytes back to the C service.
Whatever protocol guarantees are there will be fine, i.e., TCP's.

I'm not too great at programming, so please explain your solution a little bit :slight_smile:

MQ is a hassle, setting up topics, etc.
My main requirement is that on the source side, I want to do very little setup.

Thanks for your help and guidance!

You could set up a webserver on the Rust side, and then use libcurl to send HTTP requests from the C program. Libcurl is pretty easy to use for basic operations, so it shouldn't take more than a few dozen lines at the very most.

Sorry I forgot a couple of rather important details!

  1. I might have the option of invoking the C code from Go. Still, it'll be a restricted list of packages from the ecosystem that I'll have access to.
  2. In terms of volume, it'll be tens to hundreds of thousands binary log lines per minute.

I don't understand why that is relevant. You can make HTTP requests from Go, too, it that's what you would prefer.

It's fine to transmit that much data over HTTP. People regularly download multi-GB files over HTTP.

I don't understand why that is relevant. You can make HTTP requests from Go, too, it that's what you would prefer.

Ah no I meant to say, if there's a better way that's available via Go, a suggestion for some such package would also be ok.

It's fine to transmit that much data over HTTP. People regularly download multi-GB files over HTTP.

Lol I know. Just adding more info in case there's a better protocol for such streaming type data.
I've heard of NATS but don't know how much simpler it is to use compared to regular MQ.

I don't know what NATS is, so I can't judge that. I agree that for simple systems that don't need to be distributed, setting up a message queue properly is a hassle which is not worth the effort.

I don't know what NATS is, ...

I'm still researching too :slight_smile:

https://nats.io
https://www.youtube.com/watch?v=sm63oAVPqAM

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.