Do you think community needs this tool and should i spend energy on it?

For some of my projects, I don’t really want to deal with web-sockets and I only want to stream from server to clients.

And I feel often its much easier to handle streaming one way and accepting http requests for my projects.
So as a result I wrote a simple streaming library for now supports only json streaming.
Here you can see my crate.
So my actual question is, Do you also need something like this sometimes. Should I improve this project as an open source project
and add things like auth, passing initial arbitrary credentials and supporting other sorts of data other than json?

Please comment if you need a project like this and uses cases so I can decide what to do next.
Thanks.

1 Like

For one-way streaming I’m a big fan of EventSource. It’s a HTTP-based streaming response with a simple line-oriented format. You can easily send JSON with it.

The important feature it has is that every message can have an ID, and you can reconnect/resume the stream by sending last ID you’ve seen to the server.

The hard thing about such libraries is dealing with proxies that buffer responses. You’d need something to detect that (e.g. a ping or handshake message) and fall back to long polling in such case.

1 Like

Event source is nice but Im not sure its supported everywhere and it has its own protocol overhead. The idea here was to remove the entire overhead of piling a ton of extra protocol details and use plain old http 1.1 which is supported everywhere which literally needs nothing but 2-3 lines of extra headers to disable content sniffing etc. on browsers.
Resuming is a nice idea but it also adds extra overhead and memory complexity but i think i can add a feature to enable that on demand.
About proxies i didnt try it behind a proxy but as long as the proxy respects the original headers do you think will it be a problem ? @kornel

EvenSource is HTTP/1.1 (and compatible with HTTP/2). It has been a standard for 10 years. It has 91% browser support. It has 7 bytes of overhead per message (data:•\n\n).

Buffering HTTP proxies are a problem, regardless whether you use the event source format or a custom one. There are no HTTP headers to control buffering, it’s an implementation detail. In nginx you need proxy_buffering off. Last time I checked, for things based on wininet.dll you needed to send at least 4KB of padding before it’ll send any data forward. And there are some anti-virus programs are just an unfixable pit of sadness. That’s why having an option to fall back to long polling is so useful (you keep connection open when there’s nothing happening, and once you have a message to send, you send it and close the connection, and wait for the client to reconnect).

1 Like

So, I think I can convert http streamer to be compatible with Event-Source API. However after a quick search I couldn’t find any RFC describing the server end of the protocol.
Can you suggest any documentation on the server implementation of server sent events.

The protocol is documented in the HTML spec:

https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events-intro

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.