Framed from Tokio to implement a Codec that converts from bytes into Frames. This works really well, is simple and clean to implement and is quite performant.
I was wondering if there is an equivalent to
Framed but for creating
Stream adapters. For example, rather than reading in bytes and returning frames, I'd like to read in bytes, process them and then return bytes. This would be like a "bump in the wire" so to speak.
I know I can get this by implementing
AsyncWrite, but there is a fair bit of complexity in implementing the
poll family of functions that I'd rather avoid if I can.
I'd welcome any thoughts, comments or new directions to go in
Not really. The best option I have for you is to use a codec to get a stream with a chunk of bytes as item type, then use
StreamReader to get an
Cool, at least I'm not missing anything obvious
I read your previous discussion on implementing
AsyncWrite where you described it as "hard mode" and that was enough to make me think twice about implementing my own
I did have a play with wrapping Tokio's
TcpStream so that I could use
poll_read but it seems that
poll_read is private (although
poll_peek is not).
The TcpStream type has a public
poll_read method via the
If you say more about what kind of transformation you want to do, I can say something about how easy it would be.
For example, let's say I have a TCP based protocol, and I use
Framed with an appropriate codec to turn the stream of bytes from
TcpStream into individual messages. Now, if I need to XOR each byte in the stream, currently the easiest way would be to add the XOR logic to the
Codec. The downside with this is that it tightly couples the
Codec with the need for XOR. If I later on need to decrypt something first, I would have to create a new
Codec or at least add some discerning logic to the existing one. What I would prefer to do is keep the framing logic separate.
So ideally what that would look like (based on my current understanding) would be something like
Framed but it provides two buffers - one with the incoming data, and one to write the outgoing data. This way I can put an arbitrary function anywhere into a stream to transform it, and keep that logic out of my codec.
That way a developer can just focus on the traits needed to make the struct compliant, and let the
Framed equivalent handle the heavy lifting. It could be a fun project to get better acquainted with this area though
The specific case of applying an XOR to the data would not be too difficult to implement manually via the IO traits. However, this relies on the fact that:
- XOR does not change the number of bytes.
- XOR does not have a minimum chunk size. (i.e. you can work on single bytes one at the time)
If you use any other type of encryption such that either of the above fail, then it becomes a lot more difficult.
Does StreamExt::scan help you at all? The closure you give it keep arbitrary state, output an arbitrary type, and isn't required to return a new value for each incoming item. The return from the
scan call is stil a
Stream, so you can easily chain multiple calls together to keep the steps of your processing independent.
That looks interesting, I hadn't spotted that. I'll have a play. Thanks!
Does that difficulty come from trying to maintain state by any chance? The reason I ask is that, in
AsyncRead, I can't find a way to write to a buffer in
Pin<&mut Self> which would act like intermediary state.
It seems like I can either pin the inner stream to call
poll_read on it, or I can get a mutable reference to the buffer. As I need to read the data before I can write it to that buffer, I'm somewhat limited in what I can do.
So, have I missed something obvious or am I experiencing the difficulty you warned me about?
Well, adding some extra state is how you would handle it. It's just somewhat annoying to do so.
How does your code look now? Perhaps you should try using
Pin::into_inner and accessing the fields via the returned mutable reference?