Implementing a binary protocol using tokio

Normally I use Framed in tokio-util for protocol parsers, but I'm currently implementing a protocol that uses variable length "frames". I found myself implementing a pretty complicated state machine within the Framed Decoder, to parse the protocol when I remembered that one of the primary reason for async is to not have to write these convoluted state machines.

The protocol I'm implementing now is basically just a sequence of u32's and u64's, followed by a variable length payload.

It lends itself to being implemented as:

let hdr: u32 = read_u32(&mut reader, &mut buffer).await?;
match hdr {
  PUT_HDR => {
    let flags: u32 = read_u32(&mut reader, &mut buffer).await?;
    if flags & FLG_HASACK {
      let ack_id: u32 = read_u32(&mut reader, &mut buffer).await?;
    }
    let payload_len: u32 = read_u32(&mut reader, &mut buffer).await?;
  }
  _ => {
  }
}

With that said, I need this to be somewhat performant, so I want it to buffer reads, and I want to avoid unnecessary copying (as in "quote-zero-copy-endquote"). The read function will be using tokio::select!{} to monitor for cancellation, however it does not matter if data is lost because the only other wake-up is a shutdown even that should immediately abort transfers (even mid-frame).

What's a good way to do buffered reads with tokio?

You can use the BufReader wrapper to ensure that your IO is buffered.

That said, I wonder if you're using Framed incorrectly? The frames being variable sizes shouldn't be an issue at all. Unless parsing frames depend on previous frames, you shouldn't end up with a state machine.

The situation I end up in is that once I've consumed an u32 and determined what comes next, and there isn't enough data in the buffer for the following data, then the decoder needs to return Ok(None) to indicate that more data is needed. At this point I need to store state in the decoder so it knows what to do on next run. The first u32 is a frame type indicator, and the second one is (often) another u32 which is a flags field. Both of these affect how many fields follow. That is to say, I don't want to keep reparsing the same frame if it takes multiple reads to get the entire frame.

If I would instead peek the buffer and not consume any of the parts until the entire frame has been received and parsed, then I indeed do not need to store any parsing state in the Decoder. Is this what you're alluding to?

Ah, okay. The intention is generally that you would just reparse the frame after returning Ok(None).

1 Like

The number of times a complete frame will not come through is likely vanishingly small, so - in practice - I shouldn't worry about reparsing. That said, these are the types of things I can irrationally obsess about lol. Framed is one of my favorite abstractions, so I should probably just learn to live with it, and focus on the fact that reparsing generally won't happen.

Besides, reparsing a frame is usually extremely cheap for the vast majority of protocols.

1 Like