Best way to get the 3 bytes of a small u32?


#1

So I am currently writing code to read and write the RTMP chunk protocol. One annoying part of that is that the protocol designates the timestamp as a 3 byte number (so essentially a u24). Of course, there’s no such thing as a u24 in Rust so the value I want to write into my byte array is a 4 byte u32.

Assuming I’ve already added checks to make sure the u32 value isn’t a value higher than the max that 3 bytes can hold, what is the best way to get the 3 bytes making up the number?

There are several things I thought of:

  1. Using unsafe to convert the u32 into a byte array, then copy the last 3 values into my packet’s vector. The disadvantage of this is obviously I’m hesitant to utilize the unsafe flag if I don’t have to. I’m not totally sure if this is that much of a concern or not since the scope is easily manageable.

  2. Use the byteorder crate to write the u32 to a new vector, then drain bytes 1-3 off into my packet vector. Since this code will be called every time I need to send audio/video data packets off and I want it to be as low latency as possible the needless allocations don’t have me too happy.

  3. Create bitwise masks and use bitwise operations to get the values for each bit, then shift each of them over until they are all the way to the right of each value.

Are there any better ideas out there to keep things readable, safe (as possible), and quick?


#2

You don’t have to create a vector to write to it. Why not an on-stack [u8; 4]? &mut [u8; 4] will coerce to &mut [u8], and it will work fine because you know you’re only writing four bytes. Then you don’t have to drain it, just slice it. I can’t find whether the timestamp is supposed to be big-endian or little-endian, but you’d just slice the array with 1..4 or 0..3, respectively. This will be very fast, especially when compiled in release mode.

Of course, option #3 is what byteorder is doing internally anyway, but I’d personally prefer not to reinvent the wheel.


#3

You cannot directly write to an [u8; 4], because write() expects &mut self, which is &mut &mut [u8]. In other words, write wants to modify the slice itself, not only the content.

That means, you need an intermediate variable of type &mut [u8] that you can borrow again. Or a Cursor<&mut [u8]>.


#4

IMHO the best short-term solution is to do (3) and move on.

In the long term it should probably be added to byteorder.

If you use byteorder to convert a u32 and then pick out 3 bytes, you code has to care about endianness to do it correctly. So the conversion is split between byteorder and your code, making it more complex than it should be.


#5

I think this is wrong.
The protocol’s byte order (network byte order == big endian) is fixed and byteorder takes care of the host endianness. You just specify what endianness you eventually need and don’t have to care about the rest.

Of course you have to know which 3 bytes to take but this is only dependent on the protocol byte order, not on the host byte order. (Spoiler: it’s the last 3 bytes)


#6

Thanks for all the comments. I ended up reconfiguring a bunch of my function parameters from taking a Write to taking a Cursor<Vec<u8>> instead. That allowed me to take advantage of the byteorder crate without worrying too much about the extra vector allocation.

fn write_u24_be(cursor: &mut Cursor<Vec<u8>>, value: u32) -> Result<()> {
    debug_assert!(value <= 16777215, "Value is greater than what can fit in 3 bytes");

    try!(cursor.write_u32::<BigEndian>(value));
    
    {
        let mut inner = cursor.get_mut();
        let index_to_remove = inner.len() - 1 - 3;
        inner.remove(index_to_remove);
    }

    try!(cursor.seek(SeekFrom::End(0)));
    Ok(())
}

#[cfg(test)]
mod test {
    use super::write_u24_be;
    use std::io::Cursor;
    use byteorder::WriteBytesExt;

    #[test]
    fn can_write_u24() {
        let mut cursor = Cursor::new(Vec::new());
        write_u24_be(&mut cursor, 16777215).unwrap();

        // Make sure next writes are at the 4th byte
        cursor.write_u8(8).unwrap();
        
        assert_eq!(cursor.into_inner(), vec![255, 255, 255, 8]);
    }
}

If profiling shows this being a critical hot path then I’ll swap that function out for bitwise stuff.


Issue with transmute
#7

I do mean the protocol byte order, not the host byte order. Although it is fixed, picking the right three bytes is still something that complicates things, so overall I would consider writing it directly as shift and mask simpler.


#8

A simple

fn get_u24_bytes(v: u32) -> (u8,u8,u8)
{
    (
        (v & 0x00ff0000 >> 16) as u8,
        (v & 0x0000ff00 >>  8) as u8,
        (v & 0x000000ff >>  0) as u8
    )
}

would not necessarily make the code more complicated than introducing a whole new library.
Well, if you know which of the bytes you need to take from a u32.

The approach I’d take would be transmuting the u32 to a [u8;4] and taking the right bytes, but if you do want to make it easier for you to point at someone else when something breaks, feel free to ignore this paragraph.


#9

No you are right, not really sure why I was over complicating it in my mind. Probably because in my career as a developer this is the first time I’ve needed to utilize bit shifting and bitwise operators.


#10

No, but it would give you a wrong result. The middle byte should be shifted by 8 instead of 24.

Not only the right bytes, but also the right order.
Both decisions are dependent on the host byte order, which makes that solution rather complicated.


#11

I think Rob Pike nails the right attitude toward handling endianness in The byte order fallacy. It’s much in the spirit of benaryorg@'s solution minus the typo.


#12

The byteorder crate has functions to read and write arbitrary lenght integers.
See write_uint().

fn write_u24_be(buf: &mut [u8], value: u32) {
    BigEndian::write_uint(buf, value as u64, 3)
}

#13

Oh wow missed that, thanks!


#14

Thanks for pointing that out.

I feel bad for it.


#15

Couldn’t you use to_le (or big endian, whatever makes sense) first within that function so you always get the same byte-order in there?

Edit: Sorry if what I’m doing right now is considered thread jacking.