Read a u64 from a bunch of bytes

Hello everybody!

Let's say that i have a routine in which I essentially:

  1. calculate the number of bytes to read from vec, based on symbol;
  2. create a u64 from the next bytes_to_fetch;

in order to remember which is the last position visited in the vec I use a mutable reference to a counter named last_byte_taken.

ps: bytes_to_fetch can be from 1 up to 4 (included).

Now i have something like that:


fn my_routine(vec: &Vec<u8>, symbol: u64, last_taken_byte: &mut usize) -> symbol {
  let bytes_to_fetch = ... some calculus ... // calculate how many bytes i have to read

  let mut bytes = vec // take a slice of the needed bytes
            .get(*last_taken_byte..*last_taken_byte + bytes_to_fetch)
            .unwrap();

   *last_taken_byte += folds as usize;
   symbol | bytes.read_uint::<BigEndian>(bytes).unwrap()  // bitwise or between the two u64s
}

My problem here is to build efficiently the u64 from a number of bytes that can change. This solution works well but the read_uint (from the byteorder crate) takes a lot of time in order do it. I'm trying to figure out if there is a better way to do it.

I think u64::from_be_bytes is exactly what you want. To make it fit exactly 8 bytes, I would simply pad the upper bytes with zero.

For example:

let mut bytes = [0; 8];
bytes[8 - bytes_to_fetch..].copy_from_slice(&vec[last_taken_byte..][..bytes_to_fetch]);
let number = u64::from_be_bytes(bytes);

You could also do some quirky tricks (including evil unsafe code) to make this faster, but without some perf data to show that this is a bottleneck, I would leave it as is.

1 Like

There's a bunch of constructors for converting bytes to various fixed-size integers. You won't get any more efficient than those.

Also, you should just use the Read impl of slices:

fn read_u64(bytes: &mut &[u8], len: usize) -> io::Result<u64> {
    let mut buf: [u8; 8] = [0; 8];
    bytes.read_exact(&mut buf[..len])?;
    Ok(u64::from_le_bytes(buf))
}

Playground

5 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.