Panic-less and allocation-less [u8;N] to [u64; N/8] in big endian

help
1

I'm trying to make a fast u8 to u64 array conversion. That is, take the u8 slice elements 8 by 8 and transform them into u64 big endian. The problem is that I'm collecting them into a Vec just so I can get the result and return if something is wrong. I didn't want to use allocations. I pass the result as a parameter to avoid allocations also.

use std::array::TryFromSliceError;

#[derive(Debug)]
pub enum ArrayConversionError {
	TryFromSliceError(TryFromSliceError),
}

impl From<TryFromSliceError> for ArrayConversionError {
	fn from(_e: TryFromSliceError) -> Self { 
	    todo!()
	}
}

pub fn u8_to_u64_array(
	array: &[u8],
	result: &mut [u64],
) -> Result<(), ArrayConversionError> {
	for (i, v) in array
		.chunks(8)
		.map(TryInto::try_into)
		.collect::<Result<Vec<[u8; 8]>, TryFromSliceError>>()?
		.iter()
		.map(|byte_slices| u64::from_be_bytes(*byte_slices))
		.enumerate()
	{
		result[i] = v;
	}
	Ok(())
}

fn main() {
    let mut result = [0u64;1];
    u8_to_u64_array(&[0, 0, 0, 0, 0, 0, 23, 255], &mut result).unwrap();
    println!("{:?}", result);
}

Is there something that can be done so no allocation is needed?

2 
#[inline]
fn convert<const N: usize, const N8: usize> (
    bytes: [u8; N8],
) -> [u64; N]
{
    assert_eq!(N.checked_mul(8), Some(N8));
    if N == 0 { return []; }
    let mut iter = bytes.array_chunks::<8>();
    [(); N].map(|()| u64::from_be_bytes(*iter.next().unwrap()))
}

If you can't use .array_chunks() because you're not on nightly, you can use .chunks(8).map(|it: &[u8]| -> &[u8; 8] { it.try_into().unwrap() }) to polyfill it.

The branches, including the panicking branches, are removed once an explicit choice of N is given, since they're unreachable (provided you give correct values of N and N8).

  • You can achieve not having to bother writing the array lengths with a macro:

    macro_rules! convert_native_endian {( $const_array:expr $(,)? ) => ({
    const __SRC: [::core::primitive::u8; $const_array.len()] = $const_array;
    #[forbid(const_err)]
    const __DST: [::core::primitive::u64; __SRC.len() / 8] = unsafe {
        ::core::mem::transmute(__SRC) /* compile error if `__SRC.len() % 8 != 0` */
    }
    __DST
}) pub(in crate) use convert_native_endian;

    (it can be amended to always use big endianness, although it would require more fiddling and it wouldn't be pretty)

2 Likes
3

Here's one way to do this on nightly, with only one bounds check (the necessary one to make sure there's enough space in the result slice):

#![feature(slice_as_chunks)]
pub fn u8_to_u64_array(
	array: &[u8],
	result: &mut [u64],
) {
    let n = result.len();
    let (result, array) = (&mut result[..n], &array.as_chunks().0[..n]);
    for i in 0..n {
        result[i] = u64::from_be_bytes(array[i]);
    }
}

https://rust.godbolt.org/z/z11zEKTeW

The core of it is as_chunks on slice, which does the slice-of-T to slice-of-arrays-of-T conversion.

2 Likes
4

Assuming buffers have correct lengths and compiler is able to track this fact, ByteOrder::read_u64_into does not require any allocations and will compile down to a binary without panics (select "ASM" instead of "Run" to see generated assembly).

5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.