Read vec of bytes into vec of values with generic

help
1

I have the trait

pub unsafe trait NativeType: Sized + Copy {
    type Bytes: AsRef<[u8]>;
    
    fn from_le_bytes(bytes: Self::Bytes) -> Self;
}

that is implemented for a couple of native types.

How do I write a generic function to create a vector of them from a vector of bytes?

I tried the below, but I am struggling to make it compile. Note that I do not assume a specific machine endianess, thus not performing a direct pointer transmutation.

use std::convert::TryInto;

pub unsafe trait NativeType: Sized + Copy {
    type Bytes: AsRef<[u8]>;
    
    fn from_le_bytes(bytes: Self::Bytes) -> Self;
}

unsafe impl NativeType for u32 {
    type Bytes = [u8; std::mem::size_of::<Self>()];
    #[inline]
    fn from_le_bytes(bytes: Self::Bytes) -> Self {
        Self::from_le_bytes(bytes)
    }
}

unsafe impl NativeType for u64 {
    type Bytes = [u8; std::mem::size_of::<Self>()];
    #[inline]
    fn from_le_bytes(bytes: Self::Bytes) -> Self {
        Self::from_le_bytes(bytes)
    }
}

fn read_buffer<'a, T: NativeType>(values: &'a [u8]) -> Vec<T>
where <T as NativeType>::Bytes: From<&'a[u8]> {
    let chunks = values.chunks_exact(std::mem::size_of::<T>());
    assert_eq!(chunks.remainder().len(), 0);
    chunks.map(|chunk| {
        let chunk: T::Bytes = chunk.try_into().unwrap();
        T::from_le_bytes(chunk)
    }).collect()
}

fn main() {
    let a = vec![0u8, 0u8, 0u8, 0u8, 0u8, 0u8, 0u8, 0u8];
    read_buffer::<u32>(&a);
}
2

You are specifying the From trait bound in the interface, but later you use try_from() instead. Why?

First of all, in Rust, you can only use trait bounds that you specify upfront. Rust generics are not C++ templates. Second, arrays don't implement From<&[T]>, because what should that implementation do if the length of the slice is too short or too long?

Accordingly, if you use the correct trait bound, and add the Debug bound on the error type (which is needed because of unwrap()), then the code compiles.

1 Like
3

Hi,

Thank you for your help. That does indeed compile. However, it does introduce a non-trivial lifetime.

If a generic function calls read_buffer, the TryFrom<&'a [u8]> must now also be fulfilled on the parent function, which may not even have a lifetime. For example,

fn read_buffer<'a, T: NativeType>(values: &'a [u8]) -> Vec<T>
    where
        <T as NativeType>::Bytes: TryFrom<&'a [u8]>,
        <<T as NativeType>::Bytes as TryFrom<&'a [u8]>>::Error: Debug,
{
    let chunks = values.chunks_exact(std::mem::size_of::<T>());
    assert_eq!(chunks.remainder().len(), 0);
    chunks.map(|chunk| {
        let chunk: T::Bytes = chunk.try_into().unwrap();
        T::from_le_bytes(chunk)
    }).collect()
}

fn read2<'a, T: NativeType>(length: usize) -> Vec<T>
    where
        <T as NativeType>::Bytes: TryFrom<&'a [u8]>,
        <<T as NativeType>::Bytes as TryFrom<&'a [u8]>>::Error: Debug,
{
    let values = vec![0u8; length];
    read_buffer(&values)
}

fn main() {
    let ints = read2::<u32>(10);
    println!("{:?}", ints);
}

This won't compile in the example because values inside read2 can't fulfill the lifetime 'a. I get the idea that this is not needed because the bytes are being copied to over to T::Bytes, which lives in the stack, right?

4

This is what HTRBs are for. You can specify that T needs to work with any lifetime of reference:

fn read2<T: NativeType>(length: usize) -> Vec<T>
    where
        <T as NativeType>::Bytes: for<'a> TryFrom<&'a [u8]>,
        for<'a> <<T as NativeType>::Bytes as TryFrom<&'a [u8]>>::Error: Debug,
{
    let values = vec![0u8; length];
    read_buffer(&values)
}
1 Like
5

That might not work because of this issue. If it doesn't, you may have to restructure your code to avoid lifetimes there altogether.

2 Likes
6

Thanks a lot all for your help.

It did help in solving the problem. I changed the associated item to be type Bytes: AsRef<[u8]> + for<'a> TryFrom<&'a [u8]>;,

and decided to not use .unwrap, to not require "Debug" trait, via

let chunk: T::Bytes = match chunk.try_into() {
    Ok(v) => v,
    Err(_) => panic!(),
};
7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.