Generic function using variable of unknown size

1

For fun I'm trying to figure out how to write a generic function that takes a std::fs::File and reads size <T> from the file into <T>. It feels a little strange to me that when trying to create a [u8] to fill that the Rust compiler complains that it doesn't know statically the size. It seems to me that it should know at compilation based on <T> though. I know this is easily possible. I'm just not that familiar with the language.

So I'm asking for help in understanding how I help the compiler understand. The code I have so far is

pub fn write_to_type<T: Sized>(f: &mut std::fs::File) -> T {
    let mut bytes = [0u8; std::mem::size_of::<T>];
    f.read_exact(&mut bytes).unwrap();
    unsafe { std::mem::transmute::<[u8; 2], T>(bytes) }
}

The error I'm getting is

unsized locals are gated as an unstable feature [E0277]

all local variables must have a statically known size [E0277]

to learn more, visit <https://doc.rust-lang.org/book/second-edition/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait> [E0277]

the trait `std::marker::Sized` is not implemented for `[u8]` [E0277]

the size for values of type `[u8]` cannot be known at compilation time (doesn't have a size known at compile-time) [E0277]

Thoughts?

2

You can't use generic parameters in const expressions right now. That will have to wait until we get const-generics.

On another note, your function is unsafe, so it should be marked it as such.

3

This isn't just unsafe, it's catastrophically unsafe. You could use this to do write_to_type::<Box<i32>>(&some_file) to construct an owned pointer to an arbitrary memory location. Or write_to_type::<fn()>(&some_file) to execute arbitrary memory.

Frankly, this function probably shouldn't exist. Even marked as unsafe, it'd be way too easy to misuse. If you're going to do something like this, I would strongly recommend using an unsafe trait to constrain exactly which types you can use this with:

pub unsafe trait FromBytes {}
unsafe impl FromBytes for u32 {}

pub fn write_to_type<T: FromBytes>(f: &mut std::fs::File) -> T {
    // ...
}

This way, you can individually enable the trait for the small set of types for which it is actually safe to use... which is probably limited to the built-in integer and float types, and composites of those which do not contain any padding.

But, really, you should probably just use serde plus bincode (for automated [de]serialisation) or byteorder (for manual [de]serialisation).

3 Likes
4

Thanks for the reply to both KrishnaSannasi and DanielKeep. I appreciate your concern in the safety of code like this. Like I mentioned above I'm just fiddling around, really just trying to learn the language and get a grasp of things. I am aware of byteorder but really am just figuring out ways of messing with memory.

I would plan on constraining this type of code to very specific types.

I am certainly still open to more thoughts and opinions.

5

Sorry to bring this up again, but I'm still confused. Fiddling around a bit more I have the following code, and complaint from the compiler. How is it possible a primitive type can have a varying size?

image
image856×389 63.6 KB

6

You used u32 as a type parameter. This shadows the u32 type, so whenever it sees u32 in the function, it thinks that it is a generic parameter.

I think you meant to write

pub trait FromBytes {
    fn write_to_type(f: &mut std::fs::File) -> Self;
}

impl FromBytes for u32 {
    fn write_to_type(f: &mut std::fs::File) -> u32 {
        let mut bytes = [0u8; 4];
        let bytes_read = f.read_exact(&mut bytes);
        unsafe { std::mem::transmute::<[u8; 4], u32>(bytes) }
    }
}

Note that I got rid of the generic type and used the Self alias. Self is a way to talk about the implementing type when writing a trait.

next time, please post code inside a formatted block, like so

```rust
// your code here
```
7

I see that. Makes sense. Thanks for taking a few moments to answer my question. You have been great Krishna.

8

Funny enough. Announcing Rust 1.32.0 just gave me what I was trying to do painfully for free. LOL at me.

9