Constraint for size_of::<T>() in constant expression

Hi all,

I'm trying to read from a binary file and since the code to read a i32 and u8 and so on, are almost identical I wanted to use generics to reduce code repetition:

use std::fs::File;
use std::io::{Read, Error};
use std::marker::Sized;
use num_traits::{FromBytes};

trait ReadBinary<T: FromBytes + Sized> {
    fn read_be_binary(&mut self, buf: &mut T) -> Result<usize, Error>;
}

impl<T: FromBytes + Sized> ReadBinary<T> for File {
    fn read_be_binary(&mut self, buf: &mut T) -> Result<usize, Error> {
        use std::mem::size_of;
        let mut bytebuffer = [0; size_of::<T>()];
//        let result = self.read(bytebuffer);
//        let buf = T::from_be_bytes(&bytebuffer);
//        result
        todo!()
    }   
}

If I write this function for i32 explicitly everything works fine, but If I use a type parameter the compiler complains that the constant expression [0; size_of::<T>()] might fail, depending on T. This makes sense to me, because I can imagine that T might take the form of some type of which the size is not known at compile time, thus making it impossible to get a compile time array. I wanted to constrain T to everything that has a known size at compile time and after a quick search I found the std::marker::Sized trait which states: "Types with a constant size known at compile time."
However, I still get the same error:

  --> src/readbin.rs:13:34
   |
13 |         let mut bytebuffer = [0; size_of::<T>()];
   |                                  ^^^^^^^^^^^^^^
   |
   = note: this may fail depending on what value the parameter takes

What am I missing and how should I know which trait bound to add? It would be really nice If the compiler would suggest something like "size_of only works for types that implement the trait foo, bar, ..." or something more helpful.

Thanks for your help.

Googled the error message (you should always include the complete error message, not just part of it), and found a StackOverflow answer that I think is relevant.

To summarise: this isn't allowed because the current approach is buggy and unsound, and there is no agreed-upon, stabilised alternative to use instead.

This specific use might be theoretically okay, but I get the impression from some of the pages I found that there were so many problems caused by "types dependent on const expressions dependent on types" that the whole category was banished to the shadow realm. Nuked from orbit. Consigned to the memory hole. Made persona non grata. Politely asked to leave the establishment. Run down the curtain and joined the choir invisible.

Insofar as I can tell, you aren't missing anything, it just doesn't work. Hopefully it will at some point.

Until then, you might just have to vec![0; size_of::<T>()];.

1 Like

Just don't. Generics in Rust are very ill-suited to support that use-case. They are, well, generics, they are designed to work with open set of types and when you try to apply them to closed set they work, but code becomes extremely complicated and hard to both read and write.

If you want to handle just two or three fixed types then macros are the way to go.

1 Like

Thank you for your reply.
It is quite unfortunate that it does not work.
I will try khimru's approach and try macros (Never did that before, but now might be the right time to tackle it).
Quick question: Why do generics exist at all, if it can be all done by macros?

Generics exist to handle the use-case where set of choice is open: e.g. it would be pretty hard to implement vector with macros because you would need the ability to list all types of all vectors in your program upfront.

And when you work with open set of types then generics work pretty well: you declare, upfront, what properties supported types need to have and do the implementation using only these properties.

But when you implement algorithm using just fixed set of types you often, implicitly, use lots of such properties and even don't notice that. You can copy integers, you can do arithmetic operations with them, you can compare them, heck, you can assign 2 or 3 to them!

When you write generics all these abilities couldn't be taken for granted (could you assign 2 or 3 to Vec<String>? what would that even mean?) and if you try to list all the abilities you actually need you end up with huge and unwieldy constraints.

And if you want to ensure that your code only works with a fixed set of types all that work, is, mostly, exercise in futility: why not just ask the compiler to verify that all needed abilities are there? That's exactly what happens if you use macros.

Thus, as a rule, you use generics when you don't know which types you plan to work with and macros when you do know which types you want to process.

It's a rule, not law, though. Sometimes situation is not clear cut, e.g. you want to implement something only for types you know in advance, but then want to add these types easily, etc.

You recently asked a question about reading/writing bytes of primitive integers. You should probably follow the approach of num_traits and use a trait that defines the appropriate byte buffer as an associated type for each value type. You could in fact just use the ToBytes trait that we already discussed there.

Generics are not only used for emitting similar-looking code for a set of types. They are also the tool for enforcing type-level relationships and encoding invariants in the type system. Macros are a much more primitive, syntax-only device. You can't use macros to manipulate types.

3 Likes

Yes I asked a similar question. Thats why I used the FromBytes trait this time (I learned :wink: ). But the problem with reading bytes is that buffer size, which is no problem in the write case from my previous question:

impl<T> WriteByte<T> for File where T: ToBytes {
    fn write_be_binary(&mut self, val: &T) {
        let buf = val.to_be_bytes();
        self.write_all(buf.as_ref()).expect("Unable to write to file");
    }   
}

If there is a way to circumvent the buffer problem I would be happy. But a buffered [u8] solution is what I found online so I started from there.

But that's the whole point: why don't you just use the Bytes associated type if you already require the FromBytes bound? That associated type is the appropriate buffer type. This works:

use std::fs::File;
use std::io::{Read, Error};
use num_traits::FromBytes;

trait ReadBinary<T> {
    fn read_be_binary(&mut self) -> Result<T, Error>;
}

impl<T> ReadBinary<T> for File
where
    T: FromBytes,
    T::Bytes: Default,
{
    fn read_be_binary(&mut self) -> Result<T, Error> {
        let mut buf = T::Bytes::default();
        self.read_exact(buf.as_mut())?;
        Ok(T::from_be_bytes(&buf))
    }   
}
3 Likes

Thanks.
That looks completely different from the write solution. I think I am not suited for Rust. Everything I see in the forum looks completely different from everything else. I am unable to see the logic behind all this. Maybe I'm just dumb, but I think I will give up with Rust and return to C. Instead of searching for a solution to a trivial problem for several days and then having to bother people who tell me how trivial it is (which I think it is not), I can spit out a C-based solution in seconds.
The memory problems or out of bound errors or the existence of Null never bothered me, so the lack of those have never been a reason to try Rust. I really like the ease of the testing in the cargo system, as well as the algebraic types which can hold values. Also the strict matching against all varieties of an enum are a nice touch, but I do that anyways, so I have no need for a language to tell me to do it.

Thank you for answering all my questions and your patience, but I think I am not made for Rust, or Rust is not made for me.
Maybe I'll return some day.

What's "the write solution"? And why is it an issue that it looks different from it?

The above solution I offered is approximately trivial. It's short, idiomatic, and 100% deducible from the documentation.

I suspect that you might be new to generic programming. That's a separate skill on its own to learn, and the language is not at fault for some users' unfamiliarity. You probably need to take things slower and learn by building less ambitious projects, familiarizing yourself with one aspect of the language at a time.

If you don't know how to work with generics and build abstractions using traits, then work with concrete types until you have a working solution. Then look at experienced people's code for problems and solution patterns relevant to your use case until it clicks. The worst thing you can do is expecting that writing Rust will
be exactly the same experience as writing C or any other language.

2 Likes

I assume "the write solution" refers to your solution on a previous post for a method writing an arbitrary number to a byte slice.

@SpinTensor The difference between these two is that, for writing, the Bytes associated type is an output, so the ToBytes trait is sufficient to obtain a value of it. For reading, Bytes is used as an input, so a way is needed to produce a value of it before FromBytes can be used; Default provides an easy way to produce a value, and the solution just applies it as an extra bound on Bytes.

3 Likes