Is it possible to write a Generic read_integer function?

Hello,

I'm new to Rust.

I'd like to write a generic function that takes an iterator of bytes as a parameter and returns an integer.

I first tried with something like the following but it won't compile:

fn read_integer<T>(input: &mut impl Iterator<Item = u8>)
    -> Result<T, Box<dyn Error>>
    where T: Sized
{
    let mut buf = [0u8; std::mem::size_of::<T>()];

    for i in 0..std::mem::size_of::<T>() {
        buf[i] = input.next().ok_or(io::Error::from(io::ErrorKind::UnexpectedEof))?;
    }

    Ok(T::from_le_bytes(buf) as T)
}
error: constant expression depends on a generic parameter
...
error: constant expression depends on a generic parameter
...
error[E0599]: no function or associated item named `from_le_bytes` found for type parameter `T` in the current scope

So digging a little on google I found that ?Sized could help

fn read_integer<T>(input: &mut impl Iterator<Item = u8>)
    -> Result<T, Box<dyn Error>>
    where T: ?Sized
{
    let mut buf = [0u8; std::mem::size_of::<T>()];

    for i in 0..std::mem::size_of::<T>() {
        buf[i] = input.next().ok_or(io::Error::from(io::ErrorKind::UnexpectedEof))?;
    }

    Ok(T::from_le_bytes(buf) as T)
}

But it still does not compile :frowning:

error[E0277]: the size for values of type `T` cannot be known at compilation time
   --> packages/shp2bmp/src/command.rs:90:8
    |
89  | fn read_integer<T>(input: &mut impl Iterator<Item = u8>)
    |                 - this type parameter needs to be `std::marker::Sized`
90  |     -> Result<T, Box<dyn Error>>
    |        ^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time

If anyone can help I would be very grateful :slight_smile:

const issues aside, your generic bounds must cover all uses of the generic type. So you're going to need some bound that says T::from_le_bytes exists and takes a [u8; size_of::<T>()] to produce a T (or something castable to a T ... but really you just want a T).

Here's one way.

I also made your iterator input more general, but it's still sort of weird. An iterator being too short isn't inherently an IO error. It also doesn't handle the iterator being too long.

An improvement over IndexMut in some ways would be &mut T::Buf: IntoIterator<Item = &mut u8>, but it's a pain to do that with traits while simultaneously avoiding downsides, so I skipped it.

1 Like

Oh, I see. As usual I want to write things exactly as I would in C++ with template and of course it doesn't work. I hope to lose this bad habit one day :sweat_smile:

Thanks a lot!

Here's a lot simpler implementation using std traits only: Playground

You may notice I made the interface as well as the implementation more idiomatic:

  • iterators are IntoIterator and also &mut I: Iterator where I: Iterator, so taking an explicit &mut impl Iterator is the worst possible choice (it's too restrtictive in two ways for no reason)
  • I replaced the io::Error with something that actually describes the error accurately
  • I removed the need for any sort of temporary buffer by simply bit-banging the bytes into an all-0 default value, which simplifies the code and removes the necessary trait for expressing from_le_bytes().
fn read_integer<T, I>(input: I) -> Result<T, Box<dyn Error>>
where
    T: Default + BitOr<Output = T> + Shl<usize, Output = T> + From<u8>,
    I: IntoIterator<Item = u8>,
{
    let mut iter = input.into_iter();
    let mut num = T::default();

    for i in 0..size_of::<T>() {
        let byte = iter.next().ok_or_else(|| {
            format!("{} bytes are not enough to read {} of size {}", i, type_name::<T>(), size_of::<T>())
        })?;
        num = num | (T::from(byte) << (i * 8));
    }

    Ok(num)
}
3 Likes

By the way:

I don't know what you found while searching, but ?Sized is absolutely not what you want. If you want to return something by value, then it must be sized. You might have found a bad case of a red herring – the compilation errors you originally got have nothing to do with sized-ness.

2 Likes

Thank you very much !

I feel like I understand the purpose of Size but I confess that I have no idea what ?Sized means and is supposed to do.

It means "does not have to be sized".

Indeed, exactly the opposite of what I wanted :grin:.

It's syntax you won't see very often, which is why it's hard to understand. Rust has three syntaxes for naming traits, not all of which are allowed in any given place:

  1. Plain Trait, meaning "implements Trait".
  2. ?Trait, meaning "may or may not implement Trait".
  3. !Trait, meaning "does not implement Trait".

The first of those syntaxes is commonly used. Anywhere you need a generic parameter to implement a given trait, you use it or see it; however, there's a special rule for Sized which says that for type parameters and associated types, the compiler automatically assumes you wanted a Sized bound. So when you write a type parameter as just plain T, the compiler reads it as-if you wrote T: Sized, and if you write T: Clone, the compiler reads it as-if you wrote T: Clone + Sized.

This leads to the second syntax, which is only used with Sized - it allows you to tell the compiler that you don't want a Sized bound in a situation where the compiler would normally assume that you do want one. If a future edition ever adds to the set of assumed bounds (unlikely), then this syntax will be reused to allow you to remove that assumption. This means that if you write T: ?Sized as a type parameter or associated type, the compiler removes the Sized bound it would normally see on T.

And the third syntax is currently unstable, and used to indicate that a generic parameter must not implement trait. std uses it in a few places to remove auto trait implementations, but because it's unstable, only std is in a good position to use it - every compiler release, including every nightly release, reserves the right to remove this syntax or change its meaning (including changes that make users of this syntax no longer compile).

2 Likes

Thank you! Now I understand the syntax and the use case of ?Sized.

1 Like

Note that !Trait is not a bound, it's used for a negative impl. Thus, it has a different use from that of ?Trait, which is a trait bound.

3 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.