Why From<&[u8]> is not implemented for [u8; X]?

Hi,

I'm trying to implement the following code:

trait FromLeBytes<T, TArray> {
    fn from_le_bytes(bytes: TArray) -> T;
}

impl FromLeBytes<i32, [u8; 4]> for i32 {
    fn from_le_bytes(bytes: [u8; 4]) -> i32 {
        i32::from_le_bytes(bytes)
    }
}

fn read_le_primitive<T: FromLeBytes<T, TArray>, TArray>(bytes: &mut &[u8]) -> T
    where for<'a> TArray : std::convert::From<&'a [u8]>
{
    let (value_bytes, rest) = bytes.split_at(std::mem::size_of::<T>());
    *bytes = rest;
    T::from_le_bytes(value_bytes.try_into().unwrap())
}

Without the where clause, I get the following error:

T::from_le_bytes(value_bytes.try_into().unwrap())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::convert::From<&[u8]>` is not implemented for `TArray`

Which is expected. But with the where clause, I still get an error in the calling code:

fn read_le_i32(bytes: &mut &[u8]) -> i32 {
    read_le_primitive::<i32, [u8; 4]>(bytes)
}

read_le_primitive::<i32, [u8; 4]>(bytes)
                         ^^^^^^^ the trait `for<'a> std::convert::From<&'a [u8]>` is not implemented for `[u8; 4]`

At the same time, writing simply:

fn read_le_i32(bytes: &mut &[u8]) -> i32 {
    let (value_bytes, rest) = bytes.split_at(std::mem::size_of::<i32>());
    *bytes = rest;
    i32::from_le_bytes(value_bytes.try_into().unwrap())
}

-- works without any problem.

What am I missing here?

Thanks!

From<&[u8]> for [u8; 4] cannot be implemented because it can fail.
For example, &[0] as &[_] is &[u8], but it cannot be converted to [u8; 4] safely.

You can use TryFrom for such fallible conversion.
impl std::convert::TryFrom<&[u8]> for &[u8; 4] is provided.
https://doc.rust-lang.org/1.39.0/std/convert/trait.TryFrom.html#impl-TryFrom<%26'a%20[T]>

(However, this converts slice into a reference to an array, so you should copy it to get [u8; 4] itself.)

5 Likes

You also don't actually need From in your code, since you're already using try_into(), which just needs TryFrom.

Error message "the trait std::convert::From<&[u8]> is not implemented ..." is caused by the .try_into() call, which is actually std::convert::TryInto::try_into().
So what you should really care about is, implementing std::convert::TryFrom or std::convert::TryInto, but not From.
(See note: required because of the requirements on ... message lines.)

By adding dereferencing (copying) and some additional trait bounds, your code become compilable... however I think it should handle error more elegantly (for exmple returning Result or something, but not .unwrap()-ing.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a2ff0558becf6a0cb8a73b5dc9caa244

1 Like

impl TryFrom<&[T]> for [T; N] where T: Copy exist in stdlib, note the T: Copy bound. Anyway, how did you made a link to the trait impl of rustdoc? @lo48576

1 Like

By copying id attribute from inspector tool on web browser :sweat_smile:

6 Likes

Thanks for the explanation!

I guess I cannot use a TryInto constraint here, because this trait should be implemented by &[u8] so it's convertible to my T, and I'm not sure if I can express it in a generic constraint.

I also wonder why the non-generic code compiles -- I want to get to the bottom of constraints needed for this. After all, this is a code from the documentation, so I'd like to make it compile with generics.

Indeed, if I use a TryFrom constraint, the try_into() call compiles. But then another error appears:

T::from_le_bytes(value_bytes.try_into().unwrap());
                                        ^^^^^^ `<TArray as std::convert::TryFrom<&[u8]>>::Error` cannot be formatted using `{:?}` because it doesn't implement `std::fmt::Debug`

And then, if I add a corresponding constraint:

... for<'b> <TArray as std::convert::TryFrom<&'b [u8]>>::Error : std::fmt::Debug

-- I get another compile error:

read_le_primitive::<i32, [u8; 4]>(bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `<[u8; 4] as std::convert::TryFrom<&'b [u8]>>::Error` cannot be formatted using `{:?}` because it doesn't implement `std::fmt::Debug`

This is, of course, easily solvable by manually matching on the Result and not having this constraint at all.
But for me it's now more of an academic interest :slight_smile:

use ::std::{*,
    convert::{TryFrom, TryInto},
};

trait FromLeBytes<TArray> {
    fn from_le_bytes (bytes: TArray) -> Self
    ;
}

impl FromLeBytes<[u8; 4]> for i32 {
    fn from_le_bytes (bytes: [u8; 4]) -> i32
    {
        i32::from_le_bytes(bytes)
    }
}

fn read_le_primitive<'a, T, TArray> (bytes: &'_ mut &'a [u8]) -> T
where
    T : FromLeBytes<TArray>,
    TArray : TryFrom<&'a [u8]>,
    <TArray as TryFrom<&'a [u8]>>::Error : fmt::Debug,
{
    let (value_bytes, rest) = bytes.split_at(mem::size_of::<T>());
    *bytes = rest;
    T::from_le_bytes(value_bytes.try_into().unwrap())
}

works just fine.


In your defense, the error message was quite misleading. That's because there is a very general impl of TryFrom for types that are From (i.e., you can see an unfallible operation as a fallible one but where the error case is unreachable :sweat_smile:), and given the generality of that impl, it lured the compiler into suggesting that road rather than the most "obvious" one of having a TryFrom bound directly.


Also:

  • you don't really need the full general for<'any> TryFrom<&'any [u8]> bound, you can just have a TryFrom<&'a [u8]> where 'a is the concrete lifetime parameter used in the bytes input. Minor detail, really, since both work :slight_smile:

  • and more importantly: your trait definition had "redundant" <T> type parameter: you have to imagine that a trait is inherently generic over the type Self it is implemented for: "trait FromLeBytes<Self, TArray>" of sorts. So, rather than having afterwards impl<T, ...> FromLeBytes<T, ...> for T { ... } and T : FromLeBytes<T, ...>, you can get rid of that extra type parameter and have impl<T, ...> FromLeBytes<...> for T { ... } and T : FromLeBytes<...> :wink:

1 Like

@Yandros Thank you very much!

Indeed, coming from C# background I didn't realise that there is Self type in a trait :sweat_smile:

Regarding for<'any> -- it turned out to be not so minor! In this gist, a call to the function with a lifetime parameter (read_le_primitive_1) compiles, while a call to the function with for<'a> (read_le_primitive_2) doesn't:

fn read_le_primitive_1<'a, T, TArray> (bytes: &mut &'a [u8]) -> T
where
    T : FromLeBytes<TArray>,
    TArray : TryFrom<&'a [u8]>,
    <TArray as TryFrom<&'a [u8]>>::Error : fmt::Debug,
{
    ...
}

fn read_le_primitive_2<T, TArray> (bytes: &mut &[u8]) -> T
where
    T : FromLeBytes<TArray>,
    for<'a> TArray : TryFrom<&'a [u8]>,
    for<'a> <TArray as TryFrom<&'a [u8]>>::Error : fmt::Debug,
{
    ...
}

I really wonder why? :thinking:

This is really curious; I have managed to reduce it further down to the following absurdity:

use ::core::{*, convert::TryFrom};

fn check ()
where
    for<'any>
        [u8; 4] : TryFrom<&'any [u8], Error = array::TryFromSliceError>
    ,
    for<'any>
        array::TryFromSliceError : fmt::Debug
    ,
    for<'any>
        <[u8;4] as TryFrom<&'any [u8]>>::Error : fmt::Debug
    ,
{
    const _: () = { check; };
}

Rust agrees with the two first bounds, but fails on the third :woman_facepalming:

1 Like

Hmm, I guess I'm starting to understand :thinking:

Here's the version of the code that compiles:

fn read_le_primitive<T: FromLeBytes<TArray>, TArray>(bytes: &mut &[u8]) -> T
    where for<'any> TArray : std::convert::TryFrom<&'any [u8], Error = std::array::TryFromSliceError>
{
    let (value_bytes, rest) = bytes.split_at(std::mem::size_of::<T>());
    *bytes = rest;
    T::from_le_bytes(value_bytes.try_into().unwrap())
}

Apparently, when I set the bound for TArray to be std::convert::TryFrom, if it doesn't include the Error type, that means I don't really put any requirements on this type. Apparently, this conflicts with the next declaration, where I say that <TArray as TryFrom>::Error should actually be std::fmt::Debug.

It's both understandable and strange that there is no easy way (at least that I know of) to set trait bounds on the Error type. The only way I found is to add another generic parameter:

fn read_le_primitive<T: FromLeBytes<TArray>, TArray, TError>(bytes: &mut &[u8]) -> T
    where for<'any> TArray : std::convert::TryFrom<&'any [u8], Error = TError>,
          TError : std::fmt::Debug

At the same time, for this particular case there is no reason to have TError as a generic type.

The reason that you can't just talk about <TArray as TryFrom>::Error is that your TryFrom bound has a for clause.

This means that you are actually saying that TArray implements infinitely many traits: Namely it implements TryFrom<&'static [u8], ...> and TryFrom<&'foo [u8], ...> and so on for every lifetime. This means that you can't just talk about the error as if it's a single type — it could be a different type in different cases, e.g. the error type may depend on the lifetime in some way.

By adding an extra generic parameter and saying that Error = TError, you constrain it in such a way that every implementation of TryFrom must use the same error type. In particular this means that it can't depend on the lifetime. Since we now have the same lifetime in every case, you can now talk about it, and you had to use a generic parameter to give this common type a name.

4 Likes

Thanks a lot for the explanation!

I still don't see a reason why I can't say next that for any lifetime my Error should implement std::fmt::Debug:

for<'any> <TArray as TryFrom<&'any [u8]>>::Error : std::fmt::Debug

From my perspective, this should extend the previous declaration, meaning that for any of the implemented lifetimes the bound is like this.

Unfortunately writing that kind of bound is rather difficult, but can be done with a helper trait.

use std::convert::TryFrom;
use std::fmt::Debug;

trait HelperTryFromTrait<'a> {
    type Error: Debug;
    fn try_from2(val: &'a [u8]) -> Result<Self, Self::Error>
        where Self: Sized;
}

impl<'a, TError, T> HelperTryFromTrait<'a> for T
where
    T: TryFrom<&'a [u8], Error = TError>,
    TError: Debug + 'a,
{
    type Error = TError;
    #[inline]
    fn try_from2(val: &'a [u8]) -> Result<Self, Self::Error> {
        Self::try_from(val)
    }
}

fn read_le_primitive<T>(bytes: &mut &[u8]) -> T
    where for<'any> T: HelperTryFromTrait<'any>
{
    T::try_from2(&*bytes).unwrap()
}

playground

Hmm, looks like a more complicated way to write a function with the third generic parameter of TError :sweat_smile:

So I guess this is a limitation of the compiler. I suppose that it treats where clauses separately, that's why it cannot connect them so the second one extends the first one. I would say this is an expected behavior then :slight_smile:

The difference between this and introducing a new generic parameter on the function is that the parameter is introduced after the for-all has taken effect. Basically it's the difference between ∀lifetime ∃error and ∃error ∀lifetime.

Luckily you are only excluding TryFrom impls where the error type borrows from the slice, but I don't think the TryFrom trait allows that, so you should be good.

(But of course types that don't implement TryFrom could implement the helper trait directly and use 'a in their Error type, so that would in principle allow errors that borrow from the slice)

Still, there is a bug in the trait bound resolution, as shown by my previous post. It seems that it is a known limitation of the engine, that has trouble reasoning with universals quantifications rather than existential ones, and it seems like chalk should be able to handle this fine.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.