Hack to specialize `W: Write` for `Vec<u8>`?

help
1

I have an implementation that uses io::Write, but could be much more efficient for Vec<u8>.

I'm looking for ways to implement this on stable, but none are perfect so far:

  1. There's a trait vs inherent method trick, but that is syntactic, and doesn't work with trait bounds. I'd rather not have a macro as my public interface.

  2. Box::downcast requires explicit Any bound, but Write + Any would be too limiting for my API.

  3. if TypeId::of::<W>() == TypeId::of::<Vec<u8>>() is almost perfect, but… it requires a 'static bound.

I'm thinking that a Vec<u8> is going to be static anyways, so I'd like to hack around the 'static requirement. I thought I could cheat it by casting or transmuting W to W + 'static somewhere, but I can't find a loophole.

Do you know a hack for this to "transmute" the trait bound?

2

I'm aware of this trick (you left a like on that post, too, by the way) to detect trait implementations based on our existing specialization of Clone for types like arrays with Copy elements - which you could then use with your own marker trait only implemented for Vec<u8>.

(Or more concisely/directly [without needing a new trait at all] following the example in the linked post, use a struct IsVecU8<'a, T>(&'a Cell<bool>, PhantomData<T>) and directly implement Clone [modifying the cell] for IsVecU8<'a, T> for all T, but Copy only for IsVecU8<'a, Vec<u8>>.)

Of course, this is still very much a "hack", if anyone tries this - once lifetimes are involved (which they aren't for Vec<u8>, this can suffer the usual soundness issues of specialization - and also future Rust versions may remove the specializing code for cloning arrays, though it doesn't seem very likely to me (and also in this case, that would only mean a performance regression in falling back to the general Write approach).

2 Likes
3

Maybe you can find some idea in Rust RFC 1210?

4

I know you said "on stable", but since people are interested in using TypeId in more places anyway, maybe it would be fine to have a TypeId::if_static that checks for lifetimes and returns Option<TypeId>?

That'd still be enough to detect Vec<u8>, and also keeps TypeIds from ever existing for non-'static things...

3 Likes
5

This would still encounter the exact problem that I’ve described in my first answer of the linked thread. TypeId::if_static would return Some for your struct Foo(&'static str); (field is private) but if you later change it to Foo<Arg = &'static str>(T); which is supposed to be a non-breaking change, suddenly, it would have to return None.

For the case of Vec<u8> it’s very very unlikely that that would ever add a new defaulted type argument, with a lifetime defaulted to 'static; and if we ever did have such a change to Vec anyways, then the trick I described above would indeed unsoundly identify Vec<u8, Global, NewDefaultedArg<'non_static>> to be the same as Vec<u8> == Vec<u8, Global, NewDefaultedArg<'static>>. (Well, technically, the actual unsoundness would then result out of the subsequent transmute.)

(Same thing technically applies not only to adding such a parameter to Vec, but also to u8 or to Global.)

The two ways out that I see would be

  • allow types to actively promise to never become non-'static, and only allow TypeId::if_promised_static on those
  • offer - either as a full “specialization” feature or as a TypeId::if_static-style API - a way to specialize strictly for performance optimization purposes with the expressed non-guarantee of specialization, which would mean the compiler can, but doesn’t guarantee to choose the more specialized implementation over the default one. This would then mean that the kind of changes described above would explicitly be non-breaking - and as part of a more complete actual specialization language feature, the compiler could simply decide conservatively not to specialize on any trait implementations that are conditional on lifetime relations. (As part of a more complete actual specialization language feature, this might probably also make it really hard to actually specialize any associated types in a useful manner - or at least, they’d need to behave opaque to users in most places, even when the concrete instantiation of all generics is known.)
1 Like
6

Great! I forgot about that trick. Here's the implementation:

pub(crate) fn is_vec_u8<T>() -> bool {
    use std::cell::Cell;
    use std::marker::PhantomData;

    struct IsVecU8<'a, T>(&'a Cell<bool>, PhantomData<T>);
    // Rust specializes Copy and doesn't call Clone
    impl Copy for IsVecU8<'_, Vec<u8>> {}

    impl<T> Clone for IsVecU8<'_, T> {
        fn clone(&self) -> Self {
            self.0.set(false);
            Self(self.0, self.1)
        }
    }
    let cell = Cell::new(true);
    let _x = [IsVecU8::<'_, T>(&cell, PhantomData)].clone();
    if cell.get() {
        assert_eq!(std::mem::size_of::<T>(), std::mem::size_of::<Vec<u8>>());
        assert_eq!(std::mem::align_of::<T>(), std::mem::align_of::<Vec<u8>>());
        true
    } else {
        false
    }
}

#[test]
fn spec() {
    assert!(!is_vec_u8::<bool>());
    assert!(!is_vec_u8::<f32>());
    assert!(!is_vec_u8::<String>());
    assert!(!is_vec_u8::<Vec<i32>>());
    assert!(is_vec_u8::<Vec<u8>>());
}
6 Likes
7

Specialization based on TypeId (and subsequent transmutes) is quite desperate, so it would be great if min_specialization could be pushed to the finish line instead.

8

And all these asserts optimizes to nop as well! Compiler Explorer.