More trouble with boxed slices

Well, even if "auto deref" would solve the problem (maybe with other implications/downsides), I'd be satisfied if I could manually deref like I can do with borrows_foo in this Playground (which doesn't work with moves_foo easily). That's what I tried to say in my quote above.

Having to provide these seems to be like a lot of boilerplate whenever you create and implement your own traits. So you basically mean I should always do this?

pub trait Foo {
    fn foo(&self);
}

impl<T> Foo for &T
where
    T: Foo + ?Sized,
{
    fn foo(&self) {
        (**self).foo()
    }
}

impl<T> Foo for &mut T
where
    T: Foo + ?Sized,
{
    fn foo(&self) {
        (**self).foo()
    }
}

impl<T> Foo for Box<T>
where
    T: Foo,
{
    fn foo(&self) {
        (**self).foo()
    }
}

impl<T> Foo for std::rc::Rc<T>
where
    T: Foo,
{
    fn foo(&self) {
        (**self).foo()
    }
}

impl<T> Foo for std::sync::Arc<T>
where
    T: Foo,
{
    fn foo(&self) {
        (**self).foo()
    }
}

impl<T> Foo for std::borrow::Cow<'_, T>
where
    T: Foo + ToOwned,
{
    fn foo(&self) {
        (**self).foo()
    }
}

// … when does this end?

(Playground)

I can't follow here. Which "traits" do you mean specificially? Something like Foo above? Or something like Box? I assume you are referring to fundamental type constructors here? Why would adding impl<T> Foo for Box<T> be a breaking change for that crate in the above Playground?

Would it be a breaking change for std to add an impl<T, I: SliceIndex<[T]>> Index<I> for Box<[T]>, like @H2CO3 proposed in the very first response?


I understand. It's a bit tragic from an aesthetic p.o.v. though. And it's not only aesthetic because people keep running into friction (though I admit that when you try to write your code more "smartly" you seem to run into friction more often than if you just use Vec<T> and don't care).

I'm exploring this problem not just to provide a nice API in the real-world use case (it's still a goal though), but also to understand better how to write idiomatic Rust in general. It's not the first time I ran into limitations of Rust's type system, or into questionable or missing implementations of traits in std. I still think that Rust's type system overall does a good job, but… something bugs me about it because I keep running into problems here and there. I think I just have to deal with Rust not being perfect here. I still would like to understand, however, how to deal with the particular scenario(s) discussed here, in order to improve my relationship with Rust :grin:. I'll try to come up with some concrete questions on that matter at the end of this post.

But collecting directly into a Box shouldn't involve shrinking if the iterator provides a correct size_hint, right?

A potential problem I see with the borrowing case is that you can't store the returned type along with the owned_container somewhere (because safe Rust doesn't allow self-referential types).

I already accept writing &* where needed. But always infecting a return value with a lifetime is a different issue. This might not be a problem in my particular use case though, because the return value is infected with a lifetime 'a anyway (because of working on &'a self).

The docs I quoted are not just about method dispatch but about "index expressions" in particular. But I understand how this is the same as method invocation: I can invoke a method of a trait implemented by the Deref::Target type, but this doesn't make my outer (Deref) type implement the trait. The same holds for indexing operations (which get forwarded) and fulfilling the Index/IndexMut traits (which get not forwarded). So Rust is "consistent" here. Yet it causes friction in both cases (not that I have a solution for it).

&mut Indexable would likely be the workaround to choose (when needed), especially considering that the returned impl 'a + Iterator is lifetime-infected anyway (due to &'a self). So I feel more confident with that approach now (GitHub link again).

I think in my concrete case, the caller might never have to (as I said, the impl 'a + Iterator captures a lifetime anyway).

:unamused:

… might not be such a big deal, but I'll think about it. Thanks for the warning. (It's not an important crate and I'm still at 0.0.1 but maybe at some point I want some more stability.)

I think if std implemented Index for Box<[_]>, I'd rather get rid of the Deref approach. Is that a breaking change for std? For smart-pointers which fail to provide that implementation, there is still the "newtype escape hatch".

Oh, that is an interesting solution too! But I think in this case it wouldn't cover more cases than what I can cover with the Deref approach.


In this post, I promised to get back to some concrete questions:

  1. When I implement a generic smart-pointer, should I provide forwarding implementations for all traits in std? (Note that it's a lot of boilerplate.) To give an example, should deref_owned::Owned provide a blanket implementation for Index, IndexMut, Shl, Shr, AddAssign, Not, etc.?
  2. When I provide a new trait Foo, when should I add blanket implementations for &T, &mut T, Box<T>, Rc<T>, etc. where T: Foo or T: Foo + ?Sized, respectively?
  3. What is generally/usually the best way to consume a list of values that don't need to be extended, assuming I need random access[4]? Just use Vec<T> and forget about Box<[T]>?

Not sure if I forgot any question.

Anyway, your responses have already been very helpful for me to understand the practical issues with Rust's type system better (and why I experience certain friction here and there). :heart:


Update:

I tried to answer some of these questions on my own by looking into what std does and what the language provides:

struct Smart<T>(pub T);

impl<T> Deref for Smart<T> {
    type Target = T;
    fn deref(&self) -> &T {
        &self.0
    }
}

fn add<T: Add<T>>(a: T, b: T) {}
fn cmp<T: Ord>(a: T, b: T) {}
fn idx<T: Index<usize>>(x: T) {}

fn main() {
    //add(Rc::new(1), Rc::new(2)); // doesn't work
    //Rc::new(1) + Rc::new(2); // doesn't work
    add(&1, &1); // there are concrete implementations though
    cmp(Rc::new(1), Rc::new(2)); // `Rc` forwards `Ord`
    Rc::new(1) < Rc::new(2); // this wouldn't work otherwise, …
    //Smart(1) < Smart(2); // …because `Smart` is dumb
    //cmp(Smart(1), Smart(2)); // also fails because `Smart` is dumb
    idx([1]); // arrays implement `Index`
    //idx(Rc::new([1])); // `Rc` doesn't forward `Index`
    Rc::new([1])[0]; // but the operator gets forwarded (opposed to `+`)
    Smart([1])[0]; // so this works too
}

(Playground)

So trying to answer my question #1 above:

  1. When I implement a generic smart-pointer, should I provide forwarding implementations for all traits in std?

Following what std does, a generic smart-pointer shouldn't forward all traits. Exceptions are probably Ord (as well as PartialOrd, Eq, PartialEq), Display (as well as Debug), maybe Default and Hash. But not Add, Sub, etc.

A curious case, however, is Index. The standard library decided to not make Rc, Arc, etc. forward this. (Instead, it provided a concrete implementation for [T].) The language, in contrast, will perform Deref based lookup for the index operators when indexing a smart pointer. But this breaks when using Index as a trait, as shown throughout this thread.

Index and IndexMut differ fundamentally from all the other traits in std::ops (except Deref and DerefMut), because only for these, the language has built-in forwarding (when a type implements Deref). This explains why missing generic implementations of Index for smart pointers don't usually cause any trouble – because unless you want to use Index as a trait bound, you'll not experience any trouble (see Playground above, where the expression Smart([1])[0] works fine, even though Smart doesn't provide a forwarding implementation for Index).

I feel like std should provide a generic impl<U: Deref<Target=T>, T: Index<I>, I> Index<I> for U, but I'm not sure if that would cause trouble. Likely it's impossible to add such an implementation now (I don't really overview the consequences).

Given the fact that none of std's generic smart pointers implement Index, I feel like doing so in third party crates would be unidiomatic.

I guess in the end, Index is just broken as it is right now (when it comes to generics). Or maybe "broken" is a too harsh word, and you could say "not suitable for generic use". Hypothesis: Maybe Index should be avoided in bounds and only used for operator overloading then?

Following that hypothesis and getting back to my original problem, I wonder if @2e71828's proposal to use AsRef<[U]> is the way to go then. However, I previously argued that AsRef isn't meant for dereference (see also PR #99460 and Issue #45742). This leads me to the conclusion that maybe Borrow<[U]> would be the "cleanest" solution (Playground).


In regard to the actual use-case, it would look like this: GitHub commit.


  1. Specifically, you're allowed to implement an upstream trait for &T (or &mut T or Box<T>), so long as T is a local type. Thus, adding new "blanket" implementations which would conflict with such a downstream impl is a breaking change. ↩︎

  2. without introducing new traits ↩︎

  3. whatever that means ↩︎

  4. If I don't need random access, I can take an Iterator, of course. ↩︎