Slices by value

Rust slice access is great for extensional structures—AsRef<[T]> lets you access Vec<T>. For intensional structures (e.g., compressed, succinct, implicit, ...) is a disaster, as there's no way to return references. In some cases you can play with references to constants (e.g., returning &true/&false to implement Index on a bit vector, as many crates do), and if you give up on concurrent access you can return a reference to a field of the structure used for that purpose (:confounded_face:).

Assuming this is not possible, we would like to design reasonable traits for "slice by values" that are consistent with slices.

That would probably go by as in this playground. Then you would require something like Length + IndexGetValuefor just reading, and possibly add IndexSetValuefor writing.

I think also IndexGet and IndexSet would be valid names, like in this stalled RFC, but certainly we cannot use get and set as method names or ambiguity would be a disaster.

Comments? Ideas? Would you use different names/signatures? We've looked in crates.io but there's nothing general.

I'm not sure I follow. I thought the whole idea of a slice is that it is not a value, it's a view into a collection of values.

Yes, but “by value” refers to the way you get access to the values in the collection. Index returns a reference, so if your slice-like thing does not store extensionally its values, like a slice does, but rather represent them intensionallly, like a compressed bit vector, you cannot implement Index as there's nothing to refer to.

An alternative design is to address directly slices-by-value.

I think you mean indexing by value, slices don't really have much if anything to do with this per se. Your example code does not deal with slicing, which would be indexing with a range, like foo[1..3]. Being able to return (generalized) slices by value would require the ability to return dynamically-sized types (DSTs) which would certainly be nice but also quite nontrivial to implement.

1 Like

Please look at the second example. Maybe you'll understand.

Meta-comment:

I think part of what is confusing is that there's nothing intrinsically slice-specific in your trait examples themselves, only in how you use them. Namely, IIUC, you want the logical expectation of Length + IndexXxxValue<usize> to be that the valid indices are exactly everything in 0..thing.len(). That expectation would be in the documentation presumably. But in the absence of that expectation, the Length methods make sense for any collection, independently of what types the indices can be or which values will succeed.

You aren't talking about actual slices, but about things that index by usize like slices do.

Also your examples don't demonstrate a use case where you can't return by reference. Your IndexGetValue example looks like something you would blanket-implement for any C: Index<Idx, Output: Clone>, for instance. But the use cases you actually care about are those where Index is not an option.


Substance-comment:

I think you need to decide how general you want to be. The topic so far reads to me like you want to be as general as possible and do something Index-esque, but also incidentally support your use case. At that level of generality, the assumptions about how indexing by usize work (0..len() are the current indices) won't always be valid, which you'll have to handle some way; i.e. they're a somewhat orthogonal concern to the reference constraints of Index.

Without language support so that one can use actual indexing, the "work around Index's reference constraint" portion of this feels like trying to get the ecosystem to agree on some standard getter-setter traits, and I'm not too surprised that hasn't happened.

Thinking about the general solution anyway... sometimes people run into the reference constraint because they want to return owned values, but another possibility is that they wish they could return other borrow-capturing data structures instead of references, like a MutexGuard<'_, T>. (Though this comes up more with traits like Borrow I think.)

A more general approach for that issue would be something like

trait GetByKey<Idx> {
    type Output<'this> where Self: 'this;
    fn contains_key(&self, idx: Idx) -> bool;
    fn get_value(&self, idx: Idx) -> Self::Output<'_>;
}

(It would have the usual GAT rough edges if GATs were actually used, but this form is more digestible.)

The setting case doesn't need the lending ability, if it only supports setting (versus handing back something like a lock that can be passed around for the sake of mutation elsewhere).

pub trait SetByKey<Idx> {
    type Value;
    fn is_settable(&self, index: Idx) -> bool;
    fn set_value(&mut self, index: Idx, value: Self::Value);
}

(If you wanted to support things like += ..., you would need the lending ability. That doesn't seem like your actual use case so I'll ignore it.)

If you then try to generalize the "what are the current indices" portion of your actual use case, you could end up with something like:[1]

trait KeyIterable {
    type KeyIter<'this> where Self: 'this;
    fn current_keys(&'this self) -> Self::KeyIter<'_>;
}

And your by-value slice-like case could be...

trait OwnedSliceLike
where
    Self: SetByKey<usize>
        + for<'a> GetByKey<usize, Output<'a> = Self::Value>
        + for<'a> KeyIterable<KeyIter<'a> = Range<usize>>,
{}

The set of valid keys being a Range<usize> encodes your "slice-like usize indexing" condition.

Here's what it all may look like together (in alternative-to-GAT form). It does have a very "Collection trait" feel.


Alternatively, you could take a more targeted approach to what seems to be your actual use-case: slice-like usize based indexing. In which case, you can throw out the generic Idx on everything, and document or encode the valid key assumptions into the traits.

pub trait GetSliceLike<'this, Guard = &'this Self>: SliceLike {
    type Output;
    fn get_value(&'this self, index: usize) -> Self::Output;
}

pub trait SetSliceLike: SliceLike {
    type Value;
    fn is_settable(&self, index: usize) -> bool;
    fn set_value(&mut self, index: usize, value: Self::Value);
}

pub trait SliceLike {
    fn len(&self) -> usize {
        let range = self.current_range();
        range.end.saturating_sub(range.start)
    }
    fn is_empty(&self) -> bool {
        self.current_range().is_empty()
    }
    fn contains_index(&self, index: usize) -> bool {
        self.current_range().contains(&index)
    }
    fn current_range(&self) -> Range<usize>;
}

And if you don't need to support borrowing gets, throw that out too.

// No lifetime and uses `Value` now, could probably merge into `SliceLike`
pub trait GetSliceLike: SliceLike {
    fn get_value(&self, index: usize) -> Self::Value;
}

// `Value` moved to `SliceLike`
pub trait SetSliceLike: SliceLike {
    fn is_settable(&self, index: usize) -> bool;
    fn set_value(&mut self, index: usize, value: Self::Value);
}

pub trait SliceLike {
    type Value;
    // ...same methods as before...
}

  1. the iterator borrowing *self is certainly common for general collections that have key iterators ↩︎

Yes, you're right in that I conflated two different issues—the lack of a language-level trait for a random-access list (which in abstract term is what I need) and the fact that it would be really nice to have overloading for the getter/setter methods. But I find strange that there's no such trait—all languages I know with interfaces or traits have something like that. Say, Java's List and the marker trait RandomAccessList.

Your last solution is almost identical to our second solution—even the method names—except for the range genericity, so I guess that there's where we have to go. The main difference is that we named the trait ValueSlice and ValueSliceMut.

Why do you think an is_settable method is required? Isn't sufficient to use trait founds for ValueSliceMut?

Iterability is another complicated issue. Similarly to random-access lists, Rust does not have an Iterable trait and relies on implementing IntoIterator on references, which however means that to ask for iterability you have to write where trait bounds, and that sometimes becomes ugly (and has to be manually passed through). We are pondering whether to have an Iterable trait or just have for<'b> &'b S: IntoIteratorFrom<Item = usize> where it's needed (note the difference with, say, Vec, which would return references).

That was partially inertia from starting with the more general approach, but even with something slice-like, the set of settable keys may be different than the set of gettable keys.

  • E.g. a Vec-like could model push as setting the self.len() index
  • E.g. a Vec-like could set any index[1] if the intermediate values can be Defaulted

But it might not be required if you don't need such distinctions.


  1. up to isize::MAX bytes ↩︎

Getting back to the "iterable" problem, we came up (a bit obviously) with this design. (I think early versions of Rust had an Iterable standard trait, but for some reasons it has been dropped—it seems like the people in charge of the language have little or no interest in collection traits). Once again, I couldn't find anything in crates.io of this kind. There are a few specialized implementations.

The idea of IterableFrom comes from the fact that for many data structures creating an iterator and skipping is much more expensive than starting from a given point. So by implementing IteratorFrom you are saying that you can do that, fast.

GATs didn't exist before 1.0 (the only time when a stable std trait could have dropped). They stabilized in Rust 1.65 (November 2022).

Yes, but in principle if you are interested just in values (as it happens in our case) you can do like the Iterable trait of the orx-iterable crate. No GATs needed.

But, you're right: as much as lenders, Iterable was one of the motivations for GATs. I guess however that if the returned iterator has a lifetime you end up with the same problem we discussed a few weeks ago—constraining the returned item would be complicated as you have a lifetime in the middle (I haven't tried [yet]).