Help with struct holding data which implement Iterator

Hi,

I'm trying to write a data struct which should have the following properties:


// DataArray is a little more complex. I'm using tuple struct here for simplicity.
// 1.  It should be able to own data of type Vec<T> for returning from API
let data = DataArray(vec![1,2,3]);
// 2. should be able to hold a slice &[T] for passing to API
let data = DataArray(&[1,2,3]);
// 3. Should implement Iterator over &T
for i in data.iter() {
}
// 4. I should be able to take reference to internal slice:
let data = DataArray(vec![1,2,3]);
let slice:&[i32] = DataArray.data(); // or Deref, doesn't matter

I have some implementation that gives me 1 and 2.

I can't make 3 to work, because the struct takes IntoIterator which consumes the data, my naive impl allows me to iterate only once.
I can't make 4 to work for the same reason: IntoIterator consumes the data.

I'm starting to doubt that taking an IntoIterator was the right choice, but I couldn't find a common iterator struct or trait that would allow 1 and 2.

More often than not with my attempts I'm hitting the wall with GATs and can't work my way around it.

I'd appreciate any help!

struct DataArray<T, I: IntoIterator<Item=T>>(I);

struct DataIter<I: IntoIterator> {
    iter: <I as IntoIterator>::IntoIter
}

impl<T, I: IntoIterator<Item=T>> IntoIterator for DataArray<T, I> {
    type Item = T;
    type IntoIter = DataIter<<I as IntoIterator>::IntoIter>;

    fn into_iter(self) -> Self::IntoIter {
        DataIter{iter: self.0.into_iter()}
    }
}

impl<T, I: IntoIterator<Item=T>> Iterator for DataIter<I> {
    type Item = T;

    fn next(&mut self) -> Option<Self::Item> {
        self.iter.next()
    }

}


What you're trying to do is logically impossible with these interfaces.

Iterators are one-time-use-only, and IntoIterator moves ownership to the caller (i.e. destroys the object its called on). When you get items yielded by such iterator, you're the last and only owner of these items, so you can't give references to where the items were stored, because they're not stored there any more.

Vec<String> implements IntoIterator, and every time you call next() a string is logically removed from the Vec, and no longer has an address in there. The returned temporary value is the only copy, and it can't be returned by reference, because there's nothing long-lived to refer to, and it can't be iterated over again, because it has already been removed from the Vec.

When you get an iterator that is iterating over &String, then the elements aren't destroyed, and the T value happens to be a reference that works for iterating by reference. But you can't know that from your abstract interface! The IntoIterator<Item=T> has no way to declare that exceptional case. As far as the interface is concerned, those elements are non-copyable, and may or may not be references.

C++ uses iterators to describe data storage in a generic way. In Rust this approach doesn't work well - Rust iterators are for consuming data, not for holding on to arbitrary collections.

To support iteration by reference you'd have to add another trait bound like:

struct DataArray<T, I>(I) where
I: IntoIterator<Item=T>,
for<'a> &'a I: IntoIterator<Item=&'a T>;

It seems to me that Cow<[T]> technically satisfies all of the requirements you listed (although they might still not be what you are actually after): Playground

let data: Cow<[_]> = Cow::Owned(vec![1, 2, 3]);
let data: Cow<[_]> = Cow::Borrowed(&[1, 2, 3]);
    
for &i in data.iter() {
    println!("i = {}", i);
}
    
let data: Cow<[_]> = Cow::Owned(vec![1, 2, 3]);
let slice: &[i32] = &*data;
println!("{:?}", slice);
3 Likes

Thanks kornel,

I think it's exactly what I'm trying to do: I want to store either Vec or &[T] and provide an iterator over &[T] for both cases. So far I could not find any solution.

Yeah this would work for simple data like Vec and [T], unfortunately my DataArray is a multidimensional array with custom indexing, that's why I want to make it simple to use for the users. It will be returned from API with owned data and users can construct it from their data and pass as reference to other APIs.

I'll keep looking for solution. Thanks!

Can you use your own enum for this?

Update,

This is what I got so far. It ended up being much simpler than I anticipated and checks all the requirement s.

struct DataArray<'a, T> where [T]: ToOwned<Owned = Vec<T>>
{
    data: Cow<'a, [T]>,
    sizes: Cow<'a, [i32]>
}
impl<'a, T> DataArray<'a, T>
where
    [T]: ToOwned<Owned = Vec<T>>,
{
    fn new(dat: &'a [T], sizes: &'a [i32]) -> DataArray<'a, T> {
        DataArray { data: Cow::Borrowed(dat), sizes: Cow::Borrowed(sizes) }
    }

    pub(crate) fn new_owned(dat: Vec<T>, sizes: Vec<i32>) -> DataArray<'static, T> {
        DataArray { data: Cow::Owned(dat), sizes: Cow::Owned(sizes) }
    }

    fn data(&self) -> &[T] {
        self.data.as_ref()
    }

    fn data_mut(&mut self) -> &mut [T] {
        self.data.to_mut().as_mut()
    }
    fn sizes(&self) -> &[i32] {
        self.sizes.as_ref()
    }
}

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.