How to make a struct field generic for Vec and array

Hello folks,

I have a struct (and associated methods) where one of the fields could in principle be implemented as a Vec<f64>, or as a fixed-length array [f64; D] for some const D. In other words, it's expected that the stored Vec/array will have fixed length for the entire lifetime of the (mutable) struct.

My first implementation used const generics to fix the number of dimensions D and used [f64; D] for data storage. However, that relies on a nightly-only feature, and there are some limits on what features are available for arrays longer than 32.

So, I decided to rewrite to use a Vec instead. This worked fine, but halved the speed of the code (not really a surprise, as particularly with smaller length I imagine fixed size arrays are much easier to optimize).

So, having thought this through, it occurred to me that the best implementation would be to make the struct generic over the storage type, with the only constraints being that (i) it must be either an array or Vec and (or perhaps, that it must allow array-like interaction), and (ii) it must have floating-point elements.

Ideally it would also be possible to denote other fields as having the same type as the array/Vec element type.

In other words, I want to be able to craft a struct of the form

struct MyStruct<T> {
    vec_or_arr: T,
    some_float: {element-type-of-T}
    other_float: {element-type-of-T}
}

where T is constrained to being array-like or Vec-like.

(An effect of this would be that the function which initializes such struct instances -- which takes a slice as input -- would need to either just copy from the slice, or use to_vec to clone its content, depending on whether the struct data type is a fixed-size array or vec. And yes, the struct needs to own its data, not just hold a slice itself.)

So the question here is: how would I go about implementing such a generic version of my struct? The Rust book generic chapter isn't really helpful in this respect, and nor is Rust By Example.

I assume I would have to specify traits for the array-like behaviour, but which? And I really have no idea how I might go about setting other field types according to the array/Vec element type.

Can anyone advise?

Thanks in advance for any help, and best wishes,

    -- Joe

You can use the trait AsRef<[f64]> which includes both vectors and arrays (of length 32 or less).

@alice I was going to suggest the same thing (except that AsRef and AsMut or you would not be able to write to the array), but rust is for some reason not happy with the following code after uncommenting s2.0.as_mut()[0] = 1u8: playground:

use std::convert::AsRef;
use std::convert::AsMut;
use std::marker::PhantomData;

#[derive(Debug)]
struct Foo<E, T: AsRef<[E]> + AsMut<[E]>>(T, PhantomData<E>);

impl<E, T: AsRef<[E]> + AsMut<[E]>> Foo<E, T> {
    fn new(data: T) -> Self {
        Self(data, PhantomData)
    }
}

fn main() {
    let mut s = Foo::new([0u8; 32]);
    let mut s2 = Foo::new(vec![0u8; 256]);
    s.0.as_mut()[0] = 1;
    //s2.0.as_mut()[0] = 1u8;
    eprintln!("{:?}\n{:?}", s, s2);
}

(also though it is not a problem with @joseph.wakeling's MyStruct as it has some_float I would really like to get rid of PhantomData).

The T that the error message is talking about is the one in AsMut<T>, and this is because Vec<u8> implements both AsMut<[u8]> and AsMut<Vec<u8>>, so it can't figure out which to use. The type parameters on Foo are irrelevant, as you're interacting directly with the type Vec<u8> through the field s2.0, not with the type Foo.

Hi folks -- thanks very much for the advice and suggestions.

One problem with AsRef is that is seems to block me using the len() method. I take it I'm missing another trait here?

I'm not really sure I follow the reason for PhantomData being used in the second example above. I appreciate the example of showing how to include the element type, though.

Is it really necessary to use as_mut() everywhere I want to access array elements, rather than just rely on the mutability of the struct instance?

Thanks again and best wishes,

   -- Joe

Specifically, the behaviours I need are:

  • get the length
  • look up and mutate elements by index
  • iterate over the array/vec

Another option is to introduce a trait like this:

trait ArrayLike {
    type Item;
    
    fn len(&self) -> usize;
    fn get(&self, i: usize) -> &Self::Item;
    fn get_mut(&mut self, i: usize) -> &mut Self::Item;
    // possibly more functions for iterators
}

impl<T> ArrayLike for Vec<T> {
    type Item = T;
    
    fn len(&self) -> usize {
        self.len()
    }
    fn get(&self, i: usize) -> &T {
        &self[i]
    }
    fn get_mut(&mut self, i: usize) -> &mut T {
        &mut self[i]
    }
}

playground with iterators

The reason PhantomData is needed in Foo is that there isn't any field in the Foo type that uses the type E. The trait above using an associated type instead of a generic type avoids this issue.

3 Likes

Ah, nice! Thanks for the very detailed explanation and examples, I really appreciate it.

I'm slightly surprised that such an ArrayLike trait doesn't already exist TBH, as I would have expected this to be a reasonably common requirement. I suppose that for 99% of use cases, where ownership of the data isn't an issue, just supporting &[T] does the job.

I haven't explicitly tried this on my problem/program yet, but I will do so, and report back.

You still need a separate impl for every length you wish to use, but you can use a macro for this. See playground. You can also add a wrapper type around your generic array that allows use of the indexing operators: playground.

If you have any fields in your struct that should have the item type, you can use this syntax:

struct MyStruct<A: ArrayLike> {
    vec_or_arr: A,
    some_float: A::Item,
    other_float: A::Item,
}

One problem with AsRef is that is seems to block me using the len() method. I take it I'm missing another trait here?

I do not see how it does. The below code works just fine:

use std::convert::AsRef;
use std::convert::AsMut;
use std::marker::PhantomData;

#[derive(Debug)]
struct Foo<E, T: AsRef<[E]> + AsMut<[E]>>(T, PhantomData<E>);

impl<E, T: AsRef<[E]> + AsMut<[E]>> Foo<E, T> {
    fn new(data: T) -> Self {
        Self(data, PhantomData)
    }
    
    fn len(&self) -> usize {
        self.0.as_ref().len()
    }
}

fn main() {
    let mut s = Foo::new([0u8; 32]);
    let mut s2 = Foo::new(vec![0u8; 256]);
    s.0.as_mut()[0] = 1;
    //s2.0.as_mut()[0] = 1u8;
    eprintln!("{:?} {}\n{:?}", s, s.len(), s2);
}

I'm not really sure I follow the reason for PhantomData being used in the second example above. I appreciate the example of showing how to include the element type, though.

It is there only because I chose not to have some_float/etc: can’t have type parameter which is not used somewhere in the struct. You would not need it, just make some_float have type E.

Is it really necessary to use as_mut() everywhere I want to access array elements, rather than just rely on the mutability of the struct instance?

as_mut() or as_ref() are necessary because in my variant you are basically saying via traits that “structure field is coercible to the reference to the slice when prompted”. I do not see how you can do better using only built-in traits, but if you do not want to implement ArrayLike there is only one other option to make access more convenient: Deref and DerefMut: playground

use std::ops::Deref;
use std::ops::DerefMut;
use std::convert::AsRef;
use std::convert::AsMut;
use std::marker::PhantomData;

#[derive(Debug, Clone, Eq, PartialEq)]
#[repr(transparent)]
struct ArrayWrapper<E, T: AsRef<[E]> + AsMut<[E]>>(T, PhantomData<E>);

#[derive(Debug, Clone, Eq, PartialEq)]
struct MyStruct<E, T: AsRef<[E]> + AsMut<[E]>> {
    va: ArrayWrapper<E, T>,
    some_float: E,
    other_float: E,
}

impl<E, T: AsRef<[E]> + AsMut<[E]>> ArrayWrapper<E, T> {
    fn new(data: T) -> Self {
        Self(data, PhantomData)
    }
}


impl<E, T: AsRef<[E]> + AsMut<[E]>> MyStruct<E, T> {
    fn new(data: T) -> Self where E: Default {
        Self {
            va: ArrayWrapper::new(data),
            some_float: Default::default(),
            other_float: Default::default(),
        }
    }
}

impl<E, T: AsRef<[E]> + AsMut<[E]>> Deref for ArrayWrapper<E, T> {
    type Target = [E];
    
    fn deref(&self) -> &Self::Target {
        self.0.as_ref()
    }
}

impl<E, T: AsRef<[E]> + AsMut<[E]>> DerefMut for ArrayWrapper<E, T> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        self.0.as_mut()
    }
}

fn main() {
    let mut s = MyStruct::new([0u8; 32]);
    let mut s2 = MyStruct::new(vec![0u8; 256]);
    s.va[0] = 1;
    s2.va[0] = 5;
    eprintln!("{:?} {} {} {:?}", s.va, s.va.len(), s.va[0], &s.va[0..5]);
    eprintln!("{:?} {} {} {:?}", s2.va, s2.va.len(), s2.va[0], &s.va[0..5]);
}

Here you get to use .len() and [] directly for va field and do not need to implement your trait for all lengths you want to use (assuming you do not want to use fixed-size arrays with lengths greater than 32). You also do get to use slices like s.va[0..5], implementing all that with solution based on custom trait would be even more boilerplate and still require wrapper type like my ArrayWrapper.

No, there's no trait for len(). But you can get the slice then get the length of that :slight_smile:

I had a similar problem when designing the gcode crate.

This crate is primarily intended for embedded environments so it needs to work without an allocator, but at the same time a Vec can make your code more flexible (you don't have a maximum number of arguments/commands) and could potentially lead to less memory usage (each item uses the memory it needs, not the maximum expected amount).

The solution I came up with was to have a Buffer trait that gets implemented for both Vec<T> and ArrayVec<[T; N]> from the arrayvec crate.

pub trait Buffer<T> {
    fn try_push(&mut self, item: T) -> Result<(), CapacityError<T>>;
    fn as_slice(&self) -> &[T];
}

You may want to have a look at the gcode::buffers module for inspiration.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.