Better ergonomics for array of const size A+B

help
1

I am writing some code to parse packets, and since this is for an embedded target, I am not using alloc. When I first wrote this, I just used an assert that made sure the hardcoded buffer size was big enough for the HEADER_SIZE+PAYLOAD_SIZE. I thought about using a separate PAYLOAD_SIZE, and PACKET_SIZE, but I didn't like this because this forces the user to know what valid pairs of sizes are.

To solve this what I initially tried to do was use

let mut buf = [0u8; HEADER_SIZE + PAYLOAD_SIZE];

Seemed simple, but after some research it makes sense why this isn't directly possible.

I tried using #![feature(generic_const_exprs)] and specifying the appropriate where [(); HEADER_SIZE + PAYLOAD_SIZE]:, but ran into internal compiler errors. Since these features don't seem stable, I abandoned this attempt.

I thought it might actually be better to use a struct as a const param like so

#[derive(ConstParamTy, PartialEq, Eq)]
pub struct ConstParams {
    packet_size: usize,
    payload_size: usize,
}
impl ConstParams {
    pub const fn new(
        payload_size: usize
    ) -> ConstParams {
        Self {
            packet_size: payload_size+HEADER_SIZE,
            payload_size,
        }
    }
}

fn example_use<const PARAMS: ConstParams>() {
     let mut buf = [0u8; PARAMS.packet_size];
}

I like this approach because it is clean, and the user only gets the ConstParams::new() interface, so they aren't exposed to any implementation details of how I need these defined.

The problem with this approach... this isn't supported at all.

Is this sort of approach going to be possible in the future? It looks like there is good effort going into the const features.

Are there other ways to accomplish this? For now I am just going to use a separate PAYLOAD_SIZE, and PACKET_SIZE :frowning:

2

Have you considered a macro?

const HEADER_SIZE: usize = 8; 

macro_rules! create_packet_buf {
    ($payload_size:expr) => {{
        //$payload_size is expanded to a literal at compile time.
        const TOTAL_SIZE: usize = HEADER_SIZE + $payload_size;
        [0u8; TOTAL_SIZE]
    }};
}

fn main() {
    // The user only provides the payload size
    let mut buf = create_packet_buf!(32);
    
    assert_eq!(buf.len(), 40); 
}
3

I don't see a macro in the playground you linked. Maybe you didn't click the Share button?

4

This does not work when using const generics like I need.

Fails with error[E0401]: can't use generic parameters from outer item

5

well, depending what "this" exactly means, there are different "solutions", but all would be hacky (and sometimes even "ugly") in some way.

the least hacky one is to use a struct for the buffer, but only exposes a slice representation in the public api. e.g. with bytemuck:

/// note the `#[repr(C, packed)` attribute
/// `packed` is not necessary, but I put it here as a sort of documentation
#[derive(Clone, Copy, bytemuck::Pod, bytemuck::Zeroable)]
#[repr(C, packed)]
struct PacketBuffer<const PAYLOAD_SIZE: usize> {
    header: [u8; HEADER_SIZE],
    payload: [u8; PAYLOAD_SIZE],
}
/// example to use the standard `Deref(Mut)` to expose an API of a byte slice
impl<const PAYLOAD_SIZE: usize> Deref for PacketBuffer<PAYLOAD_SIZE> {
    type Target = [u8];
    fn deref(&self) -> &[u8] {
        bytemuck::bytes_of(self)
    }
}
impl<const PAYLOAD_SIZE: usize> DerefMut for PacketBuffer<PAYLOAD_SIZE> {
    fn deref_mut(&mut self) -> &mut [u8] {
        bytemuck::bytes_of_mut(self)
    }
}

fn test<const GENERIC_CONST: usize>() {
    // The user only provides the payload size
    let buf = <PacketBuffer<GENERIC_CONST> as bytemuck::Zeroable>::zeroed();
    // get a byte slice of the buffer
    let buf = &*buf;
    assert_eq!(buf.len(), 40);
}

this works around the limitation that you cannot create an array in generic context directly. but the limitation of generic_const_exprs still applies: you cannot use the "total packet size" (computable as std::mem::size_of::<PacketBuffer<PAYLOAD_SIZE>>()) for other generic constants.

if you absolutely want the packet size at compile time, then generic constant alone will not do the work, you'll have to use other tricks, such as type-level bit encoding of integers, similar to the typenum crate. you'll get compile time computation like addition and multiplication, but the trade-off is you'll have to rewrite your code to use type level integers instead, which is even more limiting than const generics in some aspects. it's possible to make an array out of it, but due to the same reason your initial try failed, it's mostly useless because you cannot reify it at runtime (array of values, instead of types).

oh, and another option: just give up generics and use code generation instead. depending on the situation, you might use build scripts or macros. since you are not working within the rust type system, you can even create a DSL to describe the data format, think protobuf.

6

You can write an equivalent of your very first snippet using typenum and generic-array:

use core::ops::Add;
use generic_array::{ArrayLength, GenericArray, typenum::Sum};

fn handle_packet<HeaderSize, PayloadSize>()
where
    HeaderSize: ArrayLength + Add<PayloadSize>,
    PayloadSize: ArrayLength,
    Sum<HeaderSize, PayloadSize>: ArrayLength,
{
    let mut buf = GenericArray::<u8, Sum<HeaderSize, PayloadSize>>::default();
    // handle packet
}

It's not pretty... But it's probably the best what we can do using existing stable Rust.

As a potential alternative you also could try something like this:

fn example_use(params: PacketParams) {
     let mut buf = [0u8; MAX_PACKET_SIZE]
     let mut buf = &mut buf[..params.packet_size];
}

It will "waste" a bit of stack space and it will be harder to pass buf around (you could use arrayvec if you really need to do it), but it may work fine for your case. And the compiler may be even able to optimize this code to use only the necessary stack space.

7

It might be a good idea to report those, as having reproductions of cases where the feature isn't solid yet would be helpful in making the feature more robust, which in turn will help it to get stabilized (eventually).

8

My understanding is that this feature is not planned to be stabilized, rather its little cousin min_generic_const_args is.

9

Agreed. I would love these features to be improved, so I made sure to report.

10

The bytemuck approach would be perfect, but it appears bytemuck does not support const generic sized arrays. It only implements Pod for arrays of size 0-32 and 48, 64, 96, 128, 256, 512, 1024, 2048, 4096.

11

You need to enable the min_const_generics feature.