Is there `<[[T; N]]>::flatten`?

help
1

Is there any function that turns &[[T; N]] into &[T]? (maybe flatten, maybe from_chunks)

I thought I had seen it added a while back, but I cannot find it. Did I dream it up, or am I just not seeing it it?

2 Likes
2

No, it doesn't currently exist. I definitely think it should, as the inverse method to https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.as_chunks.

Last time I tried to do something like it (Add Vec::flatten (as unstable, of course) by scottmcm · Pull Request #61680 · rust-lang/rust · GitHub) there was a bunch of uncertainty about whether it needs to also handle things like &[Vec3f] -> &[f32].

4 Likes
3

This definitely should exist, indeed. In the meantime, align_to() solves yet another problem:

fn flatten<T, const N: usize>(arrays: &[[T; N]]) -> &[T] {
    let (head, middle, tail) = unsafe { arrays.align_to::<T>() };
    assert!(head.is_empty() && tail.is_empty());
    middle
}

(Note: this is only guaranteed to work so far as align_to() works as one would expect in this situation, but it's unfortunately allowed to simply not do anything at all.)

2 Likes
4

Note that align_to won't work for ZSTs, though I suspect your real case isn't using arrays of ZSTs.

(ZSTs are a weird case here, since [[(); usize::MAX]; usize::MAX].flatten() needs to fail because of the length overflow.)

@Cyborus if you're willing to copy-paste a bit of unsafe code, you might as well just use the simple from_raw_parts version:

pub fn flatten<T, const N: usize>(a: &[[T; N]]) -> &[T] {
    let new_len = a.len().checked_mul(N).expect("length overflow");
    unsafe {
        std::slice::from_raw_parts(a.as_ptr().cast(), new_len)
    }
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=94c043e4e28a99654938815af5dc00d5

1 Like
5

Yes, probably ZSTs would need to be special-cased there (hopefully everything size_of-related would be optimized away).

6

Haha, I was just about to share how I would implement it, which was exactly this
Maybe a corresponding try_flatten would be good, to avoid panics

7

The bytemuck crate allows this type of conversion without any unsafe (bytemuck::cast_slice). It also allows between-type conversions that are nonsensical, but a wrapper function with a more specific signature will fix that.

1 Like
8

That only works for Copy types, though, because Pod: Copy, but the operation in general makes sense for all (Sized and non-ZST) types.

By the way, I find the error type PodCastError to be hilarious. I'm guessing it has to do with the next episode not coming out in time?

4 Likes
9

Luckily this isn't directly an issue for my current use case, as I'm working with u8s

You are not alone in this, I chuckle every time I see it

10

This goes in the category of "I wouldn't even bother", to me. The only way that can actually overflow is for ZSTs, and only in situations where people are being silly.

For an implementation in core I'd probably add more complexity to make that obvious, perhaps something like this

pub fn flatten<T, const N: usize>(a: &[[T; N]]) -> &[T] {
    let new_len = 
        if std::mem::size_of::<T>() == 0 {
            a.len().checked_mul(N).expect("length overflow")
        } else {
            // SAFETY: the slice exists in the address space,
            // so this multiplication cannot overflow
            unsafe { std::intrinsics::unchecked_mul(a.len(), N) }
        };
    // SAFETY: For ZSTs any length is fine; for non-ZSTs the object
    // already exists so cannot be more than `isize::MAX bytes`.
    unsafe {
        std::slice::from_raw_parts(a.as_ptr().cast(), new_len)
    }
}

Since, as H2CO3 mentioned, the size_of checks are trivially optimized away even at only opt-level=1.

2 Likes
11

Ah, yeah, that makes sense.

12

To propose adding this, should I simply post on IRLO? Or something else?

13

If you'd like team input before starting, the best places is the libs zulip: https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs

In general, for a single-method thing that's not setting a broad precedent, you can just send a PR.

(This one the PR will probably be harder than you like because of silly reasons around methods on built-in types like slices.)

1 Like
14

I'm just going to take this opportunity to advertise my slice-of-array crate again, which more or less exists solely to provide this function:

I suppose I should update it to consider the rare case of multiplication overflow on ZST types.

Edit: The crate has been updated.

4 Likes
15

One potential reason not to add this method is that it would become obsolete when project-safe-transmute will be implemented

16

I think there's a couple of ways to look at this, some overlapping and some exclusive:

  1. slice repr is not currently guaranteed, so even safe transmute wouldn't allow direct safe-transmute on slices anyway

  2. this could be extended later with better support for different non-array types in the receiver slice, as part of safe-transmute

  3. even if safe-transmute can do this, it's convenient to have a canonical no-type-annotations needed way to do the simple-and-useful version anyway, and it can be implemented via safe-transmute when ready. This is similar to how .as_ne_bytes() on integers is likely still valuable even when safe transmute will definitely allow that to be spelled something like safe_transmute::<[u8; 4]>. A call to .flatten() emphasizes that it's not doing things like sign changes, just the flatten.

6 Likes
17

Transmuting directly between references (and pointers) is basically never correct. Transmutation is not transitive ("T -> U is safe" does not imply &T -> &U is safe), and people confuse the fact that a simple primitive pointer cast can result in effective transmutation of the pointed values with the act of transmuting between pointers per se.

Transmuting between pointers also poses enormous future-compat problems in terms of strict provenance tracking. See eg. https://github.com/rust-lang/rust/pull/95241.

18

You're right saying that a direct transmute won't be valid, but this doesn't rule out a flatten-like method that's more generic than what's being proposed here. See https://github.com/rust-lang/project-safe-transmute/blob/master/rfcs/0000-ext-container-casting.md (literally the first example).

That's why I said it would become obsolete.

Note that .as_ne_bytes() was removed because safe-transmute would be a more generic solution Remove num_as_ne_bytes feature by hch12907 · Pull Request #85679 · rust-lang/rust · GitHub

19

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.