Vectorized memcpy

code review
1

i wrote a vectorized memcpy impl yesterday. it's very close to as fast as std. any thoughts on how to improve performance further or whether this is a good approach?

2

your implementation is not sounds, as it tries to read uninitialized byes as a fully initialized type.

Adding MaybeUninit would solve it.

3

where? no complaints from miri.

4

no complaints from miri.

as far as i can see you only test your functions with types which have no padding and are zero-initialized. that would indeed avoid the problem.

try something like copying a


#[repr(C)]
struct WithInnerPadding(u8, u32); // 8 bytes big

also you must initialize the source with the contructor, not use mem::zeroed

5 
fn main() {
    let mut dst = [None, None];
    let mut src = [Some((0_u64, 0_u8)), Some((0_u64, 0_u8))];
    
    unsafe {
        cpy(dst.as_mut_ptr(), src.as_mut_ptr(), 2);
    }
}
6

ok i see it now. is there a solution to this/is it actually a problem? glibc memcpy also copies uninit bytes. even if it's garbage data, it's not as though the garbage is actually read later. it runs fine outside of miri

7

use

ptr::write(x.cast::<MaybeUninit<$T>>(), ptr::read_unaligned(y.cast::<MaybeUninit<$T>>()));

instead of

ptr::write(x as *mut $T, ptr::read_unaligned(y as *mut $T));
8

uhh i mean ok. does that actually change how this works at all or is this just satisfying miri? more importantly, does this result in more instructions at all?

9

not really no. what may happen is some use case where your previous one was UB the compiler just removed the call because it saw the UB and now it won't ig.

it might changes a couple hints that are sent to llvm about what is initialized or not

10

added that and it appears to be just as fast (running even a bit faster? strange). thanks. any other notes?

11

How does this compare to a naive loop or autovectorized loop?

12

it's very close to as fast as glibc according to my benchmarks

13

glibc is already optimized, I meant like a single for loop

14

This seems like an XY problem to me. What are you trying to do?

Note that LLVM will regularly detect things that look like memcpy and replace them with a call to the platform memcpy, so this is often not a fruitful thing to do.

15

what are you trying to do? what is anyone trying to do? why do anything?

16

my impl is about 360us for 4000000 bytes on my machine. for the same number of bytes, copying byte by byte, i benchmark 2.2ms

17

You haven't unlock AVX512. You may want to add support for that. I tried adjusting your width to 64 and 128, still it generates AVX2

#[repr(simd)]
#[derive(Copy, Clone)]
pub(crate) struct zmm_t([u8; 64]);

if likely(Z >= 32) && !x.is_aligned_to(64) {
    W!(zmm_t);
}

while Z >= 128 {
    W!(zmm_t);
    W!(zmm_t);
}

if Z >= 64 {
    W!(zmm_t);
}

Why do you use 2 AVX2 instead of 1 AVX512 if supported?

I tried that, LLVM still generates AVX2 :< Is it because it knows AVX512 would cause downclocking for this CPU (my CPU)?

The benefit of yours (assume it has equal assembly, is no function call cost). I haven't found a way to see the assembly of the memcpy implementation because it is external symbol, it just show up as call to memcpy :<

The benefit of memcpy would be, it is platform independent. For example, the app is compiled in SSE2 CPU, then when it is shared and run in AVX2 or AVX512 CPU, it can still laverage it because it is dynamic SIMD. Yours is static auto SIMD, once it is compiled to SSE2 CPU, it can't automatically upgrade to wider SIMD implementation later when run on CPU that support wider without recompiling it. But it is not issue if you just run locally

Also there is slight error in your benchmark. The equivalent of memcpy is std::ptr::copy_nonoverlapping, not std::ptr::copy. Memcpy will UB if overlap, so the equivalent is nonoverlapping variant. Because the ptr::copy handles overlapping so it adds additional work that is not done by non overlapping ones. Your code also does not handle overlap, it uses 2 &mut params, so it prevent overlapping memory being passed, so the equivalent should be std::ptr::non_overlapping

18

Oh, so the autovectorizer doesn't work very well? What is the code for the 2.2ms benchmark?

19 
    unsafe {
        let x: *mut u64 = xxx::new(huge).unwrap();
        let y = iota(huge);
        c.bench_function("naive loop", |b| {
            b.iter(|| {
                for i in 0..huge * 8 {
                    *x.cast::<u8>().add(i) = *y.cast::<u8>().add(i);
                }
            })
        });
    }
20

That doesn't get autovectorized? Interesting, I would have thought it would.