Borrow to manipulate part of an array


#1

Hello everybody!

I want to test Rust on an emulator.

Here is a typical C use case I want to know how would be achieved using Rust:

I have a global fixed size array. This is supposed to represent harware memory.

A command tell me “load a texture of 32x32 format BGRA at adress 5605”.

In C I would do something like this (roughly):

void* ptr = &hwardware_memory[5605];

short* texture = malloc(32x32x4*sizeof(short)) // 4 because R,G,B,A

for (i in 32x32) {
    short* offset = ((short)ptr) + i*4;
    // notice reorder
    short b = offset;
    short g = offset+1;
    short r = offset+2;
    short a = offset+3;

    texture[i*4] = r;
    texture[i*4+1] = g;
    texture[i*4+2] = b;
    texture[i*4+3] = a;
}

This is actually even more complicate (because hardware_memory infos are not classical C type, more something like “14bits for color + 2bits for the palette info”) and sometime you have to write infos back into the emulated hardware memory, but you get the idea.

Such “reordering” in “quite weird” organized array is quite common in emulator code. Pointers really help in such situation.

What would be the good way to handle thing like this in Rust while keeping high performance? Can I borrow “part of an array”? For example:

array ptr[32x32x4] = &hwardware_memory[5605:5605+32x32x4]; // this would mean: "I borrow a part of the array".

Thanks in advance!


#2

This is completely possible in rust, using the &array[start..end] notation. Here’s an example which (I think) is equivalent to your C code.

(playground: https://play.rust-lang.org/?gist=d94296bc7192dfb7b620&version=stable)

use std::mem::size_of;

fn main() {
    let hardware_memory = [0u8; 65536];
    
    // Texture is now an allocated vector of the size of a texture, containing memory starting from 5605.
    let texture = load_texture(&hardware_memory, 5605);
}

fn load_texture(hardware_memory: &[u8], address: usize) -> Vec<u8> {
    
    // texture size constant
    let texture_size = 32 * 32 * 4 * size_of::<u8>();
    
    // this is now a "partial" borrow of hardware_memory, otherwise known as a "slice".
    let memory_part = &hardware_memory[address..address + texture_size];
    
    // this allocates memory of a size equal to TEXTURE_SIZE
    let mut texture = Vec::with_capacity(texture_size);
    
    for i in 0..(32 * 32) {
        let b = memory_part[i * 4];
        let g = memory_part[i * 4 + 1];
        let r = memory_part[i * 4 + 2];
        let a = memory_part[i * 4 + 3];
        
        // instead of setting the positions, we'll just push onto the end of the vector.
        // since we've already allocated exactly enough memory, this is exactly equivalent
        // to your code.
        texture.push(r);
        texture.push(g);
        texture.push(b);
        texture.push(a);
    }
    
    // return the texture
    texture
}

#3

Wow! Thanks a lot! Most of my needs are is this code snippet!

About performance, as RGBA components are quite tiny. Is it not more efficient to package them first in one QWORD (8x4=64) then assign this QWOD to the place offsetted in the “texture” array.

I’m afraid of STL containers types, even in C++ I try to avoid them on performance critical hot spot because you don’t always get efficient asm code compared to “classical array”.

What would be the faster way in rust to push r,g,b,a to texture without relying on Vec? :smile:

Thanks again! :blush:


#4

Vec is pretty much the lowest level type for representing allocated memory in rust. The .push() method isn’t the most efficient way of using a Vec, but you will want to use a Vec regardless if you are allocating any kind of array in memory. The .set_len, .get_unchecked and .get_unchecked_mut are the most efficient ways of using a Vec, .get_unchecked and .get_unchecked_mut are pretty much just adding an offset to a pointer.

These are unsafe operations, because they potentially give you access to unallocated memory. However, if you’re sure you are using them correctly, you can use the unsafe operator to access them.

// this allocates memory of a size equal to TEXTURE_SIZE
let mut texture = Vec::with_capacity(texture_size);

unsafe {
    // you need to make sure that the argument to set_len is less than or equal to the argument to
    // with_capacity, or you will be accessing/modifying unallocated memory.
    texture.set_len(texture_size);
    
    for i in 0..(32 * 32) {
        // * operator to dereference &u8 to u8, this is done automatically with [x] syntax
        let b = *memory_part.get_unchecked(i * 4);
        let g = *memory_part.get_unchecked(i * 4 + 1);
        let r = *memory_part.get_unchecked(i * 4 + 2);
        let a = *memory_part.get_unchecked(i * 4 + 3);
        
        // *x = y; syntax sets the value that x points to to y.
        *texture.get_unchecked_mut(i * 4) = r;
        *texture.get_unchecked_mut(i * 4 + 1) = g;
        *texture.get_unchecked_mut(i * 4 + 2) = b;
        *texture.get_unchecked_mut(i * 4 + 3) = a;
    }
}

The unchecked methods don’t do any bound checks, so you do need to be extra careful to not give them invalid numbers. I’m sure you are used to this though, working with C. The .set_len() call also gives possible access to unallocated memory, and the Vec won’t zero any memory with that call - so it will contain whatever was there before.

In general, it’s preferred to use the indexing operator [], and .push() for Vec unless you’ve identified a place as a hotspot - LLVM should do a relatively good job of optimizing, and using the unsafe operators does open you up to memory unsafety. However, if you need it, it’s there.


#5

Thanks a lot! I will play with both way and see which one give best results! :smile:


#6

Sounds good! Make sure to compile with optimizations for any benchmarking, by the way! (cargo build --release / rustc -O)