Writing Vec and ndarrays as bytes

I need to write (to files) some Rust Vecs (and probably some ndarrays too) in their raw binary representation, because a Python client expects to be able to read them in using code roughly similar to this:

data_raw = open(fpath,'rb').read()
data_array = struct.unpack_from('f' * ndata, data_raw)

Before re-inventing the wheel, I'd like to check whether this is a solved problem in the Rust ecosystem.

[If I do have to write this myself, do you have any hints?]

You probably want to iterate over the vec and use f32::to_be_bytes() or f32::to_le_bytes() to get the byte sequence for each number. This will ensure the correct thing is written regardless of the architecture’s endianness.

There’s also the byteorder crate which might have helpers for this.

Yes, that's what I had in mind. I've put a prototype in this playground.

  • Are there any downsides to this approach?

  • Can it be made more generic? (It's f32-specific as it stands.)

  • Is there a more direct way to match on the io error kind?

Edit: Here's a sligtly more general version which uses iterators rather than hard-wiring Vecs.

Not really. If anything it's actually more reliable and safe because you are serializing using a known endianness and not using raw pointers.

You may incur the cost of a byteswap if, for example, your computer is little-endian and your Python script expects big endian. I don't think floats have an endianness, though.

You are probably looking for a crate like zerocopy or scroll.

It depends on how you get the error and what you want to do next, but what about something like this?

use std::io::{Error, ErrorKind};

fn fallible_operation() -> Result<(), Error> {
    todo!()
}

fn main() {
    let result = fallible_operation();
    
    if let Err(e) = result {
        match e.kind() {
            ErrorKind::NotFound => todo!(),
            _ => panic!(),
        }
    }
    
    match result {
        Ok(value) => panic!(),
        Err(ref e) if e.kind() == ErrorKind::NotFound => todo!(),
        Err(other_errors) => panic!(),
    }
}

I'm sorry, I wasn't clear enough: I was merely frustrated by not being able to match directly on the kind inside the error, which forced me to nest matches. You reminded me of the existence of if in matches so now I can write

match file.read_exact(&mut buffer) {
    Ok(()) => Some(Ok(f32::from_be_bytes(buffer))),
    Err(e) if e.kind() == UnexpectedEof => None,
    Err(e) => return Some(Err(e)),
}

which is good enough.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.