Heed help with &[u8] memory storage understanding

help
1

I have some struct with data member, that should implement reference to file bytes before send it to some socket after build:

pub struct Titan<'a> {
    pub data: &'a [u8], // <-
    pub size: usize,
    pub path: String,
    pub mime: Option<String>,
    pub token: Option<String>,
    pub options: Option<HashMap<String, String>>,
}

The size of this data would be 1 Gb or more, so Vec is not an option here, that because I have added the reference with lifetime there.

I hope this implementation will not copy all UTF-8 bytes from file into the memory, but not sure it will not copy 1 Gb of the references also, because as understand, references takes more space than u8 :slight_smile: or it's not references to every byte (unlike &[&u8]) and everything ok here?

p.s. before I've used Glib Bytes, but for now I won't have this c-lang dependency in the new crate.
p.p.s. also, thoughts to replace String with &str, but I want just to understand if the data implemented properly to continue.

2

This type has a single reference to a contiguous sequence of bytes (u8). See:
https://doc.rust-lang.org/book/ch04-03-slices.html

1 Like
3

Titan is relatively small, 144 bytes. But if the reference points to a gig of bytes, then

  • those bytes are in memory somewhere, or
  • they're pretending to be in memory via mmap or such

If your concern was not duplicating a gig of bytes for the sake of a temporary data structure, but you're okay with them all being in memory in a single place, you're probably on the right track. If you never want all of those bytes in memory at once, that's a different issue.

4 Likes