How much memory does Vec<u8> occupy on amd-64 architecture?

I've made a simple lossy compressor of geo::LineString in a way similar to Google Polyline algorithm.

Code that compacts the data is here, example output:

original value: [(123333.5, 456534.8), (125333.5, 451534.8), (125243.5, 451464.8), (125212.2, 451476.4), (125812.2, 451472.4)],
80 bytes
compacted data: (123333.5, 456534.8) [128, 181, 24, 193, 132, 61, 209, 140, 1, 177, 109, 245, 48, 144, 18, 192, 169, 7, 161, 6]
16 bytes + 20 = 36 bytes

The result struct is this:

struct CompactLinestring {
    first: Coord, // like (f64, f64)
    deltas: Vec<u8>
}

I wonder how much data does this struct actually occupy in memory? What if each u8 is stored in 32-bit slot? (My architecture is AMD64.)

On disk, it did allow to compact 35Mb struct into 24Mb.

The data in a Vec<u8> is stored contiguously (no additional padding between the u8s) for all architectures.

2 Likes

When you use Vec, if you don't know how much space to reserve in advance, the implementation has to use a heuristic to decide how much additional space to reserve when you push an element onto a full vector and in general unless you are lucky or manually truncate it, it will occupy more space than the final size. So you can end up using more memory than you require (though how your OS actually allocates this extra space adds an additional layer of complexity)

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.