Find unnecessary Vec reallocations

#1

Hi all,

Is there a nice way to ensure that I allocate my vectors with sufficient capacity? I would basically like something which can help remind me to use

Vec::with_capacity(n);

where possible.

I know that I can inspect Vec::capacity before and after my operations to see if the vector was enlarged — however, this is a manual process which requires me to sprinkle temporary tracking code here and there.

Is there a more systematic way to do this? Perhaps some way of profiling a binary and detect calls to Vec::reserve (or the equivalent function which does the low-level reallocation)?

#2

There's the massif heap profiler tool which is part of the Valgrind software suite: https://valgrind.org/docs/manual/ms-manual.html
I haven't actually learned how to use massif yet, but it seems to be featureful. After profiling, there's both a CLI and a GUI tool to analyze the profile data.

There's also https://fasterthanli.me/articles/small-strings-in-rust which profiles memory by writing a tracing allocator that wraps the system allocator. It involves more changes to your code itself, and in part, was done for fun.

2 Likes
#3

Thank for the links and thanks for reminding me to look more into Valgrind! I read a bit more about it and found a nice guide about optimization of Rust programs.

I also found the Rust Performance Book, which is full of tips and tricks for optimization. Lots of stuff to study :slight_smile:

#4

I don't know of any automatic method. You could create your own newtype wrapper for Vec that logs its backtrace (or #[track_caller] info!) when constructing a Vec, and logs capacity just before the Vec is dropped. Then you could correlate these two.

1 Like
#5

Or if you're really serious about this, you could create a clone of Vec that never reallocates, and panics when it runs out of space.

#6

Thanks for the idea, that sounds quite doable!

I guess that would be equivalent to swapping my Vecs for ArrayVec via the arrayvec crate... I'll try and experiment a bit with that idea.

#7

Ah, no, I misunderstood how the crate works: an ArrayVec has a statically known size, whereas I was thinking I could specify the size dynamically.

#8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.