I'm new to Rust and I'm trying to build a fast and low-memory multi-threaded CLI application, but I was shocked to see the memory consumption afterwards. I figured out the code to blame, but it doesn't make any sense to me: Thread::spawn()
use std::thread;
use std::time::Duration;
fn main() {
// Every thread adds: ~68 MB
thread::spawn(|| {
thread::sleep(Duration::from_secs(10));
});
// Baseline: ~13 MB
loop {
thread::sleep(Duration::from_secs(10));
}
}
However I'm unable to find any reports about this on the internet, so I'm asking myself whether or not this is "expected" behavior, although I can't image.
A ps will show the following ~80 MB:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
raymond 16417 0.0 0.0 80772 1092 pts/3 S+ 08:32 0:00 target/release/threads
Is there something that I'm doing wrong, do I miss some vital information to understand why this happens, or do I misinterpreted the memory numbers?
64MB per thread is a lot, AFAIK creating a thread only reserves the virtual memory for the stack. And the default thread stack size tends to be something from 2MB to 8MB, depending on the architecture and distribution (ulimit -s).
That said, I do see the same behavior with your example (amd64 vm);
no threads
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
user 3137 1.0 0.0 13168 1044 pts/0 S+ 07:03 0:00 target/release/
I now understand that I have to look at RSS for the actual memory that is consumed by my CLI application (not the snippet above), which means ~2.3 MB. That is a total relief and totally acceptable!
Yes, it's not a problem on 64 bit where the address space is pretty much unexhaustible.
Edit: for completeness I investigated where the virtual memory usage comes from. It has nothing to do with rust, nor rust's runtime, but with glibc's allocator creating per-thread arenas. I tried setting export MALLOC_ARENA_MAX=1 and with 16 threads: