Production-grade Runtime Profiling of Rust applications

[anyon17]

I wanted to know what are some standard profiling tools for Rust application runtime data like thread stack, memory usage, async tasks, panics etc.
I tried Tokio console but it does not seem to be production ready and it's quite heavy to run.

[geebee22]

I don’t know if such tools exist ( it is hard to prove a negative ), but I don’t use them (other than standard OS tools maybe for checking memory/CPU usage ).

I tend to have features that enable logging of different types if I want to know what is going on in my program. For example logging how long some task took, the state of any caches, memory allocators, or whatever.

[jumpnbrownweasel]

@anyon17

thread stack, memory usage, async tasks, panics

I suggest looking for different tools for each of these, as I assume there is no one tool for all of them.

Memory profiling is being discussed here:

Where to look for memory leaks in Rust?

If you are able to test on Linux, Heaptrack and Bytehound are useful to investigate if you have allocations that aren't being freed (such as a cache or hash map that grows and never gets cleared). Even if you don't discover the specific issue you will most likely find other issues (needless allocations that could be optimised to reuse a buffer for example) and gain insight into how your program behaves. If you are on another OS, there are probably similar tools, but I don't know what they would…

[Vorpal]

I'm not sure what exactly you are after when you say "production grade", I'm getting the feeling this means something specific to you, which might be different than it does for me as someone who write embedded and hard realtime Linux software for industrial control.

But all the standard tools you would use for profiling C and C++ work on Rust (with the caveat that if Rust name mangling isn't supported the result is a bit annoying to read). I'm mostly familiar with Linux tooling, and can't really make any recommendations for Windows or MacOS X.

I like perf + hotspot for CPU profiling on Linux, and heaptrack or bytehound for allocation profiling (heaptrack is easier to use, and nowdays support rust demangling too, bytehound is more advanced and unmaintained but also complete and not buggy).

I would start with Profiling - The Rust Performance Book if you want to look for other options.

For async it is harder. I have experimented with using tracing spans combined with tools like tracing-tracy — profiling in Rust // Lib.rs, and also tokio-dtrace — async Rust library // Lib.rs (patched to work with Linux USDT, see the PRs in the repo).