Perf issues, 50% time spent in strerror_l

Ive been looking at the performance of a large-ish rust project at $company and it seems almost 50% of running time is spent in strerror_l calls. Confusingly, the call graph generated by google's pprof seems to indicate that a number of seemingly unrelated functions are spending a decent amount of time there.

Im wondering if anyone has encountered this issue or knows what actually calls strerror_l under the hood.

Ive checked out the strerror linux manual page, but it didnt give me any clues as to what might be calling it.

It's likely translating errno to a (localized) error message. What's your application doing? Is it possible it's generating lots of errors in libc calls? 50% sounds excessive - where does the application bottom out, so to speak? Is there a set of platform calls it makes?

At 50% I'm a bit suspicious that there's an issue with symbol resolution and a bunch of instruction addresses in unrelated functions are being resolved to sterror_l. You could try building with debuginfo to get hopefully more accurate locations.

I would think making better sense of the call graph would be approach to determining what to do after. Inlining can make them confusing.

Not tried pprof. Staring at multiple tools sometimes give a better perspective.

1 Like

Hmm, I am using the flate2 crate to read a gzip file from disk - pretty sure that's the only major source of libc calls. Ill try removing that and running again.