How many symbols can Rust Analyzer autocomplete in practice?

Editors and IDEs
1

I'm trying to autocomplete from a bindgen module that has about 32k pub items, the esp idf system library.

When try autocompleting from the namespace in Emacs (typing esp_idf_sys:: and pressing Tab), it takes about 10 seconds for symbols to appear in a popup. The popup can be filtered quite responsively, which suggests that the slowness is either from rust-analyzer itself or from the communications parsing overhead.

I have profiler details and the bindings.rs file in a gist if any Emacs / lsp-mode experts would like to take a closer look.

More broadly, I'm curious about everyone's experiences with their favorite editors. Is it just (my) Emacs setup that's slow? Or is a 5 MB source file containing 32k items enough to bring most editors and analysis tools to their knees? (I'm on an M1 Mac with 16 GB of RAM)

Of course, if anyone has a setup that lets them autocomplete in 100ms from such files, I'd love to hear about it! =D

2

Generally IDEs have a limit on the size of analyzed files. 5MB is well over that limit, so I wouldn't be surprised if RA excluded it. There should be some way to raise the limit, at least globally, but I can't point you to the specific setting.

Remember that the IDE needs to keep the entire syntax tree in memory to perform its analyses. A large file with moderately nested moderately large functions can take a really huge amount of memory. This is less of an issue if the file just defines the extern functions.

32k items on its own isn't an issue, particularly if they have very different names.

3

I threw together a benchmark using the lsp client protocol directly to exercise rust analyzer independently from Emacs. It takes about 3s to return the completion list: GitHub - lynaghk/rust-analyzer-perf-test: playing with the language server protocol in Rust.

That's definitely faster than RA+Emacs, but does suggest that the performance of RA on my hardware is still going to be too slow for fluid interactive completion from this namespace.

Since I'm already on fairly fast hardware (M1 Mac with 16 GB RAM), is there anything else I might do to speed up RA itself?

1 Like
4

@lynaghk here's the benchmarked based on rust-analyzer's own infrastructure:

One important thing to note here is that rust-analyzer doesn't do any on-disk caching, so, what your benchmark essetially measures is not the completion time per-se, but rather the time for initial analysis of the project.

The more realistic way to do this would be:

  • issue semantic-tockens request to populate caches
  • modify the code to bust the immediate caches
  • run the completion

And that's roughly what my linked modification of your benchmark does. The results I see are:

workspace loading: 6.63s
initial: 2.16s
change: 10.71µs
566ms - unqualified path completion
n completions: 22391

That is,

  • 6 seconds to run cargo metadata and stuff
  • 2 seconds for syntax highlighting (which does most of the analysis)
  • 10 microseconds to apply the change & bust the caches
  • 560 milliseconds to compute the actual completion

These numbers look reasonable to me. And, if what you see in Emacs is 10 seconds, I would chalk that up to client not keeping up with the server. I think the right fix here is to limit the number of items we return as a result of a single completion request to something like 500. The catch here is that, at the moment, it's the client who determines which completions are the most relevant, so we need to move that logic to the server first (which we want to do anyway for unrelated reasons as well): Implement server-side completion sorting and filtering · Issue #7935 · rust-lang/rust-analyzer · GitHub.

Finally, it's also useful to get at rust-analyzer with profiler and just make the stuff faster. Our profiling infra is documented a bit in rust-analyzer/docs/dev at master · rust-lang/rust-analyzer · GitHub

4 Likes
5

I would chalk that up to client not keeping up with the server.

A quick way to check this hypothesis would be to add .into_iter().take(512) here, rebuild ra from source, and see whether that helps:

6

There's actually a very relevant open PR: Add upper limit for the number of completion items by tech-ticks · Pull Request #13124 · rust-lang/rust-analyzer · GitHub

2 Likes
7

Apologies for the long delay, I fell quite ill a few hours after posting. Luckily, I'm sufficiently recovered now to pick up my Emacs yak shaving =D

Thank you for the comprehensive reply (and all of your hard work on RA)! I had a go at limiting the completion results returned by RA as per your suggestion and things are indeed much snappier. So it does seem like Emacs is the bottleneck here, not RA.

The open PR mentioned by @fdiebold does sound like it'll go a long way to improving the situation, thanks for bringing that to my attention.

2 Likes
8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.