This is more related to my other question: Rust-analyzer as persistent program? than the current one, but since you operate on another level with neovim:
Do you:
(1) never :q in neovim or
(2) somehow avoid paying the cost of 'rust-analyzer boot time' ?
My current finger memory with vim is:
move around in bash
nvim blah.rs
...
:q
move around in bash
and basically this incurs a rust-analyzer startup time on every nvim session until things like goto-def start working