Some libraries contain macros or traits or generics that the compiler must read, interpret and then use in its further processing. Obviously if you reference those libraries, or other libraries transitively reference those libraries, the compiler has to process them to reason about their contents. This is somewhat similar to C/C++
.h files, but the processing is a lot more complicated (e.g., proc macros used in generic trait definitions).
Other libraries, without any macros, traits or generics that you use, can be cached after initial compilation.
rustc takes more time than compilers for languages with less-strict compile-time checking. Much of what
rustc does is novel relative to mainstream compilers. Those parts of
rustc have not had the 100s of person-years of optimization that are found in typical C/C++ compilers.
The recommendation to use
cargo check and
cargo-watch is to do a quick screen for errors so that you can get about fixing them with little delay. When everything checks, then it's time to turn the big gun,
rustc, on the code and see what other errors it finds. You will find that you spend more up-front time in Rust than in other languages. Most developers find that is more than compensated by the reduced time debugging, and even more, by the lack of midnight calls when a problem develops in the field (e.g., at a customer site).