How to deal with stuck problem when compiling

Let me briefly describe a situation I encountered.

I'm working on a project with over 1000 crates. In my daily job, My machine often gets stuck for almost 10 minutes at the end of the compiling process!

The computer doesn't respond to anything, I can't even use the mouse! The clock also stops, like a nightmare.

Now I can only restart my computer in this way what to do when even the mouse is not working

My machine env:

  1. Ubuntu 20.04
  2. CPU: 16 AMD Ryzen 7 4800HS with Radeon Graphics
  3. Product Name: ROG Zephyrus G14 GA401IV_GA401IV

Can anyone help me!

I believe the --jobs flag will do what you're asking for. For example cargo build --jobs 2

However, it shouldn't be running so many tasks in parallel that I would expect that sort of behavior. Is it possible you're running low on memory and your system is swapping, or something like that?

1 Like

I usually use cargo build -j8. But stuck still happens sometimes.

In addition, I use RA(IDE), which eats up a lot of memory, and this might be a major cause of the stuck problems.

Does anyone have any tips on the proper usage of RA? Are there any other configurations that can be added to limit memory usage?

If you've got a project with over 1000 crates, expect to be using a significant amount of RAM (likely >1.5GB for the initial check). The only other thing I can think of is to limit the number of features enabled if you're not actively working on them.

Oh, and make sure you're up to date on releases. Last week there was a huge memory leak, but it was patched in Monday's release.

which version has a memory leak issue, stable 1.51? Monday's release refters to a nightly version?

Rust analyzer isn't shipped as a component with rustup. I don't know the versioning scheme — just make sure you don't have any updates pending.

1 Like

Since it sounds like all memory is being consumed (you should check memory usage, for example using htop?), I'd make sure to exit everything that competes for memory usage and try the build that way (including RA, browsers etc if applicable).

If it's at the end of the build, it sounds like it's a linking issue. Are you building a release build with full optimisations (e.g. LTO)? Are you using a system linker? Is it gold or lld?

2 Likes

The memory usage even reached 95% (16G Memory) + 100% (2G Swap) during the compilation process.

Maybe I should increase the swap partition space?

Are you building a release build with full optimisations (e.g. LTO)? Are you using a system linker? Is it gold or lld ?

I use the default rust usage and do not make any specific configurations.

How can I check what system linker is being used locally?

8-cores/threads could easily swamp 16Gb, sometimes I have trouble with my 6-core part and 32Gb.

On linux you can just type lld in the console if it's installed my assumption is that it's the default.

Just run ld --version. ld.gold is reported as GNU gold, ld.bfd as plain GNU ld.

Linkers are usually named like ld.gold, ld.bfd, etc. Just type ld. and hit tab-tab for autocomplete to see what is installed/available. Don't know if lld is installed as ld.lld or only lld -- if the latter you might need to do something like Cannot find 'ld' when using 'lld'

There is also GitHub - rui314/mold: mold: A Modern Linker , it's not production ready but it targets debug builds and is very fast. Not sure if it would solve memory issues.

Somewhat offtopic -- at work in our C++ projects I've tried to use split DWARF (e.g. Improving C++ Builds with Split DWARF – Productive C++) -- which stores debug info outside of object files and that makes linking much faster since the linker doesn't have to merge all debug info. Has anyone tried it with Rust?

1 Like

Split DWARF is implemented afiak.

@qaopm Thanks for your reply. It the first time to know about DWARF. It sounds so cool!

The result of ld --version:

$ ld --version
GNU ld (GNU Binutils for Ubuntu) 2.34
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

Another question, How does the system linker influence the rust compiling speed. I think I use the default installed linker in ubuntu 20.04 at present. You mean we could change to another linker manually to make rust compile faster or use less memory recourse?

I increased the swap space size from original 2G to 16G yesterday, and recompile the project after cargo clean several times. It seems no stuck happen again.

Yes, you can tell cargo to use different linkers. Generally lld is faster than gold which is faster than bfd (your system default). I have never really checked memory consumption so I can't comment on that but gold is reportedly using less memory than bfd but I don't know by how much and whether it'd make any difference in your case.

This all assumes that your memory problem is linking. :slight_smile:

You can try non-system linker by e.g. adding .cargo/config.toml in your project with this content:

[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "link-arg=-fuse-ld=gold"]

(To verify that the config is actually used, use some random string instead of gold: cargo should complain that it cannot find the linker with that random name. Then you know you're editing the right file. )

1 Like

I had the same problem on Ubuntu, started a few days ago, my screen would freeze. For me, looking at htop, I saw that coc rust analyzer was running a bunch. coc and rust analyzer just updated about 5 days ago, and the version requirements were out of sync (for me) (requiring in vim a :PluginUpdate and :CocUpdate) and so it (is possible that it) was causing rust analyzer to keep recompiling crates, which appears to have frozen things. I'm not positive this is the cause of my problems, but we will see. Not sure if that will help you.

Curious, does that sound like your problem?