Build from actual source

I noticed that when I download the source of version 1.65.0 and try to compile it instead downloads a binary by default. Due to auditing requirements, we cannot do this.
How can I change this to do the complete compilation without sideloading binaries ?
I dont have rust installed previously, running linux and have gcc 12.

The Rust Project's compiler is self-hosted, so you need a rust compiler to build it.

If you need to bootstrap from source code (ignoring that going far back enough there's probably a C/C++ compiler that wasn't compiled from source) you can look into the mrustc-project, which is supposed to be able to bootstrap the Rust compiler from C++.

2 Likes

Great, I will try that. Thanks :grinning:

Keep in mind that mrustc can not compile 1.65 rust compiler. It can only do 1.54.

That means that if you can not use binaries then you would have to slowly and painfully go from there to 1.65.

That's similar to how GNAT works, there and it's very well known that Ada is used in such environments.

2 Likes

mrustc -> rust 1.54 -> rust 1.65 it is then :blush:

That's not guaranteed to be a one-step process, either. It's only guaranteed to go one version at a time, i.e. 1.54 -> 1.55 -> 1.56 -> ... -> 1.64 -> 1.65. It may be able to skip some steps, but I'm afraid not many of them.

1 Like

Us spectators are cheering you on...

Let us know if you succeed.

8 Likes

You may be interested in this zulip thread where a guix contributor optimizes their from-source bootstrap process which lead to this commit

3 Likes

So I got mrustc compiled ok
Rust 1.54 rustc and cargo seemed to compile ok.
However and as people mentioned, compiling 1.65 directly after that failed. Looks like in libstd/core there is a crate called unicode-ident required which in turn requires std and core to compile which doesnt exist in 1.54.
So it might work if you compile each new version, but I'm out of time for now.

I find it troublesome that it is not easily possible to compile the compiler from scratch. I cant audit and verify application integrity for my own compiled rust programs.

Maybe make a shell script to automate the incremental compiles so it does it's work and your time is free for other things?

1 Like

What Linux distribution is that? From where do you get the GCC? I don't know whether the answers to these questions will help us help you, but I figure it's worth trying.

Let's be realistic here. The Rust compiler is almost a million lines of code. It depends on LLVM, which is an order of magnitude larger. The compiler is constantly changing, you'd have to audit every intermediate version. There is zero chance that you're going to audit that in any meaningful way, it's just a bureaucratic requirement.

5 Likes

This is a custom linux distribution. GCC is downloaded from GNU.org, bootstrapped and compiled from source.

Auditing does not necessarily mean you need to read every line of code before you can execute the application. However, being able to debug source code causing a problem and correct the code with a bugfix and recompile, is critical at least to us.

The question is how do you solve Trusting Trust dilemma. Do you start with Mes on trustable hardware without updateable microcode (80486? 80386?).

Practically speaking in today's world where even computer have tons of proprietary software all these motions are about of “feeling good” then about preventing real attacks.

You don't need to start from sources for that. Existing binary compiler is more than enough.

2 Likes

Unless you're using nightly, it is extremely unlikely that you will hit a compiler bug. Rustc has a huge and complex test suite. Literally all public crates are regularly checked for any kind of regressions. Even if you do find a bug, fixing it in a huge and complex codebase is very difficult, and if you want to always recompile from C it's plain impossible.

If you want to do some compiler development, you should use the standard process, which is based on downloading the latest beta branch compiler and using it to compile the source.

1 Like

So you use a binary download of GCC to compile GCC?

You can do that for Rust too. See instructions in https://rustc-dev-guide.rust-lang.org/building/how-to-build-and-run.html.