Clarifications on Rust's Relationship to Libc

Preface

I'm hoping this thread will be a useful resource for others (and myself) in clearing up what the libc dependency is, is not, and what is entailed by it. Efforts to answer this myself are listed at the bottom.

Question

I'd like to figure out the exact relationship Rust has with libc & friends, using a hypothetical situation.
Suppose our friend Bob creates his own:

  • processor: 33bit LEG processor
  • assembly: z69 instruction set
  • kernel: creationism kernel (rewritten Temple OS kernel)
  • OS: Doors OS
  • executable format: Goblin

Bob has also ported every combination of the above "system stack" with exising linux equivlents (ex: Doors OS can run on a linux kernel, an x86 processor, and on the reverse side he's made linux distros that can execute the Goblin format and run on his 33bit LEG processor). Everything Bob has written is assembly, including rewriting the linux kernel for his 33bit LEG processor. Yes; Bob is outragously productive with his spare time.

Bob really likes Rust. For each of those system stacks he created, he has a few goals:

  1. Create a binary, compiled from rust no-std source, using rustc from a tier-1 OS, targeting as many of his systems as possible
    (ex: compile rust no-std hello world-like program on Ubuntu into a Goblin executable)
  2. Create a binary, compiled from rust that uses std, using rustc from a tier-1 OS, targeting as many of his systems as possible
  3. Get rustc functioning on his system stack
  4. Be able to target Linux/ELF and MacOS/Mach-O from Doors OS

Bob wants to know which system-stack + goal combinations are do-able

  • Without modifying/extending rust-core/rust-std
  • Without creating/porting libc (but giving in and modifying rust-core/rust-std)
  • Without modifying/extending clang/gcc (but giving in about both libc and rust-core/rust-std)
  • Without modifying/extending LLVM (but giving in about all the previous things)

Bob's list ^ might seem confused or missing some stuff, like clang needing LLVM to be modified, or Rust's libc crate needing to be modified instead of rust-std.
(Coincidentally my question(s) are the same as Bob)

Exisiting Understanding:

Things Not Understood

(These may or may not be necessary to understand the answer)

  • Can rustc exist without a C compiler for the host system (is libc convenient or mandatory)
  • When targeting another system with rustup target add arm-unknown-linux-gnueabihf, it is unclear to me what that command is retreiving (and if it includes a libc)
  • Is libc a source-code dependency (c compiler required) or strictly a compiled-code dependency (only linking)
  • is libc (for the target) needed at compile time, even if the target system doesn't really use C for the kernel/OS
  • is it allowed for changes to be made to rust-std in order to support new OS, without creating an entire C/libc toolchain for that OS
  • I'm generally a high level programmer, I had to do a lot of kernel/linker/ELF etc research so I'm not partiularly an expert (or even intermediate) on OS-level designs
  • What guarentees that a fully-static binary is executable? (same CPU+kernel, but everything else can be different? what about kernel modules, OS Version, ABI, file system format, and available system calls, etc? I'm not sure)

What I've done

(Note: may take me awhile to respond)

6 Likes

You should really look into the Redox-OS project (full OS, kernel and all, written in Rust). They have had to answer all of these questions and probably are the most knowledgeable (outside the language/compiler/core Rust teams).

2 Likes

I don't know a ton about porting rustc/LLVM to new platforms, so I hope someone more knowledgeable will answer those questions. I can try to answer some of the other ones, though:

Yes. However, it requires a linker that runs on the host system and targets the target system. By default for some targets it uses a C compiler to invoke the linker, but this is configurable. Some work has been done on shipping LLVM’s lld linker with Rust and eventually using it by default for supported targets. This linker supports cross-compilation. This work would remove Rust's dependency on a C toolchain to provide the default linker.

Only linking.

No. You can compile no_std programs without libc. There are a few symbols that libcore expects to be linked in, but you can define these yourself (in Rust or C or assembly or whatever) or use any library that provides them. See the libcore docs for some details.

Similarly, libstd needs some way to call into OS-provided functionality like threading, I/O, and memory management. This doesn't have to be a library implemented in C, however.

It retrieves an archive like this one which contains the following libraries. It does not include a libc:

arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_std-a287996bfad99400.rlib
arm-unknown-linux-gnueabihf/lib/libadler-a8bb4e8244729481.rlib
arm-unknown-linux-gnueabihf/lib/libaddr2line-ce4966c04a7381e1.rlib
arm-unknown-linux-gnueabihf/lib/libstd-7d2100300ad23ef1.rlib
arm-unknown-linux-gnueabihf/lib/libcfg_if-39fffccebe5cee72.rlib
arm-unknown-linux-gnueabihf/lib/libgimli-c593ae256a2221e2.rlib
arm-unknown-linux-gnueabihf/lib/libpanic_abort-58ddabe83db2f9a4.rlib
arm-unknown-linux-gnueabihf/lib/libtest-6a8c7702af22a105.rlib
arm-unknown-linux-gnueabihf/lib/liblibc-6d42b01c4cdf6b65.rlib
arm-unknown-linux-gnueabihf/lib/libhashbrown-ad996634c3607b36.rlib
arm-unknown-linux-gnueabihf/lib/libunicode_width-f92ca432fc748ee6.rlib
arm-unknown-linux-gnueabihf/lib/libgetopts-70ef58264d2c4877.rlib
arm-unknown-linux-gnueabihf/lib/libobject-dff7326bcdfe5987.rlib
arm-unknown-linux-gnueabihf/lib/liballoc-1620e58bb5d5b468.rlib
arm-unknown-linux-gnueabihf/lib/libpanic_unwind-be6ac862d26d21f8.rlib
arm-unknown-linux-gnueabihf/lib/librustc_demangle-7698886f9361d419.rlib
arm-unknown-linux-gnueabihf/lib/libcore-b31fd07eaef1b229.rlib
arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_core-528a96701368b459.rlib
arm-unknown-linux-gnueabihf/lib/libunwind-9689da2f91341f4c.rlib
arm-unknown-linux-gnueabihf/lib/libcompiler_builtins-3c59998672e273dc.rlib
arm-unknown-linux-gnueabihf/lib/libminiz_oxide-6c787a4aca4eb60e.rlib
arm-unknown-linux-gnueabihf/lib/libterm-d9f24405a9cee77e.rlib
arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_alloc-afbb9ab8c08b0aaf.rlib
arm-unknown-linux-gnueabihf/lib/libproc_macro-9b8291987f432eaf.rlib
arm-unknown-linux-gnueabihf/lib/libstd-7d2100300ad23ef1.so
arm-unknown-linux-gnueabihf/lib/libtest-6a8c7702af22a105.so

If you download a no_std target like aarch64-unknown-none then it includes only these libraries:

aarch64-unknown-none/lib/liballoc-881733fbf8950f62.rlib
aarch64-unknown-none/lib/librustc_std_workspace_core-803ff0f63672e097.rlib
aarch64-unknown-none/lib/libcore-d740d3e4b49047ec.rlib
aarch64-unknown-none/lib/libcompiler_builtins-24748842079a27ef.rlib
5 Likes
  • Rust, the language, doesn't depend on libc. There are no-std targets. There are also targets like WASM/WASI which don't use libc in any way. If you were porting Rust's libstd to a new platform you could use direct kernel syscalls, or link with whatever not-C standard library the OS has.

  • rustc, the compiler, currently depends on LLVM, and therefore on libc.

So if you want to run the compiler on the target platform, you will need to port enough of C and C++ stack to run LLVM on there.

This might change eventually when Rust's cranelift back-end matures, and you manage to port and cross-compile cranelift for the target platform first.

8 Likes

On all major OSes outside of Linux you have to use libc or its analog, since it's the only stable way for a program to talk with OS. You can compile Rust programs for Linux which do not link libc using the MUSL targets (though it can make it difficult to dynamically link other libraries) and hopefully in future we will get a target with a proper libc-free std (see this link for more information). While I personally have high hopes for it, but unfortunately there is not much development on this front.

Important to note that this limitation is imposed by the OS not providing any other sort of stable/documented/approved interface as opposed to Rust not wanting/able to use a different interface to the OS.

-- just wanted to clarify for other readers who may not have gotten that point

1 Like

Thank you all so much for these awesome answers. I think together they cover 95% (only two minor questions at the bottom). I'll try to confirm/summarize here to make sure I'm unstanding correctly.

In this case, then

  1. Bob could acheive his goal #1 (no-std execution) simply by creating files similar to what @mbrubeck listed, e.g.
leg33-unknown-none/lib/liballoc-881733fbf8950f62.rlib
leg33-unknown-none/lib/librustc_std_workspace_core-803ff0f63672e097.rlib
leg33-unknown-none/lib/libcore-d740d3e4b49047ec.rlib
leg33-unknown-none/lib/libcompiler_builtins-24748842079a27ef.rlib

Which is entirely free of any direct C influcence, and is very similar to the case of targeting WASM/WASI that @kornel mentioned

  1. Bob could achieve his #2 goal (executable utilizing rust std) so long as he modified the rust std source (?) and as @mbrubeck mentions, Bob would also need to implment files similar to the following in order to target his OS. The changes to rust std source would need to implement the std interface for his OS using system calls for his OS, or maybe using symbols that are dynamically or statically linked from a standard library he created for his OS. As @newpavlov mentions, the standard library for the OS effectively always (but not necessarily) ends up being libc (which explains why rust std depends on the libc crate, and the libc crate RFC explains why its a crate rather than independent implementations). Here's those files that need Bob needs to implment:
arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_std-a287996bfad99400.rlib
arm-unknown-linux-gnueabihf/lib/libadler-a8bb4e8244729481.rlib
arm-unknown-linux-gnueabihf/lib/libaddr2line-ce4966c04a7381e1.rlib
arm-unknown-linux-gnueabihf/lib/libstd-7d2100300ad23ef1.rlib
arm-unknown-linux-gnueabihf/lib/libcfg_if-39fffccebe5cee72.rlib
arm-unknown-linux-gnueabihf/lib/libgimli-c593ae256a2221e2.rlib
arm-unknown-linux-gnueabihf/lib/libpanic_abort-58ddabe83db2f9a4.rlib
arm-unknown-linux-gnueabihf/lib/libtest-6a8c7702af22a105.rlib
arm-unknown-linux-gnueabihf/lib/liblibc-6d42b01c4cdf6b65.rlib
arm-unknown-linux-gnueabihf/lib/libhashbrown-ad996634c3607b36.rlib
arm-unknown-linux-gnueabihf/lib/libunicode_width-f92ca432fc748ee6.rlib
arm-unknown-linux-gnueabihf/lib/libgetopts-70ef58264d2c4877.rlib
arm-unknown-linux-gnueabihf/lib/libobject-dff7326bcdfe5987.rlib
arm-unknown-linux-gnueabihf/lib/liballoc-1620e58bb5d5b468.rlib
arm-unknown-linux-gnueabihf/lib/libpanic_unwind-be6ac862d26d21f8.rlib
arm-unknown-linux-gnueabihf/lib/librustc_demangle-7698886f9361d419.rlib
arm-unknown-linux-gnueabihf/lib/libcore-b31fd07eaef1b229.rlib
arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_core-528a96701368b459.rlib
arm-unknown-linux-gnueabihf/lib/libunwind-9689da2f91341f4c.rlib
arm-unknown-linux-gnueabihf/lib/libcompiler_builtins-3c59998672e273dc.rlib
arm-unknown-linux-gnueabihf/lib/libminiz_oxide-6c787a4aca4eb60e.rlib
arm-unknown-linux-gnueabihf/lib/libterm-d9f24405a9cee77e.rlib
arm-unknown-linux-gnueabihf/lib/librustc_std_workspace_alloc-afbb9ab8c08b0aaf.rlib
arm-unknown-linux-gnueabihf/lib/libproc_macro-9b8291987f432eaf.rlib
arm-unknown-linux-gnueabihf/lib/libstd-7d2100300ad23ef1.so
arm-unknown-linux-gnueabihf/lib/libtest-6a8c7702af22a105.so
  1. Finally, and unlike the previous two goals, in order to get rustc running natively on Bob's Doors OS (as mentioned by @kornel), at time of writing, Bob's system would need to be able to run LLVM, which in turn depends on a libc. And the realistic way to compile LLVM to run on Doors OS, would be to extend LLVM to have Doors as a target system, and then use LLVM on a different system to compile LLVM itself and a libc for Doors OS.
    This point, in my humble opinion, is the only 'hard' dependency of the Rust ecosystem on the C ecosystem (and is the final piece of the funadmental answer I have been looking for). As @kornel mentions, this may eventually change with Rust's cranelift back-end potentially replacing LLVM (but all the above^ work is still needed, it would just be with rust equivlents instead of LLVM/libc)
  2. To cross compile will mostly be the same as any other system (e.g. get the files mentioned in #1 and #2). But as @newpavlov mentions, you will often need to dynamically link to the target's libc or use MUSL to statically link it. And that is because (as @gbutler69 mentions) libc is often the only stable way to interact with the target OS. (I surmise that) system calls and simlar are not guarenteed to have identical/consistent behavior, so to standardize most systems have put in a ton of work into libc, and (although maybe an outsider could, realistically) those people would need to implement some other (e.g. lib-rs) for their system. And the people in rare case of Redox OS, have effectively done exactly that; create a stable interface that isn't libc.

Last 5% Clarification

  • Can rustc have all static links? Or, for practical purposes will LLVM, and/or LLVM's links always be dynamic? (closest documentation I could find, and ldd rustc shows at least some dynamic links)
  • In #2, is the assumption that the rust std source needs to be modified correct? Or is the larger list of files that @mbrubeck provided sufficient for targeting a new OS.

_

That covers every bit of ambiguity I had :smiley: I'm shocked at how fast and thourough the responses were, thank you again @mbrubeck, @kornel, @gbutler69, @newpavlov. I switched to Rust to get away from C/C++ being underneath every language I used, so I'm extremely relieved to hear that libc is effectively an add-on rather than a requirement. And @kornel, I'm glad you mentioned cranelift's backend as that was something I didn't know about or research. I will be watching cranelift's backend closely to celebrate when rustc can compile itself and all it's dependencies for a new target OS.

Also, if anyone is reading this, but doesn't understand these answers, do ask follow up questions. I think I understand the system well enough now to answer them myself

1 Like

Note that his statement about WASI is not entirely correct. Unfortunately (to me) the WASI target is also dependent on a MUSL-derived variant of libc. I've suggested to add a libc-free WASI target and put some work towards it, but it does not look like it will happen...

1 Like

Thanks for mentioning this and the link. That is surpising. I'll continue the discussion on WASM over there after reading more about it

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.