Dryad and cargo: unusual linking requirements


#1

Hello there!

So recently I announced dryad, which is an experimental dynamic linker written entirely in Rust (and some asm), currently for GNU/Linux x86-64.

I’ve been procrastinating getting cargo setup and working, and I’ve noticed that when I mention I can’t get cargo working with my project, there is some surprise that it isn’t easy/possible. So, I’ve come here to ask help from a rust cargo foo master, because I can’t seem to get it working.

This has some important (negative) consequences:

  1. I cannot write cargo tests
  2. I cannot use any extern crates :no_mouth:
  3. I cannot write cargo benchmarks
  4. I cannot tell people to just run cargo build to build the project, like everyone else gets to do

Diving in, here are important preliminaries describing my requirements:

  1. I must build asm stubs. In particular, they are in src/arch/x86/asm.s.
    a. I would prefer not to have these integrated into the rust source code (i.e., no inline asm).
    b. Mutabah attempted to fix some of the relocation issues using indirect loads, otherwise cargo build errors out with PIC errors. Unfortunately, I had to revert these changes as while it allowed cargo to build the project, the binary doesn’t run, and the new relocations cause ./dryad.so.1 to segfault early in the process, so they did not help :frowning:
  2. The rust code is currently compiled as an object file with --target=x86_64-unknown-linux-musl.
  3. Using the asm and rust objects, I hand link with some special flags to produce the final executable. This is the snag in the pipeline, as a dynamic linker requires all of the following, with link flags following the description:
    a. the binary’s soname has to be set to dryad.so.1: -soname dryad.so.1
    b. the binary must be compiled as a PIE. What this means is that the binary is technically a shared object (it’s e_type in the ELF header is ET_DYN), but it also has a valid e_entry (set to the _start symbol defined in the asm.s): -pie
    c. the binary must be linked without the C stdlib (we will receive all of it by statically linking with musl libc): -nostdlib
    d. as a consequence of c., we must set the entry point, in this case _start as defined in asm.s: -e _start
    e. we should “bind references to global symbols to the definition within the shared library”, i.e.: -Bsymbolic
    f. we should set the path of the program interpreter (e.g., the dynamic linker) of dryad (a dynamic linker) to itself. (technically I don’t need to do this, but I think it’s hilarious, and afaik no other dynamic linker does this. It may come as a surprise, but it does work, which you can verify by running ./dryad.so.1 after you build it with ./make.sh): -I /tmp/dryad.so.1
    g. we should gc sections so our binary isn’t enormous: --gc-sections
    h. finally, we should properly link against all the musl rust stdlibs and liblibc.rlib
    i. the resulting binary should not have any DT_NEEDED elements in its _DYNAMIC array, i.e, it does not have any dynamic library dependencies.

Essentially everything above is encapsulated in my make.sh script, if you want to see how it all works to produce an almost working dynamic linker (having TLS issues, but that’s another post ;)).

Currently, if you clone the repo and do cargo build, it ~should~ won’t build (assuming you have nightly cargo). With Mutabah’s fix (currently commented in the asm.s), cargo will build without complaining about PIC relocations, but the binary will segfault early in the process if built using ./make.sh (I don’t know why, another mystery), and the resulting cargo executable is not an executable, but a shared library without an entry point, and does not satisfy every one of the above requirements, e.g., it has rust dynamic lib dependencies, as well as libc, doesn’t have the custom soname, etc.

As such, I reverted the asm changes, and cannot build using cargo build at all…

Due to the nature of this project, I understand these are some unusual requests, but what else is a systems language for if not writing something like a dynamic linker :wink:

As a result, if you are someone interested in highly esoteric aspects of the ELF linking process, and would like to help me getting cargo to build an executable with the above requirements, I would be greatly and deeply indebted :bow: .

Unfortunately, this would almost be a moot point if we could pass linker flags via the rust build script using the -C switch, e.g.:

println!("cargo:rustc-flags=-C hi there")

But cargo currently refuses, and errors out with:

error: Only `-l` and `-L` flags are allowed in build script of `dryad v0.1.1 (file:///home/m4b/projects/dryad)`: `-C hi there`

So again, if anyone has any ideas at all, I’m all :ear:

Live long and prosper :vulcan:,
m4b