Cross-compilation not working

help
1

Hi,
I am quite new to Rust and tried crosscompiling a simple hello-world program, but it doesn't either work for mips nor armhf. I downloaded a current rustc and rustlib for mips and armhf. I also got gcc for cross-compiling.
When I try armhf, I get:

rustc --target=arm-unknown-linux-gnueabihf -C linker=arm-linux-gnueabihf-gcc hello.rs 
error: linking with `arm-linux-gnueabihf-gcc` failed: exit code: 1
note: "arm-linux-gnueabihf-gcc" "-Wl,--as-needed" "-L" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "hello.0.o" "-o" "hello" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libcollections-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/librustc_unicode-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/librand-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liballoc-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liballoc_jemalloc-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liblibc-17a8ccbd.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libcore-17a8ccbd.rlib" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "compiler-rt"
note: /usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liballoc_jemalloc-17a8ccbd.rlib(jemalloc.pic.o): In function `malloc_conf_init':
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:919: undefined reference to `secure_getenv'
collect2: error: ld returned 1 exit status

error: aborting due to previous error

When trying mips:

rustc --target=mips-unknown-linux-gnu -C linker=mips-openwrt-linux-gcc hello.rs
'generic' is not a recognized processor for this target (ignoring processor) [repeately ...]
error: linking with `mips-openwrt-linux-gcc` failed: exit code: 1
note: "mips-openwrt-linux-gcc" "-Wl,--as-needed" "-L" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib" "hello.0.o" "-o" "hello" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/libstd-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/libcollections-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/librustc_unicode-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/librand-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc_jemalloc-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/liblibc-17a8ccbd.rlib" "/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/libcore-17a8ccbd.rlib" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "compiler-rt"
note: /usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc_jemalloc-17a8ccbd.rlib(jemalloc.pic.o): In function `malloc_conf_init':
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:843: undefined reference to `__stack_chk_guard'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:919: undefined reference to `secure_getenv'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:919: undefined reference to `secure_getenv'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:1146: undefined reference to `__stack_chk_fail'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:1146: undefined reference to `__stack_chk_fail'
/usr/local/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc_jemalloc-17a8ccbd.rlib(arena.pic.o): In function `arena_run_tree_insert':
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/arena.c:91: undefined reference to `__stack_chk_guard'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/arena.c:91: undefined reference to `__stack_chk_fail'
/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/arena.c:91: undefined reference to `__stack_chk_fail'

And lots of more lines.
Am I missing something?
Thank you!

2

I'm not sure what's causing this problem with jemalloc but I can reproduce it.

In the meantime, you can cross compile your binaries against the system allocator. Or at least, that's working for me.

// hello.rs
#![feature(alloc_system)]
extern crate alloc_system;
fn main() {
    println!("Hello, world!");
}

$ rustc --target=mips-unknown-linux-gnu -C linker=mips-openwrt-linux-gcc hello.rs && echo OK
'generic' is not a recognized processor for this target (ignoring processor)
...
OK
3

My guess would be that rustc and the libs are not the same version. For example, rustc is 1.5 but the libs are for nightly.
I would go for the more reliable solution and cross-compile rust for the target I am interested in, that would guarantee that you have the correct rustc and the correct libs. Or just make sure you use the correct libs and rustc.
If you want to cross-compile it yourself, as a starting point, you could start with Ogeon's instructions.
May be @ogeon can help here too?

4

It looks like some pretty basic symbols are missing. Could it be a gcc version problem? Or some missing system library?

Also, I'm far from an expert on this. All I know is that the procedure in that guide works for the Raspberry Pi tool chain and can be used as a template for some other tool chains, as well.

5

Thanks for your help. Using the system allocator works.
I double-checked all the libraries, they all have the same git hash.

6

Added an issue: Cannot build for mips and armhf with jemalloc · Issue #30966 · rust-lang/rust · GitHub

7

Raspberry PI definitely works.
Then I think that my initial guess might be right, the issue might be that you try to cross-compile with a toolchain that was not cross-compiled for the architecture you are trying to build against.
It's 2:40 AM over here but let me see if I can get it working for OpenWRT(mentioned in the issue) based on Ogeon's instructions.

Update:
Here is a quick and dirty script that should get you started(note that the script makes the assumption that you use a Debian based distro and there is no error checking): build_mips.sh · GitHub
I am to sleepy right now to wait for all of it, so it was not tested but it should work.
After it compiles, you can follow Ogeon's instructions. Hopefully you won't have issues with how the toolchain is named and clang not detecting the "target triple".

8

Thanks. I've entered your script step by step. When configuring rust I get

configure: error: unsupported target triples "mips-openwrt-linux-uclibc" found

Here, they say it's because of an old gcc; in fact it works with a newer buildchain, e.g. http://releases.linaro.org/components/toolchain/binaries/latest-5.2/arm-linux-gnueabihf/

9

Nah, it's because LLVM doesn't know of mips-openwrt-linux-uclibc, it might know of something simpler like mips-unknown-linux-uclibc. It can't know of every <vendor> there is. Let me try and get back to you.
Anyway I remembered this morning that I forgot to put in something very important into the script: to bring the Rust git to exactly the same version of your Rust compiler...

Update:
Ok, it's a limitation in Rust configure script. I looked at the archive you mentioned in the issue rust-std-nightly-mips-unknown-linux-gnu.tar.gz and you can see that it is built for target mips-unknown-linux-gnu.
Here is another issue, glibc and uclibc are different. I don't think it will work if you cross-compile for mips-unknown-linux-gnu, it might be needed to adapt the configure script.

My guess is that it will build the compiler but I don't think it will work on the OpenWRT target.
If it doesn't work, you can still get Rust on that target by using Musl but that would mean that you can't use any of the precompiled system libraries(ex: libssl).

I bricked my OpenWRT device, so I don't have one now. I would need your help in finding if it works for your device.
I am compiling right now... Also, I will update the script, try it out.

Update:
Ok, seems like this also needs the "Where is that mips-linux-gnu-gcc compiler you were talking about?".
I will fix that and after that I will clean-up the script cause it hurts my eyes and soul looking at it.

10

Ok, I've thrown the code in functions to clear my soul a bit...
The same gist: build_mips.sh · GitHub

I haven't run the script myself yet nor did I look over it carefully but I am 90% confident that it should work. What can go wrong? :smile:
l'll update this post with my results after I run the script.

11

Thanks for your help. With the symlinks to the cross gcc, I have now built a rust with mips std crate, and cross compiling now works without jemalloc missing symbols!

1 Like
12

Mine finished compiling just now, lol...
Using Ogeon's instructions you can actually setup cargo and make it dead simple to cross-compile Rust projects.
I have it setup for an ARM soft floating point(eabi) hardware, and I just call arm-cargo, ex: arm-cargo build --release --verbose and it automatically builds for my target.
And when I want to build for my PC I just call cargo build.
The only issues I had were with crates that use C libraries(in my case: libssl and pulseaudio).

Note that it compiles but it might not work on your target system. You need to compile against the provided sysroot(OpenWRT SDK) to make sure you use the same version of the libs that are used on the target.
That's why I keep mentioning those instructions... If you used my script, your sysroot is now here:
$HOME/build-rust/openwrt-sdk/staging_dir/target-mips_34kc_uClibc-0.9.33.2

I still think this won't work on the target because the SDK uses uclibc. But I am not sure about that.

13

Thanks again.