ARM soft-float cross compilation doesn't work. All steps below

help
1

Hi everyone,

I'm using for development Arietta G25 very nice little board (http://www.acmesystems.it).
It is using ARM 9 @ 400 MHz SoC from Atmel that is fully supported by Linux mainline kernel.
However I can't create any rust binary that would work on this platform, where simple C hello world cross-compiled on other box works perfectly. Rust code is just simple "Hello World".
Here are my steps - what I'm doing wrong here ? Anyone can give me a hint ?

My environment:

spock@fx160:~/devel$ cat ~/.cargo/config
[target.arm-unknown-linux-gnueabi]
linker = "arm-linux-gnueabi-gcc"

spock@fx160:~/devel$ rustup show|grep arm
arm-unknown-linux-gnueabi
arm-unknown-linux-gnueabihf

Cross compilation seems to work for rust and C program:

spock@fx160:~/devel/titleparser$ cargo build --target arm-unknown-linux-gnueabi
   Compiling titleparser v0.1.0 (file:///home/spock/devel/titleparser)
spock@fx160:~/devel/titleparser$ file target/arm-unknown-linux-gnueabi/debug/titleparser
target/arm-unknown-linux-gnueabi/debug/titleparser: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=e636aa3692e8e0d7c0554fae6ae9e86774fae70d, not stripped

spock@fx160:~/devel$ arm-linux-gnueabi-gcc -o arm-test arm-test.c
spock@fx160:~/devel$ file arm-test
arm-test: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=8ef02e1e69d7141825d25b310ade15ebebc14cba, not stripped

Let copy both binaries to target system:

spock@fx160:~/devel$ scp arm-test g25.lan:.
arm-test
spock@fx160:~/devel$ scp titleparser/target/arm-unknown-linux-gnueabi/debug/titleparser g25.lan:
titleparser
spock@fx160:~/devel$ ssh g25.lan
Linux arietta 4.4.4 #3 Sun Mar 6 13:49:08 CET 2016 armv5tejl

Last login: Tue Aug 30 10:16:27 2016 from router.lan
spock@arietta:~$ uname -a
Linux arietta 4.4.4 #3 Sun Mar 6 13:49:08 CET 2016 armv5tejl GNU/Linux

And here is result :confused:

spock@arietta:~$ ./arm-test
Hello world !
spock@arietta:~$ ./titleparser
Illegal instruction
spock@arietta:~$

Last thing you can see when running with strace is

getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
readlink("/etc/malloc.conf", 0xbe9ab5dc, 4096) = -1 EINVAL (Invalid argument)
brk(0)                                  = 0x7f645000
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPC, si_addr=0x7f615140} ---
+++ killed by SIGILL +++
Illegal instruction

Thanks in advance !

2

What about using ulimit -c unlimited on board, and then using gdb from toolchain open core,
and find out on what exactly instruction your program crashed?

3

Looks like either some bug in rust or I'm missing some gcc option i don't know.

(gdb) run
    Starting program: /home/spock/titleparser
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/arm-linux-gnueabi/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x7f579140 in atomic_write_uint32 (x=3, p=0x7f5a842c <lg_dirty_mult_default>)
    at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/atomic.h:497
497     /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/atomic.h: No such file or directory.
(gdb) bt
#0  0x7f579140 in atomic_write_uint32 (x=3, p=0x7f5a842c <lg_dirty_mult_default>)
    at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/atomic.h:497
#1  atomic_write_z (x=3, p=0x7f5a842c <lg_dirty_mult_default>)
    at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/atomic.h:595
#2  je_arena_lg_dirty_mult_default_set (lg_dirty_mult=3)
    at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/src/arena.c:3182
#3  je_arena_boot () at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/src/arena.c:3594
#4  0x7f56c730 in malloc_init_hard_a0_locked () at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c:1243
#5  0x7f56cae8 in malloc_init_hard () at /buildslave/rust-buildbot/slave/stable-dist-rustc-cross-host-linux/build/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c:1373
#6  0x7f58f738 in __libc_csu_init ()
#7  0xb6e1c25c in __libc_start_main (main=0xbefff764, argc=-1225486336, argv=0xb6e1c25c <__libc_start_main+168>, init=0x7f58f6ec <__libc_csu_init>,
    fini=0x7f58f74c <__libc_csu_fini>, rtld_fini=0xb6fdf630 <_dl_fini>, stack_end=0xbefff764) at libc-start.c:246
#8  0x7f558660 in _start ()
(gdb)
4

what about (gdb) disassemble atomic_write_uint32?

Any way looks like jemalloc for atomic implementation uses ARM instruction which your cpu (armv5tej)
not supports. Google give you tons of links on search request "armv5 atomic".

To work around this you should at least rebuild (with -march=armv5te) and may be hack jemalloc to fix it's atomic implementation.

5

Unfortunately the target you were trying to use is ARMv6. It doesn't make much sense, but somehow, good advice to bump it down to v5 got ignored a few times.

Instead, you should follow this:

https://github.com/joerg-krause/rust-cross-libs/blob/master/cfg/armv5te-unknown-linux-gnueabi.json

6

I'm trying https://github.com/joerg-krause/rust-cross-libb, but is seems that with 1.14 nightly build there are some linking issues with ARMv5te musl build root I've created with buildroot tool. It seems that support for hardware not supported by official releases is a bit chaotic and tricky and requires reading a lot of data sheets and assembler opcodes :confused:
I will post here my final result for others when I will get running result.