Rust dynamic library dynamically called in Rust randomly crashed when statically linked to std


#1

I’m trying to dynamically load Rust code in Rust code, to be able to change the behaviour at execution time.

To achieve this, I have an independent crate declared as a dynamic library in it’s Cargo.toml:

[lib]
name = "module1"
crate-type = ["dylib"]

which declare one function:

#[no_mangle]
pub extern "Rust" fn compute(req: &str) -> String {
    println!("incoming data: {:?}", req);
    "done".to_string()
}

I then load this library in my main crate using libloading and call this function a lot. After a random number of calls, my program crash without any information.

let lib = libloading::Library::new(path).unwrap();
let compute = unsafe {
    let func: libloading::Symbol<extern "Rust" fn(&str) -> String> = 
        libloading.get(b"compute\0").unwrap();
    func.into_raw()
};
for i in 0..10000 {
    println!("{}", compute(&format!("{}", i)));
}

If I try to run using lldb, I get more details:

* thread #1: tid = 0x47dde, 0x00007fff864eb1f4 libsystem_platform.dylib`OSSpinLockLock + 7, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x200000902)
  * frame #0: 0x00007fff864eb1f4 libsystem_platform.dylib`OSSpinLockLock + 7
    frame #1: 0x0000000100033180 dynamic-load`je_tcache_bin_flush_small [inlined] je_malloc_mutex_lock + 208 at mutex.h:99 [opt]
    frame #2: 0x000000010003316b dynamic-load`je_tcache_bin_flush_small(tsd=0x000000010120d008, tcache=<unavailable>, tbin=0x000000010120e028, binind=0, rem=100) + 187 at tcache.c:119 [opt]
    frame #3: 0x0000000100018c63 dynamic-load`isfree [inlined] je_tcache_dalloc_small(tsd=0x000000010120d008, tcache=0x000000010120e000, ptr=<unavailable>, binind=<unavailable>) + 19 at tcache.h:419 [opt]
    frame #4: 0x0000000100018c50 dynamic-load`isfree [inlined] je_arena_sdalloc(ptr=0x0000000101217018, size=<unavailable>) + 343 at arena.h:1499 [opt]
    frame #5: 0x0000000100018af9 dynamic-load`isfree [inlined] je_isdalloct(ptr=0x0000000101217018, size=<unavailable>) at jemalloc_internal.h:1195 [opt]
    frame #6: 0x0000000100018af9 dynamic-load`isfree [inlined] je_isqalloc(tsd=0x000000010120d008, ptr=0x0000000101217018, size=<unavailable>) at jemalloc_internal.h:1205 [opt]
    frame #7: 0x0000000100018af9 dynamic-load`isfree(tsd=0x000000010120d008, ptr=0x0000000101217018, usize=<unavailable>, tcache=0x000000010120e000, slow_path=<unavailable>) + 201 at jemalloc.c:1921 [opt]
    frame #8: 0x00000001000186ec dynamic-load`je_sdallocx(ptr=0x0000000101217018, size=<unavailable>, flags=<unavailable>) + 300 at jemalloc.c:2669 [opt]
    frame #9: 0x00000001000043dd dynamic-load`alloc::heap::deallocate(ptr="103", old_size=3, align=1) + 61 at heap.rs:127
    frame #10: 0x00000001000048a1 dynamic-load`alloc::raw_vec::{{impl}}::drop<u8>(self=0x00007fff5fbff1d0) + 129 at raw_vec.rs:564
    frame #11: 0x0000000100003485 dynamic-load`core::ptr::drop_in_place<alloc::raw_vec::RawVec<u8>>((null)=0x00007fff5fbff1d0) + 21 at ptr.rs:60
    frame #12: 0x0000000100003530 dynamic-load`core::ptr::drop_in_place<collections::vec::Vec<u8>>((null)=0x00007fff5fbff1d0) + 64 at ptr.rs:60
    frame #13: 0x00000001000037f5 dynamic-load`core::ptr::drop_in_place<collections::string::String>((null)=0x00007fff5fbff1d0) + 21 at ptr.rs:60
    frame #14: 0x000000010000506c dynamic-load`dynamic_load::main + 700 at main.rs:34
    frame #15: 0x0000000100014f1b dynamic-load`panic_unwind::__rust_maybe_catch_panic + 27 at lib.rs:98 [opt]
    frame #16: 0x0000000100012f05 dynamic-load`std::rt::lang_start [inlined] std::panicking::try<(),closure> + 51 at panicking.rs:433 [opt]
    frame #17: 0x0000000100012ed2 dynamic-load`std::rt::lang_start [inlined] std::panic::catch_unwind<closure,()> at panic.rs:361 [opt]
    frame #18: 0x0000000100012ed2 dynamic-load`std::rt::lang_start + 434 at rt.rs:59 [opt]
    frame #19: 0x00000001000050da dynamic-load`main + 42
    frame #20: 0x00007fff8bcd25ad libdyld.dylib`start + 1

So… something went wrong while deallocating. And indeed, if I change the return type of my method to a i32, everything is OK, but I need a complex return type for my use case.

I randomly found what seems to be a solution: building my library with

cargo rustc -- -C prefer-dynamic

[details=randomly ?]While trying stuff, I added my module crate as a dependency to my main crate and notice everything was magically working even though I was still loading the library dynamically. It happens to be that libloading (through dlopen) will try to load the library in it’s LD_LIBRARY_PATH first even if I specify a path to my library. I then searched for difference, and with cargo -vv I found that the library crate was built with

rustc --crate-name module1 module1/src/lib.rs --crate-type dylib --emit=dep-info,link -C prefer-dynamic -C debuginfo=2 -C metadata=b9d361399f10d6b2 --out-dir /Users/francoism/perso/rust/dynamic-load/target/debug/deps -L dependency=/Users/francoism/perso/rust/dynamic-load/target/debug/deps`

[/details]

The main difference between the library build by cargo run and this command is that std is not present in the latest. Armed with the knowledge of this -C prefer-dynamic, I was able to find more information about how cargo statically link dependencies for dynamic library.

I have the solution to my problem, but I was wondering:

  1. Why is it crashing when std is statically linked in my dynamic library?
  2. Is it the best solution? I feel like I have fixed my issue without really understanding it.
  3. Was there another way to reach it? Having no error message at all didn’t help… The only unsafe is when loading the extern function, but it crashes when executing it which is not marked as unsafe.

git repo with minimal example to reproduce my issue


#2

If std is dynamically linked you have two copies of jemalloc - one in the main binary and one in the shared library.