Staticlib problems on iOS

#1

I’m working on a project for Unity3D and using rust for a native plugin. Fundamentally what I’m doing is pretty simple - compiling opus and libspeexdsp into static libs, pulling these into rust and exporting some functions that do some tiny extra bits of pointer arithmetic for me. Finally I use rust to compile a library which is dropped into Unity (normally a dylib).

However this process has to change slightly for iOS which does not support dynamic linking - for this Unity takes a static library and links it into your game. This fails at runtime with an EXC_BAD_ACCESS error (exactly the same error as this stackoverflow question). It looks like Opus and Unity both define a method called compute_allocation.

I checked the lib rust produces with nm and the compute_allocation function is not listed - I suspect my rather patchy knowledge of linking is letting me down, as I don’t even know how this issue is possible in that case!

Here’s a cut down version of my rust source:

pub enum OpusEncoder { }
pub enum OpusDecoder { }
pub enum SpeexPreprocessState { }

extern "C" {
    pub fn opus_encoder_create(Fs: i32, channels: i32, application: i32, error: *mut i32) -> *mut OpusEncoder;
}

#[no_mangle]
pub extern "C" fn dissonance_opus_encoder_create(samplingRate: i32, channels: i32, application: i32, error: &mut i32) -> *mut ::OpusEncoder {
    unsafe {
        return ::opus_encoder_create(samplingRate, channels, application, error);
    }
}

tl;dr Is it possible to configure rust to build it’s staticlib in a way which will fix this problem?

#2

First question, how are you linking against opus? Do you have a copy of your Cargo.toml and/or build.rs?

I have to think about it, but I’m suspicious of the stack overflow answer. Mach-o binaries (the container format for macOS and iOS) have something called “two level namespaces” which are specifically designed to avoid multiple symbol mix-ups, but this might be useless when statically linking (although the linker should be smart enough to know which symbol is being referenced, and link appropriately).

Try running nm -ufm (just preliminary googling, hopefully I’ll port rest of https://github.Com/m4b/rdr to rust and you can just run it on your lib)

#3

The relevant part of the build.rs is very simple, it emits println!("cargo:rustc-link-lib=static={}", name); for libopus.a and libspeex.a. The cargo.toml is also very simple, the only important bit is:

[lib]
crate-type = ["staticlib", "dylib"]

Technically the dylib part isn’t necessary for iOS, but we’re using this for a variety of other platforms too.

This build is run for a load of platforms (producing one .a file for each platform) and then finally we run:

lipo tmp/arm/libDissonanceNative.a tmp/aarch64/libDissonanceNative.a tmp/x86/libDissonanceNative.a tmp/x86_64/libDissonanceNative.a -create -output tmp/libDissonanceNative.a

That final libDissonanceNative.a created by lipo is what gets handed to Unity3D which itself then statically links it into the final game.

I don’t have access to a mac right now, I’ve asked another developer to run nm -ufm when he gets a chance.

#4

Indeed, the two-level namespace only exists for dynamic libraries. Since iOS does support dylibs these days (you just have to make sure it gets code signed properly), the easiest workaround is probably to just compile opus as a dylib.

#5

Unfortunately compiling a dynamic library isn’t an option - Unity on iOS does not support dynamic libraries as plugins :frowning:

#6

You copy the static lib in build.rs to the build directory or provide a path, yes? Most build.rs have a line where they copy the lib to the build directory using the OUT_DIR directory.

#7

I’m not entirely clear on what you’re asking, hopefully this will answer it…

We have a build-ios.sh script which runs the speex/opus builds, copies files around, runs cargo and then copies the output from that. Something like:

cd opus
configure
make
cp some_path/opus.a some_other_path/opus.a

cargo build --release --target=$RUST_HOST

cp target/$RUST_HOST/release/libDissonanceNative.a temp/$OUT_ARCH/libDissonanceNative.a

So this compiles opus (and speex, omitted for brevity) for the correct platform. Copies the build result into a directory in the rust project. Runs cargo, this runs the build.rs which emits:

cargo:rustc-link-lib=static=some_other_path/opus.a

Finally the output from rust is copied into a temp directory. Once all of the architectures have been run like this we run cargo lipo on the lot.

#8

Yea I was basically asking about the last part. Rust/cargo doesn’t warn or really do anything if symbols are undefined, or a library isn’t found, which can be caused by forgetting to pass the correct link name + search path.

As it stands I think you need a

println!("cargo:rustc-link-search=native={}", out_dir);
println!("cargo:rustc-link-lib=static=opus");

Note the lack of .a suffix on the link lib directive. In your case out_dir should be the path to wherever the libopus.a is. I also suggest you run this with cargo -vv

#9

BTW you can do all the configure/make/copy in your build script so you don’t have a bash shell to mess around with/another thing to ship.

Take a look at the build.rs here for example, in particular the non cmake path: https://github.com/m4b/capstone-sys

#10

I’m pretty confident Rust is finding all the symbols from opus/speex correctly. The current hacky fix is to rename the functions in Opus which collide with names in Unity3D until it works (obviously not a great long term solution, any internal change to Unity could break it again). Since this works Rust must be finding all the symbols it needs.

I’ll probably move as much of it as possible into the build.rs sometime. Because this same system is used for a variety of platforms it’d be pretty complex though - e.g. running the Opus build in visual studio on windows.

#11

Now I’m confused… It sounded like your rust static lib definitely did not have symbols from opus (compute_allocations) being one of them?

#12

Well compute_allocations is just some internal implementation detail of Opus, I wouldn’t expect it to be exported at all since it’s not marked with the OPUS_EXPORT macro (defined here). Maybe that doesn’t do what I think it does though!

Running nm on libopus.a (which is the input to rust) reveals:

libopus.a(celt_encoder.o):
         U _compute_allocation

libopus.a(rate.o):
         00000000 T _compute_allocation

Running nm on libDissonanceNative (the output from rust) shows that rust has exported a tonne of stuff but compute_allocation is not mentioned! Here is the complete file, search for “Dissonance”, “speex” and “opus” for the important bits.

So this is what’s confusing me, if Rust is not revealing the existence of compute_allocation how is Unity (which is further down the line) mixing it up?