Static build's -fPIC cc::Build flag ineffective?

I'm working on a project that has a bindgen C lib (htslib) dependency, which is wrapped by (rust-htslib) and I'm using a few -sys crates that facilitate static compilation of dependencies, such as xz/lzma:

$ git clone https://github.com/brainstorm/htsget-aws && cd htsget-aws && cargo build

I cannot compile it statically and built against htsget-aws since htslib (the underlying lib that rust-htslib is trying to wrap) has some linker issues:

(...)
" "/home/limsadmin/dev/htsget-
"/home/limsadmin/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-428d15b56101bdc7.rlib" "/home/limsadmin/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-941f7f6bc622e313.rlib" "-Wl,--end-group" "/home/limsadmin/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-e915d31ab7edbbd4.rlib" "-Wl,-Bdynamic" "-lutil" "-ldl" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-ldl" "-lutil"
  = note: /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(bgzf.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(header.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(hfile.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(hfile_net.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(hts.o): relocation R_X86_64_32S against symbol `seq_nt16_table' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(multipart.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(region.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(sam.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(textutils.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(thread_pool.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_decode.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_encode.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_index.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_io.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_samtools.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_stats.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(mFILE.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(open_trace_file.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(rANS_static.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(hfile_libcurl.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(knetfile.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(kstring.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(faidx.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(md5.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
          /usr/bin/ld: /home/limsadmin/dev/htsget-aws/target/debug/deps/libhts_sys-7dace0b999e82069.rlib(cram_codecs.o): relocation R_X86_64_32S against hidden symbol `cram_byte_array_stop_encode_free' can not be used when making a shared object
          /usr/bin/ld: final link failed: Nonrepresentable section on output
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `reads`.

To learn more, run the command again with --verbose.

Even when one of the underlying libs has everything I can imagine pointing and set to static compilation:




There seem to be some unresolved symbols on htslib though when looking at the generated libhts.a and its objects when grepping the symbols from the linker errors above:

$ cd ./target/debug/build/hts-sys-76fd95c663d78e71/out
$ ar x libhts.a
$ nm *.o | grep seq_nt16_table
0000000000000640 R seq_nt16_table
                 U seq_nt16_table
                 U seq_nt16_table

That library and symbols seem to have some priors in Debian arm64 builds for packages that depend on it, for instance "segemehl" and "pysam" itself:

http://crossqa.debian.net/build/segemehl_0.3.4-1_arm64_20190306143015.log

So before I dig further on htslib's (static) building logic (and Rust's too), do those unresolved symbol errors sound familiar even when -fPIC cc flags are passed in the aforementioned build.rs files? I've tried with regular CFLAGS="-fPIC" cargo build with same effects, unfortunately :confused:

What am I doing wrong? Any pointers/help are highly appreciated!

1 Like

I figured this out (partially) by modifying rust-htslib/hts-sys/build.rs that CFLAGS were not being passed correctly during either autoreconf, configure or make itself:

In addition, the -B flag passed to make, also made the build fail to (re)-build reliably (still don't know why):

Now at least it's just erroring out on undefined references for the symbols that I'm actually using on my own crate, htsget-aws:

htsget-aws]$ cargo build -vvv
(...)
nown-linux-gnu/lib/libcompiler_builtins-e915d31ab7edbbd4.rlib" "-Wl,-Bdynamic" "-lutil" "-ldl" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-ldl" "-lutil"
  = note: /home/limsadmin/dev/htsget-aws/target/debug/deps/librust_htslib-6602f5d5804d9af5.rlib(rust_htslib-6602f5d5804d9af5.2kgc5d1okt3lai62.rcgu.o): In function `rust_htslib::bam::IndexedReader::new::hbccbc07dfdea5d58':
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:362: undefined reference to `sam_hdr_read'
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:364: undefined reference to `sam_index_load'
          /home/limsadmin/dev/htsget-aws/target/debug/deps/librust_htslib-6602f5d5804d9af5.rlib(rust_htslib-6602f5d5804d9af5.2kgc5d1okt3lai62.rcgu.o): In function `rust_htslib::bam::hts_open::haa5c134654a9c329':
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:773: undefined reference to `hts_open'
          /home/limsadmin/dev/htsget-aws/target/debug/deps/librust_htslib-6602f5d5804d9af5.rlib(rust_htslib-6602f5d5804d9af5.32dfbenixrzvd7k8.rcgu.o): In function `_$LT$rust_htslib..bam..IndexedReader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hb5567df68290afc9':
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:525: undefined reference to `hts_itr_destroy'
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:527: undefined reference to `hts_idx_destroy'
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:528: undefined reference to `hts_close'
          /home/limsadmin/dev/htsget-aws/target/debug/deps/librust_htslib-6602f5d5804d9af5.rlib(rust_htslib-6602f5d5804d9af5.32dfbenixrzvd7k8.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h065e2094af67d830':
          /home/limsadmin/dev/rust-htslib/src/bam/mod.rs:934: undefined reference to `sam_hdr_destroy'
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `reads`.

EDIT: A bit more context for the issue:

$ pwd
/home/limsadmin/dev/htsget-aws/target/debug/deps
$ ar x librust_htslib-6602f5d5804d9af5.rlib
$ nm *.o | grep hts_open
                 U hts_open
0000000000000000 t _ZN11rust_htslib3bam8hts_open17haa5c134654a9c329E
                 U hts_open
                 U hts_open

Getting there :slight_smile:

If you're building on linux - the linux linker is sensitive to the order of the libraries; check that library containing hts_* functions is linked after the code that references them. If the library architecture is correct, it should solve your missing symbols issue.

Thanks Berkus!

This was all that was needed after the former tweaks!:

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.