Link Errors Depending on Call Location

I'm trying to do some slightly strange linking and getting some very strange errors.

Context:

I work with a tool that generates some C libraries based on a project from the vendor. This produces a library of common functions and a library of functions specific to the project.

I want to create a tool that helps you compile these C files in Rust and access the common functions and specific functions.

To do this I have the common crate. This includes a routine for building the libraries with CC and then has wrapping functions for the common functions we expect to be built.

So we end up with:

user crate -- builds --> common lib -- linked by --> common crate.

Problem:

In the common crate I have a session struct and I can use an extern block to reference the common functions in that struct and that builds and runs without issue.

If I attempt to use one of those functions in a different context I get linker errors. By this I mean:

  • Calling a function in a drop implementation.
  • Calling a function in a OnceCell initialisation.

As soon as I do this I get link errors for the symbols used in that impl block. I don't see the library in the link list.

Does the order of dependencies somehow change because I'm using outside traits?

This is the session code. If I comment out the drop impl then no build error:

//! Holds session management functions for the FPGA.
//!
use std::sync::Once;

use crate::error::{to_fpga_result, NiFpga_Status};

static LIB_INIT: Once = Once::new();

extern "C" {
    fn NiFpga_Initialize() -> i32;
    //fn NiFpga_Finalize() -> NiFpga_Status;
    fn NiFpga_Open(
        bitfile: *const i8,
        signature: *const i8,
        resource: *const i8,
        attribute: u32,
        session: *mut SessionHandle,
    ) -> NiFpga_Status;
    fn NiFpga_Reset(session: SessionHandle) -> NiFpga_Status;
    fn NiFpga_Run(session: SessionHandle) -> NiFpga_Status;
    fn NiFpga_Close(session: SessionHandle, attribute: u32) -> NiFpga_Status;
    fn NiFpga_ReadU8(session: SessionHandle, offset: u32, value: *mut u8) -> NiFpga_Status;
    fn NiFpga_WriteU8(session: SessionHandle, offset: u32, value: u8) -> NiFpga_Status;
}

pub type SessionHandle = u32;

pub struct Session {
    pub handle: SessionHandle,
}

impl Session {
    pub fn new(
        bitfile: &str,
        signature: &str,
        resource: &str,
    ) -> Result<Self, crate::error::FPGAError> {
        LIB_INIT.call_once(|| unsafe {
            //NiFpga_Initialize();
        });
        unsafe {
            NiFpga_Initialize();
        }

        let mut handle: SessionHandle = 0;
        let bitfile = std::ffi::CString::new(bitfile).unwrap();
        let signature = std::ffi::CString::new(signature).unwrap();
        let resource = std::ffi::CString::new(resource).unwrap();
        let result = unsafe {
            NiFpga_Open(
                bitfile.as_ptr(),
                signature.as_ptr(),
                resource.as_ptr(),
                0,
                &mut handle,
            )
        };
        to_fpga_result(Self { handle }, result)
    }

    pub fn reset(&mut self) -> Result<(), crate::error::FPGAError> {
        println!("reset");
        let result = unsafe { NiFpga_Reset(self.handle) };

        to_fpga_result((), result)
    }

    pub fn run(&mut self) -> Result<(), crate::error::FPGAError> {
        let result = unsafe { NiFpga_Run(self.handle) };

        to_fpga_result((), result)
    }

    pub fn close(self) -> Result<(), crate::error::FPGAError> {
        let result = unsafe { NiFpga_Close(self.handle, 0) };

        to_fpga_result((), 0)
    }

    pub fn read_u8(&self, offset: u32) -> Result<u8, crate::error::FPGAError> {
        let mut value: u8 = 0;
        let result = unsafe { NiFpga_ReadU8(self.handle, offset, &mut value) };

        to_fpga_result(value, result)
    }

    pub fn write_u8(&self, offset: u32, value: u8) -> Result<(), crate::error::FPGAError> {
        let result = unsafe { NiFpga_WriteU8(self.handle, offset, value) };

        to_fpga_result((), result)
    }
}

impl Drop for Session {
    fn drop(&mut self) {
        unsafe {
            NiFpga_Close(self.handle, 0);
        }
    }
}

How exactly are you compiling your code? What rustc version, rustc flags or cargo config and linker args? What linker? And what is the exact error from the linker?

The user crate builds the code using a build script which calls cc. I'm not adding any flags or config beyond what that adds by default.

This is on rustc 1.70 on Windows so I believe it is calling the MSVC linker.

The exact error is

libni_fpga_interface-3a1a5c2ac74636dd.rlib(ni_fpga_interface-3a1a5c2ac74636dd.3ymvxm1mx8ywmijn.rcgu.o) : error LNK2019: unresolved external symbol NiFpga_Close referenced in function _ZN77_$LT$ni_fpga_interface..session..Session$u20$as$u20$core..ops..drop..Drop$GT$4drop17h2a24a3b46a6ad111

This is with the drop section uncommented. If I uncommend the Once initialisation that also shows that symbol as unresovled.

All of the rest of the symbols in the library don't show any errors and I can use the same symbols in the Session impl without issue

Are you sure there is an exported symbol NiFpga_Close in the object file?

Yes because I can call it in other places and it seems to compile without issue.

That seems very strange. Can you share all of your code and build instructions?

I suspect you are using toolchain with the -gnu abi, this kind of behavior should not appear for msvc linker. can you check your toolchain and confirm? simply run the command "rustup default"

this is very likely caused by the strict linker order requirement, see e.g.

if my guess is correct, the fact you can call the function outside of Drop impl is pure accident. some of the object files generated by rustc are listed before the -l option while others are after, perhaps due to indeterministic codegen of rustc like caching and incremental compilation.

you can simply add a build script in your "common" crate to declare the link dependency, something like:

// build.rs
fn main() {
    println!("cargo:rustc-link-lib=user_must_provide_this_library");
}

if the library name cannot be hardcoded, you can use an environment variable, and at build time, either set the variable direct in the shell, or set the variable in your workspace's cargo config file.

Sure the repo is public at GitHub - WiresmithTech/ni-fpga-interface: NI FPGA Interface for Rust. I have some changes on a local branch but this demonstrates the issue.

The structure isn't great (still proving out the best method) but basically the repo is the common crate (ni-fpga-interface) and then under examples/host_example is the user crate.

In examples/fpga_c_interface is the C library I am trying to build.

The simple instructions are to cd to the host_example folder and try to build that. (cargo build). Then in src/session.rs uncomment the drop implementation and you will see the error.

I tried this but then the compile fails because the library hasn't yet been built when the common crate has been compiled. I guess I'm depending on the following order:

  1. Common crate is compiled.
  2. User crate builds lib. (in build.rs)
  3. User crate is compiled
  4. Exe is linked with the built lib.

When I specify the lib in the common crate it seems to attempt linking it before it has been built (or at least confirm it's existence).

It raises a question though - because I don't specify the link in the common crate perhaps just the drop implementation is linked before the lib is built - because I'm expecting it to just find loaded symbols it shows as a symbol missing instead of the lib missing.

Just to clarify why I'm going this weird way around. The expectation is that the user will provide the lib to build that they have generated from the other tool. I could just take a copy and build it in the common crate but then I need a version of the common crate to match the version of the external tool which is what I hope to avoid by having the user provide the version they have. There is an assumption of no breaking changes of course.

I am super open to other methods that allow for this workflow

is your common crate built as cdylib (i.e. generate a .so or .dll artifact)? or are you using some custom building processes that are outside the scope of cargo.

for normal lib (or rlib) crate type, the crate should be built as a static library (with rust specific metadata), so you can declare an external native link dependency just fine, cargo just record the dependency as metadata so when the crate is consumed by another crate, the final executable can be built successfully.

No just a standard lib type

the link is 404, is it private?

this in theory should work. for example, the following minimal example expect the downstream crate to provide a library named "device", with two symbols of C functions, it builds successfully on my machine

// src/lib.rs
extern "C" {
    fn open_device() -> usize;
    fn close_device(handle: usize);
}
pub struct Driver(usize);
impl Driver {
    pub fn new() -> Self { unsafe { Driver(open_device()) } }
}
impl Drop for Driver {
    fn drop(&mut self) { unsafe { close_device(self.0) } }
}
// build.rs
fn main() {
    println!("cargo:rustc-link-lib=device");
}

and here's a minimal app crate:

# Cargo.toml
# ...
[dependencies]
driver = { path = "../driver" }
[build-dependencies]
cc = "*"
// device.cpp
#include <cstddef>
extern "C" {
    size_t open_device() { return 0; }
    void close_device(size_t) {}
}
// src/main.rs
fn main() {
    let _ = driver::Driver::new();
}
// build.rs
fn main() {
    cc::Build::new().file("device.cpp").compile("device");
}

Arg sorry, must have not got through all of the dialogs. It is public now.

Thanks for the minimal example. Good to know the principle is not mad - just to work out what I'm missing.

When I tried the build.rs in the driver crate equivalent though it was throwing errors. I'll go back to try that again and see if I can work out what is different.

now I see the problem, it's quite a tricky situation.

first of all, your common crate is used both at runtime and at compile time, so the crate is actually built twice, once for the build script as build-dependencies, and the other for the actual app crate as dependencies.

this is in itself is totally fine, yet your crate has an external link dependencies that is produced by the build script, you came into this chicken and egg problem.

interestingly, from my quick glimpse of the code, it seems that the build time code and the runtime code are independent of each other, as the build submodule seems self contained and doesn't import other submodules, and the external link dependency is only used by the runtime part. so in principle, it should have worked.

however, and here is the tricky part, your library crate uses a static variable:

this prevents the linker from discarding the runtime components when building the build script, and it pulls in the required external FFI functions, which resulted the "unresolved reference" linker error. this also explains why it only happens when you referenced the functions in the lazy initialization routine or destructor.

so, the solution is simple and straightforward, you either:
a) split the crate into two, one for the build time components, one for the runtime components; or
b) keep them in the same crate, but gated with feature flags;

you can try to change your API and remove the the lazy initialization code by getting rid of the static variable, but I'm not sure whether you can get rid of the Drop implementation, as the drop glue is specially handled by the compiler and I don't know much of the details.

1 Like

This is awesome thank you!

I'll have a play to confirm but as you say, the build time part should be possible to separate to another crate without any issue

I had a quick go with feature flags and it didn't work so I just split the crate as there isn't really a problem with that here. Works now - thank you for your persistence!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.