Linking with different versions of the same C library

Continuing the discussion from Style question: match () with guard vs if else if else if else if else:

Some months ago, I decided to implement a sandbox that allows running Lua code within a Rust program. I was finally able to publish a first version of that crate (sandkiste_lua) on crates.io.

The crate uses an abstract API (sandkiste), which serves as a language-independent abstraction layer for "sandboxable" scripting languages.

Sandboxing Lua code is more difficult than it appears to be at a first glance, so I hope I covered all cases :grin:. (I wrote about some problems I was facing here.)

The Lua programming language comes in different verisons, which are (more or less) incompatible with each other. My crate links to a Lua C library, which does most of the work. To select which Lua version is to be used, a feature flag can be specified when building sandkiste_lua (see docs).

My question is: What if I want to link with two versions of the same C library at the same time? Is it possible to add a prefix to the symbols when linking? Or would it be easier to take the Lua source and modify all function names in Lua's source code (using automatic tools)? I have no experience with either of this.

My motivation is to be able to provide an upgrade path in environments where both Lua 5.3 and Lua 5.4 scripts are used. Currently, I'm forced to link my program with one Lua library only, which means I cannot support Lua 5.3 and Lua 5.4 at the same time and thus provide no gradual upgrade path for the users of my program.

I would assume that this problem (i.e. requiring different versions of the same native C library) might also arise in other contexts, such that this problem isn't limited to my particular case.

objcopy has a --prefix-symbols option for this.

2 Likes

Also make sure to not name your crate *-sys if you want to be able to link multiple versions. Cargo only allows a single *-sys version at the same time.

Oh, very good to know! I guess I could make a build.rs script which performs the necessary actions (though might be pretty platform dependent then, unless I invest a great effort).

One thing that I also need would be to make bindgen add prefixes too (as this will read the header files and not the object files). I don't think bindgen has such an option. Perhaps the easiest way would be to post-process bindgen's output?

Could you elaborate what you mean? I have seen that the manifest does not support specifying metadata indicating linking with more than one library. However, I haven't understood yet what the links of Cargo.toml really does. On most systems, the Lua library has a platform-dependent name with the version number added (to avoid collision between versions). I let the build.rs figure out the library name and emit that with:

fn main() {
    let mut include_dirs: Vec<String> = Vec::new();
    let mut lib_dirs: Vec<String> = Vec::new();
    let mut lib_names: Vec<String> = Vec::new();

    // use `pkg-config` binary to determine Lua library name and location
    {
        /* … */
    }

    /* … */

    // link with Lua
    for dir in &lib_dirs {
        println!("cargo:rustc-link-search=native={}", dir);
    }
    for name in &lib_names {
        println!("cargo:rustc-link-lib={}", name);
    }

The links key in Cargo.toml seems to be irrelevant, or does it serve any purpose during building?

Does Cargo (or crates.io) really treat crates with a -sys suffix differently? I would be surprised if it does.

Yes, it adds an implicit links based on the crate name.

You get an error when two crates define the same links value. It has no other purpose.

I can't confirm that. I created a dummy-sys crate as follows:

cargo new --lib

The Cargo.toml looked like that:

[package]
name = "dummy-sys"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

Calling cargo build succeeded.

I then added a links key with a different library name:

 [package]
 name = "dummy-sys"
 version = "0.1.0"
 edition = "2021"
+links = "yummy"

Only then (and not because I named the crate -sys), cargo build complained:

error: failed to parse manifest at `/usr/home/flocke/tmp/dummy-sys/Cargo.toml`

Caused by:
  package `dummy-sys v0.1.0 (/usr/home/jbe/dummy-sys)` specifies that it links to `yummy` but does not have a custom build script

Adding a dummy build-script with an empty fn main() {} fixed the error. There was no complaint that there is no libyummy on my system.

Only if I explicitly add two links keys to Cargo.toml, I get a problem:

 links = "yummy"
+links = "superyummy"
error: failed to parse manifest at `/usr/home/jbe/dummy-sys/Cargo.toml`

Caused by:
  could not parse input as TOML

Caused by:
  TOML parse error at line 6, column 1
    |
  6 | links = "superyummy"
    | ^
  Duplicate key `links` in table `package`

I assume that naming a package -sys in the end is only a convention without any effect on the build tools.

1 Like

Oh, then I should probably do something like:

links = "lua_with_prefixed_symbols"

Such that it will be possible to use the crate with other crates that link with Lua without prefixing the symbols (as there would be no collision if my own crate prefixes the symbols).

Indeed, the exact value isn't interpreted by cargo other than disallowing two crates with the same links value.

I think you should use a different value for each lua version, assuming that you include the lua version in the symbol prefix.

1 Like

That would mean I have to make a -sys crate for each version, because one crate may only have one links entry. That, in turn, would pollute the namespace with version-numbered versions of the same crate (e.g. myluacrate51-sys, myluacrate52-sys, myluacrate53-sys, myluacrate54-sys, myluacrate55-sys, etc).

Apart from the bloating up the registry, it would be pretty messy to handle in a repository.

That's why I would prefer something like links = "lua_with_prefixed_symbols" or maybe not using the links entry in Cargo.toml at all.

I'm not sure if the whole concept of links and -sys works well in this case.

It also enables sharing env variables with downstream crates via DEP_<links name>_<variable name>.

All env variables during building a crate X are shared with crates that are dependent on that crate X? Or what exactly do you mean?

No, only specific ones. See paragraph containing DEP_ in Build Scripts - The Cargo Book

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.