Cargo - generating --extern flags for rustc

Hi there,

I have been doing some research on how better to integrate cargo projects with beardbolt - a godbolt/compiler-explorer like tool for Emacs.

The default example that comes with beardbolt (and its predecessor RMSBolt) is only really suitable for rust code that doesn't include any dependencies. I however would like to work with cargo dependencies.

I've been able to get it working with a dependency by manually generating a rustc command for beardbolt to use (edited for legibility):

rustc --crate-name example --edition=2024 --crate-type bin -C opt-level=0 \
  -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps \
  --extern ndarray=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps/libndarray-dd3e9807fe477367.rlib

beardbolt will then generate the following command (edited for legibility):

rustc --crate-name example --edition=2024 --crate-type bin -C opt-level=0 \
  -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps \
  --extern ndarray=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps/libndarray-dd3e9807fe477367.rlib \
  -C debuginfo=1 --emit link /tmp/beardbolt-dump-ErjehI.rs \
  -o /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \
&& objdump -d /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \
   --insn-width=16 -l -M att \
> /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o.disass

I want to be able to generate the --extern flag dynamically - but I can't see a way of doing this.

  • I checked the cargo metadata command, but it doesn't seem to contain these output directories
  • I don't seem to be able to do a dry run of cargo build
  • The checksum in Cargo.lock doesn't match the hash used in the rlib path
  • When multiple builds happen, you can have multiple rlibs, and I don't know how to figure out which one is the correct one

I'd really appreciate it if someone could point me in the right direction, thank you!

JH

Instead of using --extern command line directive, you can just add extern crate crate_name; in your .rs file. I prefer this way, but I do not use Cargo at all. I also have a LinkedIn article regarding using rustc without Cargo.

1 Like

I just tried that, and it doesn't seem to work (unless I'm doing something silly :slight_smile:)

// NB: I based this from looking at
// https://doc.rust-lang.org/edition-guide/rust-2018/path-changes.html#more-details
extern crate ndarray;
use ndarray::array;
fn is_rms(a: char) -> i32 {
    match a {
        'R' => 1,
        'M' => 2,
        'S' => 3,
        _ => 0,
    }
}

fn main() {
    let a: u8 = 1 + 1;
    if is_rms(a as char) != 0 {
        println!("{}", a);
    };

    is_rms('a');

    let a = array![1, 2, 3, 4];
    let b = array![8, 7, 6, 5];

    let c = vec![1, 2, 3, 4];
    let d = vec![2, 3, 4, 5];

    dbg!(a.dot(&b));
}
rustc --crate-name example --edition=2024 --crate-type bin -C opt-level=0 -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps -C opt-level=0 -C debuginfo=1 --emit link /tmp/beardbolt-dump-XuRiAs.rs -o /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \
&& objdump -d /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o --insn-width=16 -l -M att > /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o.disass


error[E0463]: can't find crate for `ndarray`
 --> /tmp/beardbolt-dump-XuRiAs.rs:1:1
  |
1 | extern crate ndarray;
  | ^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0463`.
Compilation exited abnormally with code 1

It seems that extern crate doesn't solve the problem for me?

you can use cargo build --build-plan, but this option is unstable, so you need a nightly cargo:

You need to specify '-L' command line option with the location of where all your .rlib files reside. Like:

-L all=some_dir

But where such directory is, perhaps Cargo knows. I specify the directory myself.

1 Like

I actually did that in the example in my last reply, but it didn't seem to make a difference!

I am using cargo 1.94.0-nightly (b54051b15 2025-12-30), but it seems like build-plan has now been removed, so that doesn't seem like a feasible option anymore unfortunately.

However, that hint led me to this issue, which led me to cargo-outdir, which in turn led me to cargo build --message-format=json. From here I can find the .rlib paths! However they are only printed if they are built, so it doesn't fully solve the problem for me unfortunately. I was hoping that cargo check --message-format=json would include them, but it only seems to for one of the dependencies in the tree, autocfg.

Because .rlib files need to be created before you reference them. Cargo builds them automatically accordingly the dependencies tree. But, you may need to do that manually.

1 Like

That doesn't seem to be right:

with the following directory structure:

❯ tree .
.
├── Cargo.lock
├── Cargo.toml
├── example.rs
├── flake.lock
├── flake.nix
├── rustfmt.toml
└── rust-toolchain.toml

I run cargo b, then I get the .rlib file:

❯ tree . 
.
├── Cargo.lock
├── Cargo.toml
├── example.rs
├── flake.lock
├── flake.nix
├── rustfmt.toml
├── rust-toolchain.toml
└── target
    ├── CACHEDIR.TAG
    └── debug
        ├── build
        │   ├── matrixmultiply-9574377cd328d2a3
        │   │   ├── invoked.timestamp
        │   │   ├── out
        │   │   ├── output
        │   │   ├── root-output
        │   │   └── stderr
        │   ├── matrixmultiply-a26a70b8a9f6cca5
        │   │   ├── build-script-build
        │   │   ├── build_script_build-a26a70b8a9f6cca5
        │   │   └── build_script_build-a26a70b8a9f6cca5.d
        │   ├── num-traits-20a5f138287d8cd4
        │   │   ├── build-script-build
        │   │   ├── build_script_build-20a5f138287d8cd4
        │   │   └── build_script_build-20a5f138287d8cd4.d
        │   └── num-traits-b2c60e1228872d4c
        │       ├── invoked.timestamp
        │       ├── out
        │       ├── output
        │       ├── root-output
        │       └── stderr
        ├── deps
        │   ├── autocfg-3b49e0afa777d415.d
        │   ├── example-9fbeb0ccfc303efd
        │   ├── example-9fbeb0ccfc303efd.d
        │   ├── libautocfg-3b49e0afa777d415.rlib
        │   ├── libautocfg-3b49e0afa777d415.rmeta
        │   ├── libmatrixmultiply-4f656cfa2cfe6aad.rlib
        │   ├── libmatrixmultiply-4f656cfa2cfe6aad.rmeta
        │   ├── libndarray-dd3e9807fe477367.rlib                       <-- I have the rlib file now
        │   ├── libndarray-dd3e9807fe477367.rmeta
        │   ├── libnum_complex-a61a6d232334fa79.rlib
        │   ├── libnum_complex-a61a6d232334fa79.rmeta
        │   ├── libnum_integer-0abbc409bfc4a755.rlib
        │   ├── libnum_integer-0abbc409bfc4a755.rmeta
        │   ├── libnum_traits-24d676e28b4809e9.rlib
        │   ├── libnum_traits-24d676e28b4809e9.rmeta
        │   ├── librawpointer-4cd601dc393479b1.rlib
        │   ├── librawpointer-4cd601dc393479b1.rmeta
        │   ├── matrixmultiply-4f656cfa2cfe6aad.d
        │   ├── ndarray-dd3e9807fe477367.d
        │   ├── num_complex-a61a6d232334fa79.d
        │   ├── num_integer-0abbc409bfc4a755.d
        │   ├── num_traits-24d676e28b4809e9.d
        │   └── rawpointer-4cd601dc393479b1.d
        ├── example
        ├── example.d
        ├── examples
        └── incremental
            └── example-3b7raf6z6t1ms
                ├── s-hepkj98x8h-0v1t6l0-4dl9nvpyh6534wyb4a32bqee2
<snip>
                └── s-hepkj98x8h-0v1t6l0.lock

15 directories, 103 files

I then run:

rustc --crate-name example --edition=2024 --crate-type bin -C opt-level=0 \
  -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps \
  -C opt-level=0 -C debuginfo=1 --emit link /tmp/beardbolt-dump-71IBIj.rs -o /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \ \
&& objdump -d /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \
  --insn-width=16 -l -M att > /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o.disass

With the following rust file:

extern crate ndarray;
use ndarray::array;

fn main() {
    let a = array![1, 2, 3, 4];
    let b = array![8, 7, 6, 5];

    let c = vec![1, 2, 3, 4];
    let d = vec![2, 3, 4, 5];

    dbg!(a.dot(&b));
}

And I get:

error[E0463]: can't find crate for `ndarray`
 --> /tmp/beardbolt-dump-DrU97k.rs:1:1
  |
1 | extern crate ndarray;
  | ^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0463`.
Compilation exited abnormally with code 1

It seems I'd need to do something like this to get what I need (I have no idea how naive this is!):


    1. Find the direct dependencies of the package: `cargo tree --depth 1 --prefix=none | tail -n+2`
    2. Find the the rlibs of all the matching deps in `cargo b --message-format=json` (where .reason == "compiler-artifact", .target.name == "ndarray", with the value being .filenames | /.rlib$/ ):

      {
        "reason": "compiler-artifact",
        "package_id": "registry+https://github.com/rust-lang/crates.io-index#ndarray@0.17.1",
        "manifest_path": "/home/john/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ndarray-0.17.1/Cargo.toml",
        "target": {
          "kind": [
            "lib"
          ],
          "crate_types": [
            "lib"
          ],
          "name": "ndarray",
          "src_path": "/home/john/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ndarray-0.17.1/src/lib.rs",
          "edition": "2021",
          "doc": true,
          "doctest": true,
          "test": true
        },
        "profile": {
          "opt_level": "0",
          "debuginfo": 2,
          "debug_assertions": true,
          "overflow_checks": true,
          "test": false
        },
        "features": [
          "default",
          "std"
        ],
        "filenames": [
          "/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps/libndarray-dd3e9807fe477367.rlib",
          "/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps/libndarray-dd3e9807fe477367.rmeta"
        ],
        "executable": null,
        "fresh": true
      }

    3. Convert each match into an `--extern` string

The problem is that Cargo hides libndarray.rlib under name libndarray-dd3e9807fe477367.rlib, you can try to reference it as extern crate ndarray_dd3e9807fe477367 as ndarray;, however I am not sure that it will work due using 'dash'.

1 Like

That sounds right to me, but I wasn't able to get that to load either!

❯ ls target/debug/deps | grep ndarray
libndarray-382e4fc797d1bf82.rmeta
libndarray-dd3e9807fe477367.rlib       # This is the lib I want to include
libndarray-dd3e9807fe477367.rmeta
ndarray-382e4fc797d1bf82.d
ndarray-dd3e9807fe477367.d
// I tried with lib prefixed as well - like in the rlib filename - 
// with the same error returning
extern crate ndarray_dd3e9807fe477367 as ndarray;

fn main() {
    let a = array![1, 2, 3, 4];
    let b = array![8, 7, 6, 5];
    let res = a.dot(&b);

    dbg!(res);
}
rustc --crate-name example --edition=2024 --crate-type bin -C opt-level=0 -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps -C debuginfo=1 --emit link /tmp/beardbolt-dump-qtrAGX.rs -o /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o \
&& objdump -d /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o --insn-width=16 -l -M att > /home/john/.local/state/emacs/beardbolt-sandbox/beardbolt.o.disass

Produces

error[E0463]: can't find crate for `ndarray_dd3e9807fe477367`
 --> /tmp/beardbolt-dump-qtrAGX.rs:1:1
  |
1 | extern crate ndarray_dd3e9807fe477367 as ndarray;
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

In context: -L dependency=/home/john/.local/state/emacs/beardbolt-sandbox/target/debug/deps try to replace dependency by all.

No luck, unfortunately - exact same issue. I've pushed the code up to an unlisted git repository ~johnhamelink/cargo-rustc-disassembly-example - sourcehut git which demonstrates the problem - maybe there's something obvious I'm missing?

You can run the compilation steps I'm running in the Makefile. I included the nix flake I'm using for my development environment.

Thanks for your help so far!

Okay, indeed, it looks like extern crate in the source doesn't work (perhaps it's the 'dash' issue, as I pointed earlier). However, CLI --extern directive works perfectly. Here is your source:

use ndarray::array;

fn main() {
    let a = array![1, 2, 3, 4];
    let b = array![8, 7, 6, 5];
    let res = a.dot(&b);

    dbg!(res);
}

And here is a no Cargo build script:

project  =example
main=${project}
common =..${~/~}..${~/~}simscript${~/~}comm-build.7b:file
crate_dir=./target/debug/deps
comp opts=[--extern, ndarray=./target/debug/deps/libndarray-eba4934cbedcdb29.rlib]

include(common);

Finally, you can use the document I created some time ago to understand as the linking mechanism works.

Note, that I use my own build script, however it should be easy to port it in Make you are using.

Rust is reading crates' metadata, and will distinguish them by their internal hash. Cargo adds extra-filename which IIRC affects that.

Cargo's file naming scheme and build dir layout are private implementation details, and mixing them with DIY builds will not work well.

You either should use rustc yourself for everything and build every dependency from scratch, building Rust like you would build a C project, or build via cargo commands without invoking rustc directly on Cargo's temp files.

1 Like

That will not work. The crate name encoded in the crate metadata is still ndarray. Which means that if you use extern crate ndarray_dd3e9807fe477367, rustc will reject it as being the wrong crate. In any case rustc will already look for anything starting with libndarray and ending with .rlib in the crate search path if you use extern crate ndarray.

Read my another post in this thread. It works, regardless your good note.

Despite your very clever recommendation, rustc doesn't care how you build 'rlib' file, and how you name it either. rustc only cares that the file name starts with 'lib' and has extension 'rlib'. I explained rustc behavior in my article.

This is more complicated, because rlib contains metadata identifying its own dependencies. You don't need to specify --extern for everything your program uses, only for the few top-level crates that your crate can refer to directly, and then rustc will use metadata to discover the rest. There's a non-trivial matching algorithm there:

You are absolutely correct. It's because I just mention,

and then specified just one extern directive,

Voilà, rustc compiles the file without problems and Cargo. It is why, the hybrid approach works perfectly. Thank you for confirming that.