Binary vs library size difference

help
1

Hello! I want to create a shared library out of a Rust crate, one that I can load/dlopen() later in one my C++ projects (practically a plugin). To do that, I am using the cdylib crate-type.

[lib]
crate-type = ["cdylib"]

Creating the library works properly. The part where I'm having problems with is the library size which is bigger than I expected. Previously to converting the crate to a library, I used it as a binary, which had a smaller size that I was OK with. I have the following reduced example:

Cargo.toml

[lib]
crate-type = ["lib", "cdylib"]

[profile.release]
codegen-units = 1
lto = true
opt-level = 'z'
panic = 'abort'

src/main.rs

fn main() {
    foo::do_something();
}

src/lib.rs

#[no_mangle]
extern "C" fn the_symbol() {
    do_something()
}

pub fn do_something() {
    println!("hello");
}

With this example, cargo builds both a binary and a library out of the same code.

Binary size

$ strip -s target/release/foo && ls -lh target/release/foo && ldd target/release/foo
-rwxr-xr-x. 2 master master 231K Sep  9 11:09 target/release/foo
	linux-vdso.so.1 (0x00007ffc7cd90000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f10f322e000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f10f3213000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f10f3049000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f10f32b7000)

Library size

$ strip -s target/release/libfoo.so && ls -lh target/release/libfoo.so && ldd target/release/libfoo.so
-rwxr-xr-x. 2 master master 263K Sep  9 11:10 target/release/libfoo.so
	linux-vdso.so.1 (0x00007ffc23fc0000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fc977d04000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc977ce2000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc977cc7000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fc977afd000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fc977d7a000)

As you can see, there is a difference of 32K between the binary and library. In the project I am working on, the difference is bigger:

Stripped binary and library

$ ls -lh target/release/libbar.so target/release/bar
-rwxr-xr-x. 2 master master 439K Sep  9 11:13 target/release/bar
-rwxr-xr-x. 2 master master 559K Sep  9 11:13 target/release/libbar.so

And when I'm also using Xargo, the difference is even more noticeable:

Xargo.toml

[dependencies]
std = {default-features=false, features=["panic_immediate_abort"]}

Stripped binary and library

$ ls -lh target/x86_64-unknown-linux-gnu/release/libbdsetter.so target/x86_64-unknown-linux-gnu/release/bdsetter
-rwxr-xr-x. 2 master master 163K Sep  9 11:18 target/x86_64-unknown-linux-gnu/release/bdsetter
-rwxr-xr-x. 2 master master 299K Sep  9 11:18 target/x86_64-unknown-linux-gnu/release/libbdsetter.so

Where is this difference coming from? Is it possible to reduce the library size to match the one of the binary? Might be related to https://github.com/rust-lang/rust/issues/37530, but I'm still not sure if there's any solution for my problem.

2

You could set debug = true in your [profile] section, then compare symbol tables to see what's excluded in the binary.

3

Even without debug = true symbol tables are available.

1 Like
4

These are the symbol tables, in case someone has an idea:

There are more symbols for libfoo, but I can't tell much else by looking.

5

Without knowing your exact use case I assume that in a binary the compiler optimizes each peace of unused code away whereas the compiler is forced to keep each peace of public API in the library because someone might want to link against it.

I think it might even be possible that Rust (or LLVM) further analyzes your main() and strips unreachable paths from your code.

6 Likes
6

Here's a cleaned up diff: --- bin 2020-09-09 16:09:45.409165475 +0200+++ lib 2020-09-09 16:09:37.8771650 - Pastebin.com

1 Like
7

A wild guess-- can you make sure rustc is executed using the same parameters in both bin and lib cases? Just increase verbosity to make sure you're really using the same profile.

1 Like
8

Nothing looks wrong here:

cargo build -v --release
   Compiling foo v0.1.0 (/home/master/github/foo)
     Running `/home/master/.cargo/bin/sccache rustc --crate-name foo --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --crate-type cdylib --emit=dep-info,link -C opt-level=z -C panic=abort -C codegen-units=1 -C metadata=57b6cdda4d28c9c9 --out-dir /home/master/github/foo/target/release/deps -L dependency=/home/master/github/foo/target/release/deps`
     Running `/home/master/.cargo/bin/sccache rustc --crate-name foo --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=z -C panic=abort -C lto -C codegen-units=1 -C metadata=041d6d6a81f45a2f -C extra-filename=-041d6d6a81f45a2f --out-dir /home/master/github/foo/target/release/deps -L dependency=/home/master/github/foo/target/release/deps --extern foo=/home/master/github/foo/target/release/deps/libfoo.rlib`
9

lib.rs isn't built with -Clto unlike main.rs.

1 Like
10

binary can keep only library methods that are actually used in this specific binary, and throw out everything else.

library must keep all public methods.

1 Like
11

As usual, I recommend using cargo-bloat and cargo-asm for analyzing binary size.

Combining the JSON output from cargo-bloat --message-format json with jq gives you extremely powerful options for analysis and comparisons directly in the terminal.

1 Like
12

Ah, indeed, I missed that. After you mentioned it, I had one more look and I noticed a issue that describes the problem I'm hitting:
https://github.com/rust-lang/rust/issues/51009

If I remove crate-type lib from Cargo.toml, lib.rs is now built with -C lto:

/home/master/.cargo/bin/sccache rustc --crate-name foo --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type cdylib --emit=dep-info,link -C opt-level=z -C panic=abort -C lto -C codegen-units=1 -C metadata=57b6cdda4d28c9c9 --out-dir /home/master/github/foo/target/release/deps -L dependency=/home/master/github/foo/target/release/deps

After stripping libfoo.so, size is now 223K, even less than the 231K for the binary in the original post.

-rwxr-xr-x. 2 master master 223K Sep 10 10:57 target/release/libfoo.so

For my project, with cargo (binary had 439K):

$ ls -lh target/release/libbar.so   
-rwxr-xr-x. 2 master master 435K Sep 10 10:51 target/release/libbar.so

With xargo (binary had 163K):

ls -lh target/x86_64-unknown-linux-gnu/release/libbar.so             
-rwxr-xr-x. 2 master master 155K Sep 10 10:07 target/x86_64-unknown-linux-gnu/release/libbar.so

Thanks for your help everyone, I'm glad there was a solution for the problem.

3 Likes
13

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.