Why would binaries for Rust crate would bloat even the dependencies are not used?

Repo for reproduce: GitHub - winston0410/serverless-rust-repo

I tried to build two binaries in apps/example/Cargo.toml, each binary will be used for a serverless handler:

[[bin]]
name = "v1-hello-get"
path = "src/v1/hello/get.rs"

[[bin]]
name = "v1-bye-get"
path = "src/v1/bye/get.rs"

Build with the follow command:

cargo +nightly build -Z build-std-features=panic_immediate_abort -Z build-std=std,panic_abort --release --target aarch64-unknown-linux-gnu

Check the size of ./target/aarch64-unknown-linux-gnu/release/v1-hello-get and ./target/aarch64-unknown-linux-gnu/release/v1-bye-get

ls -lh ./target/aarch64-unknown-linux-gnu/release/v1-hello-get
ls -lh ./target/aarch64-unknown-linux-gnu/release/v1-bye-get
-rwxr-xr-x  1 xxx  staff   697K May 25 10:06 ./target/aarch64-unknown-linux-gnu/release/v1-hello-get
-rwxr-xr-x  1 xxx  staff   697K May 25 10:06 ./target/aarch64-unknown-linux-gnu/release/v1-bye-get

After that uncomment the commented code in apps/example/Cargo.toml and apps/example/src/v1/hello/get.rs, and build again

# aws-sdk-dynamodb = "0.12.0"
# aws-config = "0.12.0"
    // apps/example/src/v1/hello/get.rs
    // let region_provider = RegionProviderChain::default_provider().or_else("eu-west-2");
    // let config = aws_config::from_env().region(region_provider).load().await;
    // let client = aws_sdk_dynamodb::Client::new(&config);

Even though we haven't done anything to apps/example/src/v1/bye/get.rs, the size of the binary increased.

-rwxr-xr-x  1 xxx  staff   3.1M May 25 10:18 ./target/aarch64-unknown-linux-gnu/release/v1-hello-get
-rwxr-xr-x  1 xxx  staff   1.2M May 25 10:18 ./target/aarch64-unknown-linux-gnu/release/v1-bye-get

Which is undesirable for a serverless environment, each handler should be independent. How can I fix this?

If you don't actually use a dependency rustc won't even load the crate, let alone link to it. Are you sure you aren't using the dependencies?

1 Like

Sorry I don't quite understand why you mean there. My situation is I am building two binaries, and only one has used some extra dependencies, but both the size of both binaries has increased.

Enabling lto=true in [profile.release] should fix it.

I have done that as well initially, that is the optimization config I have:

[profile.release]
strip = true
lto = true
codegen-units = 1
opt-level = "z"
panic = "abort"

Ooof. Then something has to be referencing it.

Have you tried cargo bloat --release? It will list which symbols take space, and maybe that will be a hint what is getting pulled in.

2 Likes

Hmm after running cargo bloat --release I got this:

base ❯ cargo bloat --release --bin v1-bye-get      
    Finished release [optimized] target(s) in 0.06s
    Analyzing target/release/v1-bye-get

 File  .text      Size     Crate Name
71.7% 100.4% 1022.5KiB [Unknown] __mh_execute_header
 0.0%   0.0%        0B           And 0 smaller methods. Use -n N to show more.
71.4% 100.0% 1018.1KiB           .text section size, the file size is 1.4MiB

Hmm I have no idea what can I try next

You may need to re-enable debug symbols and disable stripping for cargo-bloat to work.

Oh right that's a silly mistake.

Before having unused dependencies:

base ❯ cargo bloat --release --bin v1-bye-get -n 10
    Finished release [optimized] target(s) in 0.06s
    Analyzing target/release/v1-bye-get

 File  .text     Size          Crate Name
 1.6%   5.1%  36.9KiB lambda_runtime lambda_runtime::Runtime<C>::run::{{closure}}
 0.8%   2.6%  18.8KiB            std addr2line::ResDwarf<R>::parse
 0.6%   1.8%  13.3KiB      [Unknown] __mh_execute_header
 0.5%   1.7%  12.1KiB           http http::header::name::StandardHeader::from_bytes
 0.5%   1.5%  10.9KiB          hyper <hyper::proto::h1::role::Client as hyper::proto::h1::Http1Tran...
 0.5%   1.5%  10.7KiB            std std::sys_common::backtrace::_print_fmt::{{closure}}
 0.4%   1.3%   9.2KiB          hyper hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_write
 0.4%   1.2%   8.8KiB            std addr2line::Context<R>::find_frames
 0.4%   1.1%   8.2KiB          hyper hyper::client::client::Client<C,B>::retryably_send_request::{{...
 0.3%   1.0%   7.6KiB            std addr2line::ResUnit<R>::parse_lines
26.2%  84.1% 612.6KiB                And 5965 smaller methods. Use -n N to show more.
31.2% 100.0% 728.8KiB                .text section size, the file size is 2.3MiB

After having unused depedencies:

base ❯ cargo bloat --release --bin v1-bye-get -n 10
    Finished release [optimized] target(s) in 0.10s
    Analyzing target/release/v1-bye-get

 File  .text      Size          Crate Name
 1.1%   3.6%   36.9KiB lambda_runtime lambda_runtime::Runtime<C>::run::{{closure}}
 1.0%   3.1%   31.9KiB             h2 h2::proto::connection::DynConnection<B>::recv_frame
 0.6%   1.9%   19.7KiB          hyper hyper::client::client::Client<C,B>::connect_to::{{closure}}::...
 0.6%   1.8%   18.8KiB            std addr2line::ResDwarf<R>::parse
 0.5%   1.7%   17.5KiB             h2 h2::codec::framed_read::decode_frame
 0.5%   1.5%   15.3KiB             h2 h2::proto::connection::Connection<T,P,B>::poll
 0.4%   1.4%   14.1KiB             h2 h2::proto::connection::Connection<T,P,B>::poll2
 0.4%   1.2%   12.1KiB           http http::header::name::StandardHeader::from_bytes
 0.3%   1.1%   10.8KiB         hyper? <hyper::proto::h2::client::ClientTask<B> as core::future::fut...
 0.3%   1.0%   10.7KiB            std std::sys_common::backtrace::_print_fmt::{{closure}}
25.6%  82.8%  843.0KiB                And 8219 smaller methods. Use -n N to show more.
30.9% 100.0% 1018.6KiB                .text section size, the file size is 3.2MiB

So I guess the bloat are introduced by h2. But again I haven't used that dependencies in the v1-bye-get binary.

I wonder if maybe h2 has some global variable or exports a public symbol that makes it look used to the linker.

I would look for additional features activated in some the dependencies you DO use (especially some http2 feature, considering your cargo-bloat output), by those two crates you added or some dependencies of them. Also remember to check their dev-dependencies if they don't have resolver = "2" enabled, as features from dev-dependencies were unified with runtime dependencies in the v1 resolver.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.