Crates.rs now shows crate sizes, but what's a crate size?

1

Crates.rs now shows size of the crate and all of its dependencies :partying_face:

rs

  • If a crate directly contains non-Rust code, there's a mini piechart with language breakdown. It uses the same colors as GitHub.

  • Byte size is based on uncompressed source code size. Tests, benches and examples are excluded. Falls back to tarball size for odd crates without source code.

  • Dependencies, ah that's a fun one!

    • Weight of dependencies is based on the full dependency tree (basically all the deps that Cargo would build). Dev deps are excluded.

    • There are many ways to count "size" — download size, compile times, bloat of the final executable. I'm most concerned about the last two, so I'm counting most of the weight from runtime deps.

    • It's an "amortized" weight. If you already have some (sub)dependency in your project, then it doesn't cost anything extra to add another crate using the same dependency. So I adjust weight of a dependency based on its popularity — if a crate is used by 75% of all crates, then I add only 25% of its weight to the overall weight. I also arbitrarily discount weight of optional and platform-specific dependencies.

I have some open questions though:

  • What's the size of libc? It compiles down to nothing, but that's still a ton of source code to compile.

  • What's the size of winapi? It's a huge crate, and a few projects include it unconditionally, so you download megabytes of Windows' object files, even on Linux.

  • Should Windows-only dependencies be included in the dependency weight? Linux-only? Even if I evaluate deps for all platforms and take an average, that boils down to adding 1/(number of platforms)th weight of Windows' deps.

  • Some crates contain files that appear to be unused junk. These files are still counted towards crate's weight, mainly because it's nearly impossible to know which files are used and which aren't. I'm not sure what do about them. Hopefully crate authors will notice excessive weight of their crates and exclude the junk?

  • How should I count weight of built-time deps towards the total weight of dependencies? How about proc-macro crates?

9 Likes
2

It's only a single megabyte compressed and it's all Rust code. The MinGW import libraries are behind target specific dependencies that are only downloaded when building for those exact targets (or you use a cargo command that forces all dependencies to be downloaded like say clippy).

That said, the Windows API headers themselves are 9MB compressed so winapi still has a lot of room to grow, and that's not even including the winrt stuff.

Also don't forget that the total size of a crate and all its dependencies doesn't reflect the reality of adding a dependency on that crate, because you may already be indirectly depending on some of its dependencies. This is particularly notable for crates like winapi which are a common dependency of almost everything.

1 Like
3

That in general is a tough one, because I don't know what target the user (website visitor) is using. For now I use a fudge factor of weight * 0.25 for all target-specific dependencies.

I could try to adjust it for platform popularity, or maybe include Windows deps only when you visit the website from Windows based web browser? :slight_smile: I'm not sure what's the best solution here.

I already try to account for that by multiplying weight of a dependency by (1 - (fraction of all crates that contain this dependency)).

4