Aren't there any efforts to bring Rust's dependency number down?

Doing any simple thing in Rust, like getting a random number, already brings 7 dependencies. These are 7 possible crates that could be hijacked to have malicious code inserted into it. Also, 7 dependencies that I'd have to audit (let's be honest, no one audits these crates).

A simple Rust project will have more than 100 dependencies, and an app will likely have 1000 dependencies.

Rust's safety gets weakened by the necessity of having so much dependencies.

Will this ever be like this?

11 Likes

It depends. ripgrep is not simple and it has fewer than 100 dependencies. Many of those dependencies are crates that I wrote by splitting code out of ripgrep so that others could use it. Are you suggesting that I shouldn't have done that just to reduce a count?

Similarly, are you suggesting that regex shouldn't have split some of its code out into regex-syntax, memchr and aho-corasick? What about all the folks using those crates directly?

What I'm trying to say is that it would help for you to be more specific about the actual problem you're trying to solve.

44 Likes

From what I understand, one of the Rust design principles is to prefer smaller crates. This is consistent with the mantra “only pay for what you use”. A side-effect of a more tailored dependency tree is likely what you are describing; more dependencies.

11 Likes

if all the crates are on the same repo and are written by the same person it's as if it were one.

This is not the problem. The problem is that, to get a random number, we must rely on 7 crates by 7 different people. If just one puts a malware, suddenly everyone will download that. And no, no one audits crates.

2 Likes

To be fair, the rustc itself is relied on the rand crate. You can't avoid to be affected by those 7 people if you're using the official rust toolchain.

4 Likes

it's just an example. Anything basic in Rust relies on 5 to 10 crates by unknown people who could easily install malware. The BitPay crypto wallet, which relies on NPM, already had a vulnerability shipped to the Play Store because of this very problem. No one audits dependencies.

1 Like

And no, people audits crates.

https://github.com/RustSec/cargo-audit

13 Likes

I agree that trust is a concern with modern dependency ecosystems like crates.io. I think you'd be interested in crev. See also this blog post about it.

12 Likes

Using third-party crates makes things faster and easier, but is by no means required. I spent my first year with Rust using nothing but the standard library and the stable compiler, yet still managed to write plenty of useful programs.

15 Likes

...which you can just vendor into your own project and either check them for correctness or even rewrite.

1 Like

Actually, all the default dependencies of rand are maintained by either the rust-random team or the Rust project itself, except for one (ppv-lite86, maintained by kazcw). So although there are seven crates involved, there's only one additional author to trust.

8 Likes

That's not the case for ripgrep. Many of them are in different repositories.

Show me. What is your Cargo.toml? What are the 7 dependencies? Who are the 7 different authors?

2 Likes

For

[package]
name = "rand_dep_test"
version = "0.1.0"
edition = "2018"

[dependencies]
rand = "0.8.3"

I get the following lock file:

# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "cfg-if"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"

[[package]]
name = "getrandom"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9495705279e7140bf035dde1f6e750c162df8b625267cd52cc44e0b156732c8"
dependencies = [
 "cfg-if",
 "libc",
 "wasi",
]

[[package]]
name = "libc"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56d855069fafbb9b344c0f962150cd2c1187975cb1c22c1522c240d8c4986714"

[[package]]
name = "ppv-lite86"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac74c624d6b2d21f425f752262f42188365d7b8ff1aff74c82e45136510a4857"

[[package]]
name = "rand"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ef9e7e66b4468674bfcb0c81af8b7fa0bb154fa9f28eb840da5c447baeb8d7e"
dependencies = [
 "libc",
 "rand_chacha",
 "rand_core",
 "rand_hc",
]

[[package]]
name = "rand_chacha"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e12735cf05c9e10bf21534da50a147b924d555dc7a547c42e6bb2d5b6017ae0d"
dependencies = [
 "ppv-lite86",
 "rand_core",
]

[[package]]
name = "rand_core"
version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34cf66eb183df1c5876e2dcf6b13d57340741e8dc255b48e40a26de954d06ae7"
dependencies = [
 "getrandom",
]

[[package]]
name = "rand_dep_test"
version = "0.1.0"
dependencies = [
 "rand",
]

[[package]]
name = "rand_hc"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3190ef7066a446f2e7f42e239d161e905420ccab01eb967c9eb27d21b2322a73"
dependencies = [
 "rand_core",
]

[[package]]
name = "wasi"
version = "0.10.2+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd6fbd9a79829dd1ad0cc20627bf1ed606756a7f77edff7b66b7064f9cb327c6"

The following crates are used:

  • cfg-if
  • getrandom
  • libc
  • ppv-lite86
  • rand
  • rand_chacha
  • rand_core
  • rand_hc
  • wasi

Of these getrandom and all rand* crates are maintained by the rust-random team. libc is maintained by the Rust project itself. cfg-if is maintained by @alexcrichton who is part of the Rust project and maintains a lot of important crates. (Scaling back my involvement in Rust - Rust Internals) ppv-lite86 is used for SIMD and is maintained by @kazcw. Wasi is only used when targeting wasm32-wasi and is maintained by @alexcrichton and @sunfishcode.

The only crates that could be removed are cfg-if at the cost of ergonomics. It is only 187 lines of code including comments and tests, so not much potential for malicious code. And ppv-lite86 which will be easier once we get stable portable SIMD.

To get raw random numbers from the OS/cpu, the getrandom crate is used. This crate uses libc or wasi to interface with the OS. The rand_core crate defines a common interface for pseudo random number generators (PRNG) to conform to. The rand_hc and rand_chacha crates implement two PRNG algorithms. The rand crate finally uses getrandom in combination with the PRNG algorithms to quickly produce high quality randomness by giving every thread it's own PRNG that is seeded with the OS randomness.

The getrandom and all rand* crates could in theory be merged into one, but that would cause bloat when you only need a single part. In addition it would slowdown compilation as rustc itself is not parallelized, but multiple crates can be compiled at the same time.

30 Likes

This is a non-problem.

You can write code without external dependencies. Rust isn't forcing you to depend on any external crates.

If you depend on something, then audit it and thereafter lock to the audited version (Wait! Cargo already does this!).

Combining code into larger crates doesn't decrease the surface area needed for an audit, it increases it.

Use cargo-vendor and cargo-audit as previously mentioned.

Might I suggest: instead of handwaving about a nebulous problem (too many dependencies). Suggest what you'd like to see and explain how it would work, especially in what way it would be better than the current situation.

14 Likes

Well, I just did a brief overhaul of tiny-rng. Depends only on core.

4 Likes

A practical tip for reducing the number of transitive dependencies of your own projects is to look at the crate features of your direct dependencies and disable any that you aren't using. Activating a feature often pulls in a new dependency or two, and if those crates in turn have extra dependencies the effect could be substantial.

I'd also echo @2e71828's point in a slightly different form: the typical dependency footprint of a Rust project depends substantially on what kind of project it is. The libraries I'm working on currently—dealing with parsing a specific binary format, image decoding, and data compression—don't collectively have more than 4 direct dependencies (euclid, bstr, rgb, imgref), and three of those (euclid, rgb, imgref) I'm primarily using for interoperability and to avoid reinventing the wheel, not for complex functionality that I'd be unable to implement myself. (I'm also pulling in atty and argh for my binary crates.) But if your project needs to do something async, or run a web server, or talk to the GPU, you might end up with a large dependency tree.

9 Likes

@guerlando, do you have a suggestion in mind on how to have fewer dependencies?

Maybe some level of curation would help, for this and other crate-level quality issues. The ecosystem may be a bit young for this, but the free for all does not seem very sustainable.

One possible UX is to reserve short names for curated crates, though it's possibly already too late for this :slight_smile:

I'm not fond of the idea of elevating some crates over others, even if only by name, except maybe those maintained directly by the Rust team. Unrestricted open source has a way of gravitating toward a couple successful solutions while allowing plenty of space to experiment and grow. People playing gatekeeper is best avoided.

2 Likes

That phenomenon of open source gravitating towards a couple of winners works indeed well at a relatively macro scale, e.g. gcc/clang, linux/bsd, gnome/kde, etc but these projects are themselves well tended gardens. The value-add is often the curating: Linus Torvalds is doing almost nothing else but select what gets in, having defined scope, quality criteria and entry points and passionately defending against natural entropy.

At a smaller level of granularity it's a major pain to find small quality composable components outside the monolithic behemoths. The open-door ecosystems are a mixed blessing at best. This is a pretty universal predicament: see pip or npm, browser extension stores, etc.

3 Likes