Cargo dependency hell

Can we do this for real? e.g. send an automated "welcome email" when a new author publishes a package, with suggestions like this?

edit: if you want to add backup owners, see this: Bus Factor +1 for crates (now dead)

8 Likes

I'm 100% a complete noob, so I'm not going to know what I'm talking about here.... Is this something workspaces can help? I was listening to the New Rustacean podcast last night (an older episode), and he went over workspaces briefly. It sounded like a way to have individual projects all within an umbrella object.

A minor correction: We don't edit dependencies in place for Firefox. We use cargo-vendor so our automated builds don't depend on crates.io or GitHub. But changes to dependencies are always done by re-vendoring from the original sources.

(Individual developers might edit-in-place on their own workstations temporarily, during development, but such changes aren't checked in to the Firefox repository directly: they get published to crates.io first. And our tools don't currently support edit-in-place workflows very well.)

6 Likes

Just use Eco: https://github.com/pistondevelopers/eco

It spits out a list of tasks to integrate your library ecosystem.

Npm actually does not suffer from this, see How npm2 Works | How npm Works

It's more complicated in newer npm versions but the result is still the same, no dependency hell. To my knowledge, npm is unique in this. If you know about any other platform I'd love to know.

Yeah, that's the only possible solution. I don't know how easy/safe it would be for rust to allow linking modules against different versions of the same crate, but I imagine that it can lead to a plethora of other problems.

Ever wondered why so many binary only libraries come with their own version of malloc and free? Just imagine that you and one of your dependencies use different versions of the dynamic memory manager. Or in fact any resource manager for a shared resource in you program. You are steering fast into the realm in very undefined behavior. So it's best to catch this during the build to avaid any possible problems.

Given how long dependency hell exists and how many efforts to solve it failed, we as developers and maintainers just have to live with it. It's the price you pay for using new modules or modules which are not well established and maintained properly.

That was not meant to be a complaint, I think rust is great at what it tries to achieve and I think this is reasonable tradeoff. Node is not so much focused on performance/memory because it can't be fast anyway :slight_smile:

BTW: I am really curious if there is any other platform without dephell, it's unlikely that node is alone...

How is that different from Cargo's behavior?

1 Like

Node downloads (and loads) dependencies in multiple versions. So there are no conflict, in cost of memory

Cargo will also download and load dependencies in multiple versions.

6 Likes

Oh, that's great then! Is it anywhere explained in more detail? Does it work out of the box or do I have to configure it somehow? I couldn't google anything useful. Rust is still new for me and I don't know where to look yet.

You can always do in your Cargo.toml

[replace]
bindgen:0.8.1 = { path = "my_bindgen" }

And replace any crate what you want with your fork, that
have fixed dependicies.

What else can be done, if some person stop works on crate?

3 Likes

That's the default behavior, so it works out-of-the-box.

For semver-major incompatible versoins it's exactly the same as Node. If dependencies end up needing two conflicting versions of a library, Cargo will use two versions of the library and make sure each crate gets the version it wants. It's 100% automatic and reliable for Rust crates (with some caveats if you want to mix types/traits between crates).

The only hell here is C, which just can't do that. For C dependencies Cargo has links = "C library name" attribute that intentionally raises an error if there's more than one version of the same C library in one executable. So technically it's not Rust/Cargo's fault, but e.g. OpenSSL's problem that it can't coexist with another OpenSSL version.

15 Likes

This is really great news! And thanks for details.

Sounds workable to me... if the fork is no longer needed, just say so in the README, and remove from the registry.

but it's not possible to remove anything from the registry (apart from asking personally for a manual intervention). crates.io only supports yanking of individual versions, but even yanking all versions, which is a laborious process, doesn't quite work yet.

in case it helps: https://dependabot.com/ has support for Rust now. I've used it on some of my repos and it seems to work quite well :slight_smile:

4 Likes

Once a crate is published to crates.io the author should receive a notification any time a crate is published with their work as a dependence. For instance, I publish MyCrate and it is awesome. Game changing, really. So now someone publishes TheirCrate which depends on MyCrate. I should be notified that this dependency relationship exists. When I update MyCrate, even for minor versions but definitely for major versions, the author of TheirCrate should be notified of the change automatically. No details, just a message "The dependency MyCrate for your package TheirCrate has been updated. See MyCrate docs for details that may affect TheirCrate."

This won't prevent breakage and dependency hell, but it will at minimum notify maintainers of the need to modify their code or fork the dependency for their needs.

8 Likes