Rust should have a big standard library and here's why

@alice your argument for a big standard library is bad API which is an understandable argument but I don't think it's completely valid. You just said the standard library wants to maintain backwards compatibility even if it means keeping bad API while crates aren't really burdened with that. My takeaway from that isn't that we shouldn't have a big standard at all this is how I see it: In many cases a language can't learn from it's mistakes and correct them without breaking backwards compatiblity so is this really a good reason not to improve upon the standard library and expand it? Think long-term, what would be more beneficial, a big standard library with occasional improvements that break backwards compatiblity or a small standard library that doesn't improve upon itself to keep backwards compatiblity.

1 Like

I mean breaking backwards compatibility can cause massive splits in the ecosystem. Just look at python 2 vs python 3. I don't think it's a good method for improved adoption.

17 Likes

@alice While that's true Rust will never reach it's full potential without doing so, @Ixrec linked a discussion and what somebody said is a perfect example of why Rust would be a better language if it didn't prioritize backwards compatiblity:

Cloudflare uses Actix-web in production globally. Note that Actix-web had 8 breaking releases, giving it many chances to improve it’s public API (and it will be able to change it in the future). The maximum number of breaking releases allowed in the standard library is 0, so such large and complex codebase wouldn’t be able to gradually evolve.

If Rust focuses on backward compatibility it won't be able to grow and evolve so do we really want to handicap the language like this or do we want to improve upon it and expand it?

I always see the argument of integrating things into the standard library, but honestly, most things should not be in the standard library, but in official crates. All the net stuff and collections are a very good example of things, that should've never been put into stdlib. I mean, what does a Heap and terrible performing LinkedList have to do in a standard library, that no one ever uses? Should've just put those into officially maintained crates, instead.

11 Likes

Rust has already committed to stability. Backwards-incompatible changes wouldn't just be bad for the ecosystem on their own; they'd also be breaking a promise. Reputable organizations don't just reneg like that.

13 Likes

@notriddle There could be numerous solutions to this:

  1. Give a notice ahead of time for when changes break compatiblity before pushing it to the stable branch.

  2. Keep the old backwards compatible part of the thing that was changed under a different use statement (Maybe std::bc::path_to_old_implimentation)

  3. Do both 1 and 2.

1 Like

@SmushyTaco IMO the key reason these arguments aren't persuading us is that, in practice, there's very little distinction between:

  • a standard library with solid backwards compatibility guarantees
  • a 3rd party crate on a centralized package repository with solid persistence guarantees

Whether the old library was a std thing or a separate crate, your code can continue to use it forever without worrying about getting broken.


In fact, this is a general pattern with most objections to the "small std policy": they aren't actually affected by whether std is small or large, but are mixing that up with some other more important factor (e.g. whether the language has a decent package manager, or a centralized package repository, etc).

9 Likes

Maybe, if you make use of cargo vendor, but that's neither the default behavior nor is it promoted.

@Ixrec Not only would 3rd party crates be less widely adopted but libraries using them are usually on slightly outdated versions of the crate which result in the same dependency being used multiple times with different versions in one application. Having things integrated in the standard library would also solve that annoying problem.

The last time Rust did anything like that, it took about a year to get everyone switched. And that was a soundness fix, so it really did have to be done.

I should probably give full disclosure here: I don't actually have an opinion on whether Rust should ever have a batteries-included standard library. I don't think it's time yet, but I won't rule it out forever.

But the stability promise has pretty solid reasoning behind it. The goal is that people should be able to upgrade their compiler without waiting for all their dependencies to upgrade. If people avoid upgrading because they're afraid of breaking things, then it's likely to send the whole ecosystem into a gridlock. The go-to example is Python 3, but the fact that there are still shops stuck on C99 when there's C11 and C18* is also a good example of what breaking changes can do to you. The libraries don't upgrade because the applications don't upgrade, and the applications don't upgrade because the libraries don't.

* Two-digit dates are funny.

11 Likes

The solution to the problem you see, OP, is not a larger standard library. It is, put rather bluntly, more standard libraries.

std can stay lean and 1.0, and, say, serde can be shipped with the language distribution, and be usable without downloading it from crates-io (but, ideally, still available and compatible).

The solution isn't too stuff everything into std. It's to develop high quality libraries and "bless" them (either officially or in the community) to drive ecosystem-wide acceptance at the same level of std with separate versioning, so they can provide the breaking change semvers.

11 Likes

Moderation note: Folks, this thread started two hours ago and has already seen ~12 replies. Let's please be considerate of the fact that this topic has been discussed, at length, numerous times in the past. I'd like everyone participating in this discussion to pause before responding and consider whether their reply meaningfully advances this discussion forward. If you're not sure how to take the discussion forward, then I would advise you to bring up concrete specific examples that supports your viewpoint, and also recognize that their are immense trade offs at either end of this debate.

27 Likes

@notriddle That's why option 2 would be a thing, to break the cycle of applications and libraries staying on older versions of the language. That would allow Rust to make the long-term changes that benefit the language without completely leaving others in the dark. This let's applications compile with the latest version of Rust while giving the maintainers of certain libraries all the time they need to move from the old implimentation to the new.

@BurntSushi Rust should have a big standard library and here's why - #9 by SmushyTaco From the way I see it there wouldn't be any tradeoffs with option 2 and 3, it would maintain backwards compatibility while allowing the language evolve and make the necessary changes needed to keep the language moving forward. It would be the perfect solution for a big standard library and it wouldn't cause a split in the community because the old implimentation would still remain in the library. The only tradeoff would be having to slightly modify your imports if you aren't ready to update to the new changes.

@CAD97 then we'd have the issue of people using outdated versions of crates. Rust should have a big standard library and here's why - #12 by SmushyTaco

No it wouldn't solve it. It would take the existing issue, subtract a few crates from that, and what's left would still be essentially the same problem.
As long as a crates ecosystem exists, you will have instances of your root application transitively using 2 (or more!) versions of the same dependency. It's unavoidable.

(note: consider replying to multiple people in one post (by using quotes), it helps avoid cluttering the thread.)

If the versions are semver compatible, then only one version will be compiled into the end product. And if the library is shipped with the core libraries, then anyone using the current semver-major version will be using at least the version shipped with the language distribution.

The only time multiple versions of the same library are used in the same compilation is when they are semver-incompatible versions. And you cannot "fix" that; the crates have incompatible APIs.

You either commit to API stability (thus stagnation) and everyone can use the same major version, or you release new semver-major versions, allowing for breaking improvements, but users have to manually decide to upgrade. That's a fact of how semver works. You don't change anything by calling it std.

8 Likes

C++ is a bad example here. Here's a problem that the C++ Standards Committee is debating now (emphasis mine):

Current implementations of std::regex​ are often 2x slower than equivalent use of regular expressions in interpreted languages like Python and PHP, and are roughly 10-100x slower than equivalents in modern systems languages like rust or go.

C++ ended up with poorly performing regex implementation, and C++ is stuck with a bad implementation, because fixing it will break ABI. Rust hasn't matured to the point of worrying about ABI yet, but the same class of problems applies to all standard libraries: once something is in, it may be not be possible to fix or improve it.

Rust had built-in serialization already. It's is now the rustc-serialize crate. If that was blessed to be in the standard library, we'd be stuck with it, instead of having much nicer serde.

So I think Rust is doing much better than other languages. Instead of having to tell users "no no no, don't use this part of the standard library! Use the faster better leaner 3rd party crate instead!" it can skip the first part, and go straight for "use the faster better leaner 3rd party crate".

39 Likes

Coming from Java it took me a bit of getting used to small stdlib. However, I love it now (that's not to say I don't see any problems in crate system, but that's another story). Often, precisely because stdlib doesn't have this-or-that, community recommends a well known, well tested crate - the same for everyone so we don't have such split as in Java. I mean, lack of features in stdlib means ecosystem evolves faster to a better solution than stdlib would have ever be.
One thing I would consider though, is breaking backwards compatibility once in a while. Regardless of how small stdlib will be, sooner or later a mistake will be made and we'll have to live with it forever. I would accept a situation where once a year about 1-3% crates is broken. It would be easy enough for maintainers to stay up to date, and those unmaintained crates shouldn't be used anyway. As long as deprecated code doesn't compile (instead of "compiles but doesn't work") personally I'm fine with it. It's even better because it gives me hopes that language will evolve and stay modern. I wish some things were gone from Java.

2 Likes

Even C has problems with it's standard library. And that is a pretty minimal library.

For example the ancient and extremely error prone string functions the use of which is discouraged now a days.

Or various thread unsafe functions.

It makes a lot of sense to me to let the Rust developers and user community develop whatever libraries they need, outside of a minimal and essential standard.

Then, as long as the original libs are available forever programs can use them forever. No breakage.

As and when new improved versions of the same functionality come along, with whatever breaking changes, people can upgrade their programs to use them. Or not.

In a world with package managers like Cargo and package repositories like the Crates system it becomes unnecessary to include everything and the kitchen sink into a big language + standard library blob.

Oddly enough I have seen Bjarne Stroustrup campaigning for more stuff to put into the C++ standard library, so that C++ can hold it's head high compared to Java. Perhaps that desire is now realized to be a bad idea, what with the arrival of 'modules' in C++.

10 Likes