Rust should have a big standard library and here's why

Rust has a high quality standard library but the mentality with a lot of Rust developers is that only certain certain things should be in the standard library because of how easy adding dependencies is, a big standard library definitely isn't needed but what many people fail to realize is that having things standardized is far more beneficial because more people would utilize these things and the logic across open source programs would be far more consistent (As opposed to one open source project using one crate and another using a slightly different crate that does the same thing). Some good examples of things that should be standardized are: Crossplatform terminal text coloring, a clean implimentation of case insensitivity (UniCase is a great example of this), JSON parsing, and regular expressions (Yes there's a lot of examples I left out, I can't speak for everyone so put what you'd like to see in the standard library as a reply here :slightly_smiling_face:). If Rust had a huge standard library like languages like C++ have we'd see wider adoption of the language.

6 Likes

Big standard libraries result in parts of it having a bad design that cannot be replaced due to backwards compatibility. I believe we already have this issue with mpsc channels, as there were some improvements we can't put in the standard library due to backwards compatibility. See also pythons standard library http implementation β€” as far as I know it's not very widely used because of a bad api.

As for json parsing, we more or less have a standard: serde. It's sufficiently widely used that the api guidelines for writing new crates even mention implementing their traits (link). Note also that while it's true that C++ has regexes, that was not released until 2011 β€” 26 years after the first release.

You can probably find lots of old threads about this if you look around a bit.

32 Likes

This is a common concern, and sometimes the Rust community + crates.io ecosystem does fall into precisely this sort of trap, but it's also demonstrated an ability to converge on a de facto standard several times over. serde being the most obvious example.

And there is a LOT of past discussion on this. For starters, I'd recommend this as one of the more in-depth "recent" discussions:

and the thread for the old "Rust Platform" proposal (since it goes into why we ended up not doing this):

7 Likes

@alice your argument for a big standard library is bad API which is an understandable argument but I don't think it's completely valid. You just said the standard library wants to maintain backwards compatibility even if it means keeping bad API while crates aren't really burdened with that. My takeaway from that isn't that we shouldn't have a big standard at all this is how I see it: In many cases a language can't learn from it's mistakes and correct them without breaking backwards compatiblity so is this really a good reason not to improve upon the standard library and expand it? Think long-term, what would be more beneficial, a big standard library with occasional improvements that break backwards compatiblity or a small standard library that doesn't improve upon itself to keep backwards compatiblity.

I mean breaking backwards compatibility can cause massive splits in the ecosystem. Just look at python 2 vs python 3. I don't think it's a good method for improved adoption.

13 Likes

@alice While that's true Rust will never reach it's full potential without doing so, @Ixrec linked a discussion and what somebody said is a perfect example of why Rust would be a better language if it didn't prioritize backwards compatiblity:

Cloudflare uses Actix-web in production globally. Note that Actix-web had 8 breaking releases, giving it many chances to improve it’s public API (and it will be able to change it in the future). The maximum number of breaking releases allowed in the standard library is 0, so such large and complex codebase wouldn’t be able to gradually evolve.

If Rust focuses on backward compatibility it won't be able to grow and evolve so do we really want to handicap the language like this or do we want to improve upon it and expand it?

I always see the argument of integrating things into the standard library, but honestly, most things should not be in the standard library, but in official crates. All the net stuff and collections are a very good example of things, that should've never been put into stdlib. I mean, what does a Heap and terrible performing LinkedList have to do in a standard library, that no one ever uses? Should've just put those into officially maintained crates, instead.

8 Likes

Rust has already committed to stability. Backwards-incompatible changes wouldn't just be bad for the ecosystem on their own; they'd also be breaking a promise. Reputable organizations don't just reneg like that.

11 Likes

@notriddle There could be numerous solutions to this:

  1. Give a notice ahead of time for when changes break compatiblity before pushing it to the stable branch.

  2. Keep the old backwards compatible part of the thing that was changed under a different use statement (Maybe std::bc::path_to_old_implimentation)

  3. Do both 1 and 2.

@SmushyTaco IMO the key reason these arguments aren't persuading us is that, in practice, there's very little distinction between:

  • a standard library with solid backwards compatibility guarantees
  • a 3rd party crate on a centralized package repository with solid persistence guarantees

Whether the old library was a std thing or a separate crate, your code can continue to use it forever without worrying about getting broken.


In fact, this is a general pattern with most objections to the "small std policy": they aren't actually affected by whether std is small or large, but are mixing that up with some other more important factor (e.g. whether the language has a decent package manager, or a centralized package repository, etc).

5 Likes

Maybe, if you make use of cargo vendor, but that's neither the default behavior nor is it promoted.

@Ixrec Not only would 3rd party crates be less widely adopted but libraries using them are usually on slightly outdated versions of the crate which result in the same dependency being used multiple times with different versions in one application. Having things integrated in the standard library would also solve that annoying problem.

The last time Rust did anything like that, it took about a year to get everyone switched. And that was a soundness fix, so it really did have to be done.

I should probably give full disclosure here: I don't actually have an opinion on whether Rust should ever have a batteries-included standard library. I don't think it's time yet, but I won't rule it out forever.

But the stability promise has pretty solid reasoning behind it. The goal is that people should be able to upgrade their compiler without waiting for all their dependencies to upgrade. If people avoid upgrading because they're afraid of breaking things, then it's likely to send the whole ecosystem into a gridlock. The go-to example is Python 3, but the fact that there are still shops stuck on C99 when there's C11 and C18* is also a good example of what breaking changes can do to you. The libraries don't upgrade because the applications don't upgrade, and the applications don't upgrade because the libraries don't.

* Two-digit dates are funny.

8 Likes

The solution to the problem you see, OP, is not a larger standard library. It is, put rather bluntly, more standard libraries.

std can stay lean and 1.0, and, say, serde can be shipped with the language distribution, and be usable without downloading it from crates-io (but, ideally, still available and compatible).

The solution isn't too stuff everything into std. It's to develop high quality libraries and "bless" them (either officially or in the community) to drive ecosystem-wide acceptance at the same level of std with separate versioning, so they can provide the breaking change semvers.

8 Likes

Moderation note: Folks, this thread started two hours ago and has already seen ~12 replies. Let's please be considerate of the fact that this topic has been discussed, at length, numerous times in the past. I'd like everyone participating in this discussion to pause before responding and consider whether their reply meaningfully advances this discussion forward. If you're not sure how to take the discussion forward, then I would advise you to bring up concrete specific examples that supports your viewpoint, and also recognize that their are immense trade offs at either end of this debate.

26 Likes

@notriddle That's why option 2 would be a thing, to break the cycle of applications and libraries staying on older versions of the language. That would allow Rust to make the long-term changes that benefit the language without completely leaving others in the dark. This let's applications compile with the latest version of Rust while giving the maintainers of certain libraries all the time they need to move from the old implimentation to the new.

@BurntSushi Rust should have a big standard library and here's why From the way I see it there wouldn't be any tradeoffs with option 2 and 3, it would maintain backwards compatibility while allowing the language evolve and make the necessary changes needed to keep the language moving forward. It would be the perfect solution for a big standard library and it wouldn't cause a split in the community because the old implimentation would still remain in the library. The only tradeoff would be having to slightly modify your imports if you aren't ready to update to the new changes.

@CAD97 then we'd have the issue of people using outdated versions of crates. Rust should have a big standard library and here's why

No it wouldn't solve it. It would take the existing issue, subtract a few crates from that, and what's left would still be essentially the same problem.
As long as a crates ecosystem exists, you will have instances of your root application transitively using 2 (or more!) versions of the same dependency. It's unavoidable.

(note: consider replying to multiple people in one post (by using quotes), it helps avoid cluttering the thread.)

If the versions are semver compatible, then only one version will be compiled into the end product. And if the library is shipped with the core libraries, then anyone using the current semver-major version will be using at least the version shipped with the language distribution.

The only time multiple versions of the same library are used in the same compilation is when they are semver-incompatible versions. And you cannot "fix" that; the crates have incompatible APIs.

You either commit to API stability (thus stagnation) and everyone can use the same major version, or you release new semver-major versions, allowing for breaking improvements, but users have to manually decide to upgrade. That's a fact of how semver works. You don't change anything by calling it std.

8 Likes