Context
There has been a lot of conversation around the standard library bundled with the Rust programming language. While some favour a comprehensive standard library, others favour a minimal one with third-party crates providing missing functionality.
The first step in any software development activity is requirements gathering. And, in this case, those requirements are not clear. Let's start with building a list of things that can potentially be in the standard-library but are not as yet. The purpose here is just unbiased enlisting of items.
Primary Goal
To eventually arrive at a stated long term vision for std-lib and a policy towards inclusion/exclusion of items within it.
Secondary Goal
To address community concerns around functionality and crates which are not a part of std-lib but are fairly popular in their usage.
Note:
The list is not meant to imply an endorsement towards including something in the std-lib. It's just an un-opinionated listing of functionality which is not present in std-lib but could be.
Next steps
Once we have a relatively comprehensive list, we can work on designing a detailed survey / information gathering to start digging further on specific items.
I'll start by throwing in a few that I can think of over the top of my head:
If you think of Rust as a language for the next 40 years, you have to really think what can you design and put in the stdlib now, and be able to promise that it will still be the best interface in 40 years. We can make that bet about Vec, UTF-8 strings, Option, because they're the simplest thing we can do, and we know these things have worked fine for decades.
But what else is going to be relevant? What's not merely a fad or technology that's going to be clunky and useless, and be a baggage for the language in a few years?
Python's standard library has support for the IFF format, which was used by image and audio formats on Amiga and classic Macintosh.
There were times when everything had to be XML, and surely you'd have XML, XSLT and schemas in the standard library.
And is any "standard" GUI framework going to be even good enough to be used at all, and then survive for entire Rust lifetime without any breaking changes?
There's also another proposal to have many "standard libraries" that are curated collections: Curated crates, rather than Big Standard Library IMHO that direction is more promising, because you could have "standard web stuff library" with OAuth, servers, etc. and a "standard embedded library" with HAL, USB, mini network stack, etc., and they could live and die separately without being a burden for each other or the core language.
Cannot agree with you more there. However, I just want to emphasize that the intention here is to not endorse any opinion on either side of the argument - big or minimal std-lib.
However, we should try and understand what is it that people want, use and why existing solutions do not comfort them. (Hopefully) Leading to a design of a sustainable policy around progressing std-lib (in whatever way) so that there is a formal basis for decisions made with respect to std-lib in future and any discussions around extending / maintaining std-lib can have a reference of sorts.
To that end, we need to list what people think can potentially be included in the std-lib as a first step.
In terms of what people want, I think it can be approximated to: whatever they're working on now.
The interesting bit to me is why people want something in the standard library, and I suspect it's:
Ease of installation. Historically (pre-Cargo, pre-npm) dependencies have been difficult to install, so stdlib had an advantage of being already installed.
Ease of discovery. All the functionality is widely known to users of the language, and it's documented in a well-known place.
Curation. There's an assumption that there's a high bar for inclusion into stdlib, so the stuff in stdlib is good.
Trust. Stdlib is maintained by the Rust team, not a random person on the internet.
Stability. Stdlib is not allowed to break, unlike random dependencies.
And I think if some other solution satisfies these criteria, it'll be as good or better than having stuff in stdlib.
Ease of installation: cargo is pretty good. We should have cargo add built-in, so that users don't need to copy'n'paste stuff.
Curation: this is still somewhat missing. Proposed in the other thread that I've mentioned. The Rust team wants to be neutral about 3rd party crates, but maybe they could officially recommend some of the most common crates?
Trust: still missing. cargo-crev is working on it.
Stability: crates ecosystem follows semver and is doing OK I think. There is some churn, but that's because the language is still evolving.
I don't think the bar is "this must still be the best interface possible in 40 years". By that metric, std should be empty. 40 years ago linked lists would have been seen as a more pragmatic option over vectors, UTF-8 didn't exist, and Option monads were completely unheard of outside of academia. It's impossible to say what computer architecture will look like 40 years or what design patterns will be considered "best" then.
In my opinion, things included in the standard library should 1) work as advertised without bugs and 2) be good enough for 80+% of users. There are many instances of things in the standard library right now that work but aren't the fastest or the most memory efficient or the smallest code size but they are good enough for the vast majority of Rust applications (format!(), HashMap<_, _, S = RandomState>, etc come to mind). The std cannot be "best in class" for everyone because there are many different trade-offs that can be made. This is where the crates.io ecosystem can provide the "best" crate for whatever metric you are optimizing for while the standard library provides a very good default.
However, I'm not suggesting that everything that meets those requirements should be included. There is little value to anyone to including and maintaining code in std that nobody uses. I've often seen people citing the infamous "Python's standard library is where code goes to die" quote as support for a minimal std. Let's not forget that Python is a 40 years language, in part, because of its large standard library.
There are always going to be better designs out there. Let's not make perfect the enemy of good and prevent the standard library from growing simply because there might be a better design someday.
Maybe! There are languages stuck with UTF-16 or 8-bit codepages. Standard library's stability guarantee is brutal, and it seems inevitable that over time it'll become more and more outdated and problematic.
Imagine Rust had nothing built in at all. If std was just another crate from crates.io that you'd add to you project. It could even be versioned and evolve freely over time. That would be fine for things like HTTP servers, template engines, LINQ, GUI, hashes, regexes.
The only places where "std is just a crate" breaks is:
basic types used to exchange data between libraries, like strings. From C and C++ we know that having 7 different string types in a project is painful.
features that need integration with the compiler. Box is magical. ? needs glue code for Result (unstable Try trait). async needs a Future type.
So I think a valid option would be to limit std to just these things.
This is a problem, and it gets worse if some parts of std become really outdated. Something being in std can be interpreted as an endorsement, or at least encourage use of std features due to laziness or familiarly. It creates a paradox: on one hand it's inviting to use std features, and on the other hand it's not the best option and better solutions should be encouraged.
Perhaps, the real error in a new language like Rust is following in the tradition of having a library called "std" - code for base functionality could easily have been placed under a "system" or "core" namespace with us users none the wiser. This little factor of the "std" namespace creates the expectation that Rust should behave like the languages and ecosystems it is trying to improve on.
Instead of fattening the std library, most issues around finding the most convenient tooling can be fixed by a system that tags crates with usability metrics (e.g. one or more of; use cases, performance, popularity, documentation, code comments, lines of code, version etc).
This will make flexible searches for the best crates possible, pleasant and trivial - emphasis on "trivial".
Then maybe, on top of the above system, a search plugin that interacts with cargo.toml can be created for Rust editors and IDE's - such a plugin would preferably have an easy to use, highly informative GUI.
Such a search engine could even be wired into RLS or Clippy for crate suggestions.
Moderator note: Let's keep general/philosophical discussion about expanding the standard library in the existing thread or in other separate threads, so people who are interested can use this space for listing/discussing specific things that could potentially be added to the standard library or other such collections.
One thing that would be amazing for my use case is basic data structures for 2D, 3D and 4D vectors/matrices and basic operations (like addition, subtractions, inner and outer product, multiplication, etc).
The reason is that every crate has its own implementation or uses a different linear algebra crate. There's nalgebra, geo_types, euclid, cgmath and a few others I haven't used yet. In one project, I had to include three and translate between them, because every algorithm crate I included used a different one (lyon uses euclid, geo-booleanop uses geo_types, spade uses nalgebra).
I don't even care about having the actual math implementations in std, but just having a common struct for storing them (like the solution used for Futures) would be great.
Most 3D math crates (euclid, nalgebra and cgmath, certainly) support mint interfaces, so it might be possible to have those crates that rely on them instead rely on mint.