Why are there no concurrent collections in std?

#21

I’d like to push back, perhaps slightly, against the notion that concurrent collections are really so fundamental that they belong in std. Every domain has their own set of capabilities that are fundamental. In mine, high performance computing, it’s MPI, parallelism, and simd abstractions. For web development, it’s http, templating, and async I/O. For compilers, it’s string interning and context free grammars. All of those things can be provided by third-party libraries of generally high quality.

The current state of std is that its capabilities are fairly universal, not sure I’d want it to grow to include things like concurrent collections, which, at least in my experience, aren’t as fundamental as you seem to think.

13 Likes
#22

@polarker why not contribute to someone on www.patreon.com/ or such to work on the functionality faster?

#23

I concur. If anything, there should be a speed bump against using concurrent collections, as they inevitably carry contention penalties. I’d rather have a moment to consider if my application would do better with a non-Sync collection managed by a message-processing task in an actor pattern, or something similar.

2 Likes
#24

People, that keep wanting std to include concurrent collections, completely misunderstand the purpose of std: it is meant to be the foundation to build the ecosystem upon. It is not meant to be anything like Python’s standard library. I feel that the same people complain about Rust’s ML-inspired syntax being too foreign to a typical C++ guy. They expect Rust to make the same mistake as every other language out there, just because it is convenient to do so.

std should include the absolute minimum of primitives, that can be used to create more complex data structures and algorithms. Concurrent collections are not fundamental and concurrency itself is not fundamental, but ordinary collections are, just like sets in mathematics. Concurrency is a too complex and broad subject to be brought into std just because some people are too lazy to perform a quick search on crates.io. How many ways are there to create a concurrent data structure? There are at least two categories: lock-free and… not lock-free, I guess. They can be implemented using different techniques. Which of those belong to std?

The core team already has a very long list of things to do, they have work planned for years ahead. I would not want them to lose focus and start spreading efforts on things that are not mission-critical. Regardless of what others think, I still consider Rust as not yet mature enough, and it does not mean it is bad. It only means in it still has room to grow, but it can become bad if it repeats mistakes that were made 20 years ago.

One last thing: “batteries included” approach is extremely overrated. Come on, it is 2019, add a line to Cargo.toml and have your batteries downloaded from a server automatically. If you do not trust your Internet connection, copy the sources of the library to your project and relax.

16 Likes
#25

If writes are rare, evmap might be faster I suppose.

#26

I agree with all the people who say that crates.io is superior to the stdlib.

However, I think there is a legitimate problem here, which is discoverability.

The nice thing about the stdlib is that it’s extremely discoverable: there is only a single stdlib, so there’s no choices to make, and everything is contained in one place.

Whereas with crates.io… you have to search for things (did I use the right keywords? did I miss a crate in my search?), and then you have to choose between multiple different competing crates (which one is the most full-featured? which one is well tested? which one is well maintained?)

All of this searching takes time, and it takes mental energy. It is so much easier to just rely on the stdlib, since it requires no choices, and it’s guaranteed to be well designed, well tested, and well maintained (well… for the most part).

I saw some comments that crossbeam is the de-facto concurrent crate. But if you’re new to the Rust community, you don’t know that. This is insider community knowledge that is only obtained after being a part of the Rust community for a while.

That’s why people keep asking for a stdlib, because a stdlib doesn’t require that sort of insider community knowledge.

So there is clearly a problem with discoverability. I hope we can all agree on that.

But the solution isn’t the stdlib. Instead, the solution is something like the awesome lists: a carefully curated list of “blessed” crates.

I’m not saying we should just copy the awesome lists, just that we should copy the idea of a curated list of crates.

There should be some minimum criteria for a crate to be put onto that list (such as it being widely used, full featured, well tested, well maintained, etc.)

And that list needs to be official: it needs to be promoted on the official Rust website. It needs to be promoted on the crates.io site. It needs to be extremely easy for newcomers to find the list. It cannot just be some third-party list.

The reason it needs to be official is because of both discoverability and trust: people who are new to Rust are naturally going to look at the official Rust website. Many people don’t even know that the awesome lists exist.

As for trust, it needs to be possible for people new to Rust to trust that the list is accurate and up-to-date. Nobody wants to use obsolete crates.

I think this is a far better solution than a stdlib, we just need to decide as a community to do it (and make it official).

8 Likes
#27

This has been discussed and attempted multiple times, please see:

I agree with you that knowing the best crate to use is a problem, but I disagree that the solution is as simple as what you’ve suggested. It’s hard to come to a consensus on what the criteria for a crate should be.

The current best solution we have is the Rust Cookbook that recommends tried-and-true solutions using high quality crates to specific problems. It could definitely be expanded; the other huge problem in this space is finding people with time to keep resources like this up to date!

7 Likes
#28

The problem with “The Rust Platform” and “stdx” is that they are a monolithic crate which just re-exports other crates. That’s not my idea.

My idea is to simply have a text document/web list of crates (similar to the awesome lists). It’s lighter weight, lower maintenance, and gives it a more informal feel (whereas an umbrella crate is more “locked down” and has to deal with things like semver and docs).

Adding metrics to crates.io is closer to what I’m proposing, but not quite. It proposes an automatic metric (which has a lot of flaws), whereas I’m proposing a social metric (decided on a case-by-case basis by people, not algorithms).

So I’m not sure if it’s fair to claim that my idea has been rejected, when my idea wasn’t even tried in the first place!

However, I do agree with you that this is a Hard problem, and reaching consensus will be Hard. But that hasn’t stopped the Rust community from tackling other controversial and Hard problems. We’re an optimistic community that rejects zero-sum thinking.

I’m a bit optimistic that the maintainer situation might fix itself, once an official mechanism is in place: there are probably people who would be willing to contribute to the blessed list, but are too intimidated to contribute to Rust itself.

1 Like
#29

I would also like to note that the Rust Cookbook isn’t even listed on the official Rust website, and I personally had never heard of it.

I like the way it presents things (it is similar to my idea, but taken even further), but people can’t use or contribute to something they don’t know exists!

#30

https://github.com/rust-unofficial/awesome-rust is another curated list of libraries and resources.

#31

Ah, I see how I wasn’t quite clear. The reason I cited those examples was that most of the controversy was around which crates should be included in these bundles, which is a problem your idea will run into as well.

Ah, social metrics either involve moderation (and our moderation team is already stretched to its limits) or lots of trusted folks’ time (and those folks are usually busy with other things the community finds valuable).

It sounds like you’re really passionate about making this happen! If that is the case, the best next step to take would be for you to create a candidate list, document the criteria you used to create the list, open an RFC, and work on building consensus.

Great point! This was suggested, but the feeling at the time was that it’s not official enough yet to link to on the main site.

1 Like