Why `itertools` is not part of STD?

Crate itertools implements a lot of useful and often essential algorithms based on pattern iterator, extending STD.
Why is it not part of STD?
Is there reason why it should be not a part of STD?
Is that harmful to have in STD something what is not totally necessary?


Should I ask the question on the internals forum?

3 Likes

Well, itertools has all of the tools that the standard library didn't think of to include.

1 Like

Alice, thanks for making it explicit. I was not sure if it is so.

I am curious to find out answers on rest questions. Do you have some? :slightly_smiling_face:

There are probably some of the tools in itertools that make sense to add to std, and it has certainly happened in the past that something in itertools was added to std. Some of them probably don't make sense to add.

10 Likes

Why?

There's a trade-off between convenience and overhead of code and complexity. More methods on the core Iterator trait would mean more documentation to search through, more code to compile, more code to maintain and optimize.

As you add more and more specialized methods, at some point they will be so rarely used that they will get in the way of most users more often than they help few special users.

9 Likes

Also, itertools can afford for the breaking changes if the API somewhere proved to be non-optimal, but std can't - if something is stabilized, it's here forever.

23 Likes

Is there any formal guide what should be in STD?

As a member of libs-api, no, there isn't. I think an informal guide could probably be written with input from the team. We don't all share a brain and have the same opinions on what should or shouldn't be in std, although I do suspect we broadly share the same value that the crate ecosystem should be leveraged as much as possible.

17 Likes

What do you mean with that?

The general rule that's been quoted before is roughly that

  • if needs language integration, it goes into core (e.g. async and Future)
  • if it's a universal bit of vocabulary for communicating between libraries with an obviously correct API, it likely should go in std
  • if it can be implemented as a crate, generally prefer providing it externally first where the API can be iterated on

Of course, there is a long tail of further items that could be added to this list with varying specificity and consensus, but these three are generally considered the core guidelines that are built off of.

Other language's stds prefer a more "batteries included" approach; Rust more generally isn't afraid of saying "there's a trade-off here that std doesn't want to choose for you, use an external library instead". The availability of reusable libraries is a core part of Rust's library design.

14 Likes

For further information see existing discussion elsewhere; for example I could find this blog post that appears to describe the situation appropriately

6 Likes

there's a trade-off here that std doesn't want to choose for you, use an external library instead

Thanks. That explains a lot.

Thanks. Interesting reading.

The Rust language's stability guarantee states that if code compiled with any version of the standard library, it will continue to do so in future with minimal hassle

Is that so? As far as I understand Rust 2021 is not backward compatible with Rust 2015, no?

One possibility is for the Rust project to create a super-library that contain commonly used, vetted crates that work well together. This approach has seen some success in other languages like Haskell.

Is that such crate?

Still reading.

1 Like

It is backward compatible in that crates written with Rust 2021 can seamlessly work with crates written with Rust 2015. You may continue using Rust 2015 as long as you like for all of eternity, it will never break or become obsolete. All Rust has done is extend the surface area of the language to cover both editions β€” it’s just a flag that may or may not be enabled.

4 Likes

But only if developer keeps edition = "2015" in his cargo file. Right?

Note that Iterator is in core, where it doesn't even have access to an allocator. So Itertools gets extra possibilities just from that, like collect_vec.

12 Likes

(a) This is the default. (b) Yes, but because of this interoperability requirement, changes to APIs are almost completely impossible.

Excuse me, on what question is that answer?

I thought you also meant to ask "shouldn't this make it possible?"

1 Like