Specification, standardization, and independent implementations

I tend to agree with this talk at RustConf 2019 in that aiming for a Rust standard owned by an entity such as ISO, ECMA, or maybe even a company such as Mozilla (not that I dislike Mozilla) is detrimental to Rust as a language because of the association between voting rights and being a nation-state or large company with deep pockets as well as ISO requiring a fee to even read the official standards. When Rust gets a formal standard, I think it should be owned by the Rust community and managed using the systems that the Rust community has already decided on for managing Rust currently (see https://www.rust-lang.org/governance).

4 Likes

programmerjake,

Not off topic at all. And I totally agree. When there is talk of multiple implementations of a programming language we naturally start thinking about specifications, standards, conformance requirements etc, etc.

In that presentation Isabella Muerte, who is highly respected in the C++ standardization world and now a Rust enthusiasts, pleads with the Rust community to not go down the road of ISO standardization and the like. Arguing that it closes people out of the loop and makes it so that only high roller corporations and such get a say. This is quite the antithesis of the community ethos Rust has built and is in turn has been built on. I was surprised to hear that coming from someone in Isabella's position but it makes so much sense.

Sadly Isabella did not have time to fully develop the anti-standards body case in that presentation. I hope Isabella gets to expound on it another time.

The Symbiotic Relationship of C++ and Rust: https://www.youtube.com/watch?v=YZomx3Jt4Xs&list=PL85XCvVPmGQhDOUIZBe6u388GydeACbTt&index=14

As it happens I was already composing a reply to make the same point, so here it is:

Speaking as a naive outsider to compiler things...

In the bad old days of the "software winter", the time of close source software and the distribution of "secret" binary only executables by multiple vendors, it made a lot of sense to have standards defined that potential purchasers could use to evaluate offering against. Standards the vendors could use to work against and create compatible products. This is the model that C/C++ etc grew up in. Hence the ISO standards for those languages, the bureaucracy of standards committees and so on. Likewise for Javascript and so on.

Sometimes vendors would even manage to produce products that were compatible with each other out of all this. Although it was often in their best interest not to do do. Gotta lock customers into your thing some how, right?

It's not clear to me that all this heavy weight standardization makes any sense in the Open Source world and especially the Rust community. Does it make sense to have multiple teams working on multiple versions of the same Open Source thing? Would it not be better to have the talent work on one definitive version? Isn't all this duplication of effort a waste of talent, effort, time, when they could be doing better things?

And of course having multiple implementations means they will always be out of sync and causing chaos for users that try to swap from one to another. It would be divisive to the community.

As it stands, we have LLVM and GCC. To use them to create Rust programs they need a front end. Given the above, is it not better to have one good, definitive, Rust front end that is use by both of them?

1 Like

I tend to agree that sharing the Rust frontend does have definite benefits, however, if Rust can't be used in some safety-critical systems (cars? am not familiar enough to know for sure) due to not having independent implementations for bureaucratic/avoiding-shared-bugs reasons, I think the somewhat-obvious benefits of being able to use Rust in safety-critical scenarios as well as ending up with a much better formal spec outweighs the problems that independent implementations will cause.

Having multiple independent frontends is one of the most effective ways of finding spec holes, otherwise, the spec doesn't really matter since everyone relies on the actual frontend code rather than the spec.

2 Likes

@ZiCog As a person who has personally edited over 15k pages of IEC standards, I fully agree that a formal specification owned by ISO is inappropriate to Rust. I did not propose that; I merely stated that a stable specification with that level of detail is needed for independent implementations of rustc and its tooling to be possible.

Given the current rapid rate of evolution of rustc and its support tooling, my conclusion was (and is) that there is no effective way to avoid cross-contamination of frontend implementations, and thus no way to provide the implementation independence demanded for some classes of safety-critical systems.

IMO, ownership of formal specifications for Rust and its proximate tooling should remain free to the public, similar to the RISC-V specifications. That will require ownership to be placed in a non-profit foundation with an adequate charter. But that's a subject for another thread.

1 Like

TomP,

Speaking as a naive outside observer/user I agree with you.

At some point a rigorous specification and conformance tests etc is a good idea but it seems too soon.

For a long time I touted the standardization of C/C++, Javascript and such as a very good reason to use them over others that were one man shows. It ensured longevity and consistency for one projects and products.

I'm somewhat convinced now that the actual ISO standards and the like are not the main point. As Isabella Muerte points out it's all about the community of developers and users. Without a community one might end up with standards that are dead, see ANSI standard BASIC for example. Closing the standards documents to the public just does not fit well.

The RISC V model is interesting. Anyone can read that spec. and build what they like with it. But if you want to advertise a product as a RISC V you will need to be a member of the foundation and get your design through the conformance tests.

I keep hearing about this "implementation independence demanded for some classes of safety-critical systems." So far in my years of experience with such things I have never seen anyone actually make use of that.

Multiple independent implementations may well be a sensible requirement when you don't want to become dependent on a single vendor of closed source solutions. It makes less sense to me in the Open Source world. We can all fork the code and be a vendor. If you are building billion dollar products on something you can pay to have someone maintain it if need be.

1 Like

I think we should specifically avoid the RISC-V model (not that I dislike the RISC-V ISA, I'm currently working on building a RISC-V-based GPU/CPU as part of Libre-RISCV), because, even if they publish the specifications and anyone can implement them, the RISC-V Foundation is currently much closer to ECMA and/or Khronos in that, to effectively participate in the decision-making process, you have to be a member and sign a bunch of NDAs, and, only then, are given access to the internal mailing lists and other communication channels which is where the majority of the development occurs. We (Libre-RISCV) have repeatedly been ignored, even though we have repeatedly explained that we can't be a RISC-V member due to our funding source (NLNet) and ethics requiring that our activities are public because we're trying to prove that our processors don't have any malicious backdoors (unlike Intel and others, where it's plausible that there are backdoors in among all the proprietary firmware -- see Intel Management Engine, also Dual_EC_DRBG) and therefore are safe for everyone to use without everyone needing their own national clone.

3 Likes

Every programming language in the mainstream has a formal specification and more than one toolkit available, be it open source or not (see below for some samples).

Someone may interpret that as a mere coincidence, or otherwise as a tacit message from the market: "a programming language offering a single toolset and without a specification is not to be trusted in the long term".

So, a complete, formal and properly maintained spec for Rust is needed.

If someone is willing to help the community by improving the existing toolset or starting anew, I am pretty sure she should be the only one to make such a decision.

Java
https://docs.oracle.com/javase/specs/

C
http://www.open-std.org/jtc1/sc22/wg14/

Python
http://marvin.cs.uidaho.edu/Teaching/CS515/pythonReference.pdf

C++
https://isocpp.org/std/the-standard

C#
https://www.microsoft.com/en-us/download/details.aspx?id=7029

JavaScript
https://www.ecma-international.org/publications/standards/Ecma-262.htm

PHP
https://phplang.org/

Go
https://golang.org/ref/spec

2 Likes

[Note: This thread was split off from the GCC front-end discussion.]

4 Likes