Using Rust for railway software / [no_core]

[oeble]

Hello !

I'm working on a R&D project at Alstom which aims to evaluate new languages which could be used for our software developments as replacement for C and Ada (among others). In the context of railway/signaling system or application software, I believe Rust can be a good candidate.
The applicable standards to the railway software development define some constrains on the used languages and associated tools (especially the compiler) but even more on everything that is linked to the final executable. To avoid the certification process, the usual way is to remove any runtime (no libc for example).
So here is my first question, is it possible to compile some Rust code without the libcore (it's not very clear). If yes, does the language still usable ? Do you have examples of what is not available then ?

Notes:

• Obviously it will be no_std.
• An example of target is a bare metal armv8

Thank you for your answers.

[steveklabnik]

Hi @oeble!

libcore is usually not something that you remove; it's not really a "runtime" exactly. The important bit is definitions of bits of the language in Rust itself, which is why the issue talks about 'lang items'.

Is there a specific reason that you're looking to remove core, rather than "remove any runtime"? Rust with libcore is a bit conceptually lower in my mind than no libc; removing libc is closer to removing libstd. That said, these distinctions, in some sense, are fairly arbitrary. It sounds to me like it's strictly about lessening the amount of code you'd need to have reviewed? It's roughtly 30 KLOC, so that's more than zero, but not that much. I can understand that every bit counts though! Most people that do embedded work still use libcore, so while it can work, you'd be running into some interesting rough edges.

One thing you could do is, there are about ~70 "lang items", which are said implementations of stuff the language expects, in libcore. Copying just those out into a new package/straight into your code would be much smaller, for example, the first one is in cell.rs:

#[lang = "unsafe_cell"]
#[stable(feature = "rust1", since = "1.0.0")]
pub struct UnsafeCell<T: ?Sized> {
    value: T,
}

You'd remove the #[stable] declaration, and you'd be good to go. You could only include the ones you actually use, and with no_core, it will error on the ones that you use but don't have defined, so you could even only add them as you go. There's one related issue though; as this feature isn't stable, you'd be forced to pick a nightly build of the compiler. Doesn't mean you'd have to update at any particular frequency, of course, and due to Rust's development, nightly is fairly stable for this use-case, I'd think, as it's mostly higher-level stuff that'd be changing, or compiler internals you're not using. As I haven't done this myself, I can't truly advise on that, though.

Very excited about the idea of using Rust with trains!

[oeble]

Thanks for your answer.

The certification requirements is far more work than a review but the fact that libcore is small is certainly an argument to keep it. I will try your suggestion, for now it can be enough to know that working without libcore is possible. The use of the nightly channel may be a problem for the certification of the compiler itself. However, I think we can easily have this feature available with everything else equal to the stable channel with the cost of rebuilding the compiler (am I wrong ?). This is probably more acceptable (I will analyze that later).

[kornel]

It may be worth noting that libcore doesn't add anything to your executable that you don't use.

If I take a no_std example and compile it without importing libc, it outputs very bare-bones program:

.text
	.file	"playground0-5b48a64088e77c8b76ba6366b22a5646.rs"
	.section	.text.main,"ax",@progbits
	.globl	main
	.p2align	4, 0x90
	.type	main,@function
main:
	xorl	%eax, %eax
	retq
.Lfunc_end0:
	.size	main, .Lfunc_end0-main


	.section	".note.GNU-stack","",@progbits

You can even compile individual .rs files with rustc --crate-type=staticlib to get object files and control linking yourself.

[jethrogb]

@oeble

However, I think we can easily have this feature available with everything else equal to the stable channel with the cost of rebuilding the compiler (am I wrong ?). This is probably more acceptable (I will analyze that later).

Yes, you can manually build a version of the stable compiler that allows the use of unstable features.

[bluss]

And if you build Rust projects without libc you'll run into needing these basic functions, which are expected to be provided. Just memcpy and similar. https://github.com/alexcrichton/rlibc (and implemented in 90 lines of Rust).

[rpjohnst]

For certification purposes, I would treat libcore more as part of the compiler implementation than as a separate library. Most of what it provides are "lang items," so it is thus fairly tightly integrated with the compiler and those items need to be implemented the right way to preserve compiler correctness. (For instance, they might change with a compiler update and if you had your own copy that could break things.)

Perhaps a better comparison than libc would be libgcc.

[Michael-F-Bryan]

It sounds like you want a no_std program. That will compile without any of the standard library, which is the "runtime" that you are wanting to avoid. Even on microcontrollers you'll have access to libcore, providing a happy medium between usability (e.g. even though you're entirely constrained to the stack and don't have access to Vec or String you can still use &str) and being lightweight/platform-agnostic (e.g. it doesn't even assume the presence of an OS or allocator).

I've played around with running Rust on microcontrollers in the past and although it's still a young space my Rust programs have turned out to be significantly more robust and maintainable than C programs!

[oeble]

@kornel

You can even compile individual .rs files with rustc --crate-type=staticlib to get object files and control linking yourself.

This seems to be a good alternative to keep control on what is linked to the final binary. Thanks.

@rpjohnst

Most of what it provides are “lang items,” so it is thus fairly tightly integrated with the compiler and those items need to be implemented the right way to preserve compiler correctness. (For instance, they might change with a compiler update and if you had your own copy that could break things.)

Updating the compiler will require a new certification so having our own and smaller version of libcore to update is most likely not a significant work regarding the cost of the new certification. This is important to keep in mind but not our concern for now, we will evaluate what is the best strategy later.

@rpjohnst

Perhaps a better comparison than libc would be libgcc.

Yes probably. About libgcc we have different strategies among the projects some may link it, some not. But knowing that we have the choice is what is important for now.

Thank you everyone for your replies. So, my conclusion is that libcore can be removed with the cost of the reintroduction of whatever is needed in the software code. Removing libcore does not make the language unusable but it is probably better if we can keep it as it is.

[skade]

It might be worth noting that libcore has a stable, committed interface (that means, you can extern crate core), it may only be expanded in future versions.

Implementing libcore yourself relies on details that are not yet committed interface (mostly some annotations, that's why you need nightly to build it).

I'd argue that the strict policy the Rust implementation has to clarify what's reliable interface and what not helps building software, but that's certainly up for certification authorities to decide.

Also, I'd like to note that the notion of having extended service releases[1] of Rust comes up again and again, we'd be interested in your opinions on the subject.

[1]: Similar to Firefox, where a version of the browser is kept under extended support for a certain timeframe.

[oeble]

Having "extended service releases" is definitely something that can put more weight in the balance in favor of Rust. If a (blocking) issue on a specific version of Rust is discovered, the capacity to have a correction for that issue without any other important change is obviously something we value. It is then easier to demonstrate the absence of breaking changes and reduce the testing, review and validation activities of the impacted products to the minimum.
For us, the time between two software releases is easily 1 or 2 years. And once in production, changing/updating the software of an electronic board cost often a lot (if often require physical manipulation, it may have impact on the exploitation, requires new testing of each use case...). Then we often prefer to live with old software (and old compilers) that have proven their reliability, and have a known list of acceptable, analyzed and justified bugs rather than take the risk of using a newer version. The newer versions can bring unknown bugs that will then be discovered later in the development process or worst once in production. The later is considered unacceptable in the railway industry (and probably in industry in general).
With an ESR software, this is less problematic because even if the expected maturity is gained during the development process, updating to the corrective version introduce less risk to have a new bug or regression (and less work to prove it).
The same kind of consideration may also applies to the certification process which can fails because of problematic bugs. And updating to a corrective version with only few changes reduce the work needed to update the ongoing certification process.

[skade]

@oeble

For us, the time between two software releases is easily 1 or 2 years. And once in production, changing/updating the software of an electronic board cost often a lot (if often require physical manipulation, it may have impact on the exploitation, requires new testing of each use case…). Then we often prefer to live with old software (and old compilers) that have proven their reliability, and have a known list of acceptable, analyzed and justified bugs rather than take the risk of using a newer version. The newer versions can bring unknown bugs that will then be discovered later in the development process or worst once in production. The later is considered unacceptable in the railway industry (and probably in industry in general).

I have a bit of a problem understanding the broad term "industry". That is certainly not the whole "software industry", as there's quite a bit of development happening with rather fast compiler iterations. In many contexts, for example the compiler changing optimisation behaviour (probably becoming worse) is not problematic - it might lead to a performance regression in a system, which is always bad, but might not be a make-or-break thing. But there's obviously also many places where that is an issue. Which fields would be included in that?

How much treatment would such an ESR need, though? Obviously, if there's no changes intended at all, you could just pick a version and stick with it. But there are probably classes of bugs you want to have fixed, I imagine that being mostly security issues or major performance blunders? Would backports of (for example) tooling and API be of interest?

I mean, the huge advantage of ESR is usually that it moves on a known base (e.g. same version of LLVM and such).

[juleskers]

@skade

I have a bit of a problem understanding the broad term “industry”.

Oeble seems to be talking about "physical computing"- "industry", where such strict certification is fairly common. One of the certification requirements is often "proven correctness" over a certain range of inputs.

E.g. "must activate emergency brakes ALWAYS if train conductor does not trigger dead man's switch", or " max. speed MUST be limited to 15km/h if crossing a yellow signal". and then a test procedure with 5000 permutations of train speed, weather conditions, 29th of February, previous inputs etc.
Once you pass such a test suite, you are not allowed to change a single bit in the binary, or you lose certification.
The medical industry has similar restrictions, which is a big reason my department labels their stuff "for research purposes only". Once you certify, you are basically frozen in place by the certification costs.

"One in a million" bugs happen surprisingly often if you have a national railway network, moving e.g. 20 million commuters daily...
Or a ICU heart rate monitor beeping for 60 heartbeats-per-minute year-round, or analysing 3 _bil_lion DNA bases per cancer patient, 4000 times a year.
When physical human lives are impacted by mistakes, "meh, this update shouldn't break anything" isn't good enough.

Fun related reading on how some things aren't fixed in railway contexts: The axle count of trains in Switzerland must not be a multiple of 2^8

Ironically, this paralysis also means many bugs are never fixed, leading to medical security nightmares (media.ccc.de), such as rickrolling actual heart rate monitors, or crashing the million-dollar MRI by running a port scan on the guest WiFi...

[skade]

Well, yes, but I'd like to hear a clear definition. "In the industry" is prone to communication errors ;). Alone the number of ISOs for different fields is a lot of fun. I'm aware of all these problems and stories, what's more interesting is finding out what would be direct ways of making the situation better for people at specific places.

[lf94]

This is one of the paths I want to take with Rust. Do you have any open source examples?

[Michael-F-Bryan]

It really depends on what your use case is, but a good start for embedded things is to look at @japaric's blog. Another really good resource of his are the discovery and copper guides. The discovery guide takes you through getting up and running with a particular microcontroller, while the copper guide is a bit more advanced and teaches you how to directly interact with the hardware, effectively making your own HAL.

Quite a lot of crates are already designed to work in a no_std context. There's even a "no standard library" category on crates.io, although I'm sure that doesn't include all the no_std crates out there.

[oeble]

Well sorry, I was not clear. The only part that I generalize to the whole "industry" is the fact that a major bug in production is not acceptable. Yet, what I was saying is probably true for a lot of "physical industry" like @juleskers highlight it. His examples are exactly what I have in mind. We are creating safety critical software in the sense that a major bug may lead to humans to be harmed or worst (in real life, with all the existing protections, you need a lot of critical bugs to see something harmful happen, hopefully no-one will never see it).

@juleskers

Ironically, this paralysis also means many bugs are never fixed, leading to medical security nightmares (media.ccc.de), such as rickrolling actual heart rate monitors, or crashing the million-dollar MRI by running a port scan on the guest WiFi…

This is also very true...

About the ESR subject, we can (and it is what we do for some compilers) stick to an old certified version with its known list of acceptable bugs. The kind of bugs that we want to have fixed are not always security ones but more any kind of bug that may lead to unexpected hazardous behavior (I remember this example of a compiler optimization (I have no reference on it) that would lead to unexpected result if the sum of two compared variable overflows on 32bits. This was very specific to a target but for us this kind of bug need a new version with a new certification)

@skade

I mean, the huge advantage of ESR is usually that it moves on a known base (e.g. same version of LLVM and such).

Well, in my opinion, from usual software development point of view, having a known base, the base you rely on, is still true if you change your compiler version as long as you rely on stable functions and standards language feature (this is very true for C and ADA, less for Rust with the "unstable" features). I do not think that staying on the same version of LLVM really matter. Changing a gcc version shall not change the compiled program behavior or the way you use the compiler (ESR or not). However, the new gcc version may introduce a new optimization mechanism in some context that you exactly have in your software, or maybe new feature that you are not using. You, and many other, may think that cannot bring any issue, but when making safety critical software it has to be proven. This where we would have a lot of work to demonstrate that this potential issue does not matter. For my point of view this is where ESR is interesting.

@skade

what’s more interesting is finding out what would be direct ways of making the situation better for people at specific places.

I think that ESR is one thing that help because it allow to use something we know will gain maturity with less risk of regression. Another thing that help is the ability to easily list (or extract a list) of the known bug applicable to a given version (even very old).

However, I have never worked for the certification of a compiler, this will be new for me. All project I worked on before already had their certified compilers (some have more than 10 years). I will talk about this with someone more experienced to have his insight. All of what I said is my personal opinion and my coworkers may have a different one.

Note: Firefox ESR is supported for one year so less than the time we may take from the start of specification to the release of a version of our software. (Our product are then supposed to run 30 years but without software update in most case). So one year of support, feels almost like a minimum to me.

[juleskers]

@skade

I’m aware of all these problems and stories, what’s more interesting is finding out what would be direct ways of making the situation better for people at specific places.

Ah, perfectly understandable desire! Thanks for clearing that up!

As for the "no_core" ideas: If the rust compiler is going to need to be certified anyway, wouldn't it be a good idea to certify its included version of LibCore with it as well? (Actual question, I'm not familiar enough with this level of the internals to judge!)
I'm assuming that the compiler certification is going to be a big ticket item anyway, and sneaking in libCore along with it probably won't make the cost (that) much bigger, but will get you a LOT of fancy toys/primitives to (re)use in the firmware. The great "high" level API (iterators, etc) that Rust offers is one of the big value-adds over C.

Another question: how realistic would it be to spread the costs of this big compiler certification? If there were a public ESR branch of Rust, that multiple companies could reuse, is it possible (politically, technically and legally) to share this single "upstream" certified compiler over multiple projects/companies? (And thus provide a basis for multiple companies to pay into a single "certify rust" moneypot)

[oeble]

About core, it is probably a very good idea to certify it too. From what I know and if the Rust experience goes well I will push for this certification but its not something I really have the control on. The certification for libcore will not have the same level of requirements as the rest of the compiler because it is linked to the final binary (don't ask the details, I do not have them). However because libcore is quite small, it will probably be acceptable to realize this activity.

I've never heard about the costs of a certification being spread through some kind of agreement between the companies which want to use it, but I'm not expert on that.
It is possible however, to find already certified compilers on the market such as the GNAT Pro for ADA which is certified T3 (the maximum for a compiler, allowing to build SIL4 railway software so the highest level of safety). ADAcore sells licenses and support for it along with a bunch of tools to help for the certification of the software product itself. I don't think that Mozilla want to provide such support or service...

[juleskers]

@oeble

ADAcore sells licenses and support for it

Ah of course, if it is hard/expensive to do, people will want, justifiably, to make money off of it.. Interesting clash between open-source thinking and commercial enterprise, no idea what to do there...
Anyone feel like starting a "certified Rust" startup?