Rust for embedded development: Where we are and what's missing


#101

The embedded development industry is very good at coming up with terms that sound good but in practice are so broad that they aren’t useful in communication. HAL is a great example: what approach is not covered under “Hardware Abstraction Layer”? There are higher-level HALs like Arduino that provide an API to expose a defined system down to specific IO ports, there are lower-level HALs that are thin wrappers around peripheral registers on a specific chip, and any number in between.

These all have different audiences, and very often they are built on top of each other. It’s a good thing to have multiple types of HALs being experimented with in parallel for different types of users - one of the strengths of Rust is that has such a broad dynamic range, and can be used for very low level and very high level APIs at the same time, without much overhead.

That said, initialization and configuration is an interesting puzzle to solve. The Arduino approach is to define a set of I/Os and make the vendor responsible for building a board to support those I/Os and provide a board-specific library as an abstraction - basically, defining a common hardware and software platform to build on top of. Of course, there’s a big universe of embedded developers for which specifying and building custom hardware is the whole point, and Arduino doesn’t (and was never intended to) work for them.


#102

The iMX6 / iMX7 platforms (Cortex-A, not Cortex-M like most of the discussion here) are interesting targets because they are one of the few Cortex-A chips that appear to have enough hardware documentation to do bare-metal work.

For me, this particular project is exciting to see because it’s got a ton of detail about the usually unwritten stuff that takes forever to figure out - how to deal with the bootloader, how to connect to it with a debugger, not to mention all the low-level work of how to bring up a board.


#103

Well, the low-level API already exists in large parts (i.e. those MCUs for which a SVD description exists) and works quite well thanks to @japaric’s svd2rust crate. The real question is what to put on top of that to make the low level layer manageable without having to repeat oneself all the time and to ensure portability which is exactly was a HAL (no matter how you define it) is all about; if it cannot provide that it’s totally worthless to me (at least).

I’ve no problems working with the low-level API or even addressing the hardware with assembly instructions myself but then again this will not yield anything useful for the public because it will only work for the specific MCU on a specific PCB design. Also not having a HAL by my definition creates a huge hurdle for beginners to even get into game and a really steep learning curve.

I must admit I hate Arduino; the IDE is utter crap and the ecosystem has too much focus on quick and dirty implementations BUT (and this is why I brought it up) the ecosystem, support and resources (documentation, forums, …) are gigantic and that absolutely deserves nothing but everyones greatest respect and gratitude.

If Rust for embedded development wants to strive at the very least we need a really low barrier for entering the game…


#104

Unfortunately, SVD only really covers register definitions and IRQ assignments, and it doesn’t do a particularly good job with either, plus the quality of the files provided by vendors (if they exist at all) leaves plenty to be desired.

I agree that this leaves a big gap to be filled, and vendors have no incentive to fill it in a way that benefits anyone except for themselves. In fact, I believe that they see this as an opportunity to create lock-in with their custom C and C++ development environments, and they’ve been successful so far in doing this.


#105

@therealprof

Nope

That’s what you say but your examples keep saying otherwise. Just quoting this:

-use blue_pill::led::{self, Green};
+use nucleo_f103rb::led::{self, Green};

fn main() {
    Green.on();
}

This is effectively “duck typing”. There’s nothing that guarantees that
blue_pill::led::Green and nucleo_f103rb::led::Green have the same API. To
properly write a program that’s generic across devices you have to write the
program using generics and the HAL traits; I have shown how to do that above.
The example you have quoted has not been written using generics so it’s not
portable across devices (or boards).

But in the peripherals you need to know HAL implementation details namely
which registers

Like I said before one way to remove this is to remove the references from the
API by pushing the synchronization into the implementation of the API. I’m not
going to do that for the blue-pill because that would negatively affect the
performance of the applications I have planned for it. Every particular
concurrency framework will probably deal with this "peripheral name leak"
problem differently (so that neither memory safety or performance are lost);
there’s no solution for that problem in the RTFM framework at the moment.

This. And embedded-hal sits at the bottom. embedded-hal is meant to be used
to implement other higher level HALs without having to re-write register
manipulation code in their implementations.
(This is an explicit design goal)

Indeed. I expect that solving this will also solve the "peripheral name leak"
problem for the RTFM framework. Or least the configuration solution could be
leveraged to solve that problem. An Arduino-like solution is out of question;
the solution needs to be flexible enough to solve the general problem rather
than Just Work for a few boards. And the general problem has the requirement
that the application and not the HAL / library should decide how to wire pins
to peripherals.

@coder543

Interesting. Someone on #rust-embedded mentioned that they got Rust working
(bare metal style and pass the boot process) on a Cortex-A processor (A7 IIRC)
and that they were using a svd2rust generated crate for register manipulation.
It probably was one of these iMX processors since they are well documented
(apparently they generated a SVD file from the reference documentation).


#106

Well, that’s your example, not mine. I’ve just selected that for simplicity reasons. I don’t even think that led::Green is actually something that should even exist and if it does it should merely be an alias for a specific GPIO pin and use the GPIO semantics. But my point remains that code using the HAL should not need to know which registers are needed to address a specific pin.

I still fail to see why that would be necessary. The way I imagine things is that instead of specifying registers in the peripherals one would specify specific resources and the HAL would take care of mapping that to the required registers to implement the requested resource.

Sorry, but that is a stark misrepresentation of what Arduino actually does. The only thing Arduino does to specifically address some specific boards is alias logical functions to physical pins. However there’s nothing preventing anybody from just addressing GPIOs using the physical names (or pin numbers) to implement complete custom solutions just using the chips. In fact this is so generic that once support is available this usually supports the whole family of MCUs at once and the onus is on the user to actually support the correct pins that support the requested functionality; e.g. I could “port” a project from a STM32F103 to a STM32F303 in just a few seconds.

I totally agree and that is exactly what Arduino does. However there’re usually defines at the beginning of the application defining which pins to use. So you change a few lines by picking the right configuration for the target, compile it and – boom it works… No need to say, oh, for this board I need to reserve this register and this register because I want to use GPIO A1, and then initialise the GPIO with the correct registers and clutter the whole program with board specific details…


#107

I’ve seen far wore efforts than SVD…

Seems you haven’t come in touch with Cypress PSoC then. There’s some decent lock in; everything else is really open compared to them…


#108

And the general problem has the requirement that the application and not the HAL / library should decide how to wire pins to peripherals.

I’d argue that it’s the function of the chip crate to define the peripherals, and BSP to map peripherals to pins based on the board layout, and to implement the HAL (relying on chip crate functionality wher e it’s common across boards). The application should, where possible*, be BSP/board agnostic.

  • Of course different boards have different features, so that’s not always possible. But if your application just blinks DEFAULT_LED and writes a choice phrase to DEFAULT_UART then it should be run unmodified on top of the BSP for most boards.

#109

I’ve been using this basic hierarchy to organize embedded crates:

  • Application - This is where main lives. You might have many application crates depending on a system or board.
  • System - A board + any external devices / shields configured as a unit. Not always required.
  • Board - A specific MCU + devices on a PCB - for instance, a development board.
  • Device - A non-MCU device connected via some form of I/O. Chip + Board independent.
  • Chip - A set of closely related variants of a MCU, generally sharing a datasheet.
  • Vendor Common - Peripherals shared by MCUs from a single vendor.
  • Architecture Common - Peripherals shared by MCUs of the same architecture (i.e. Cortex-M).

There are plenty of possible variations, particularly at the Board + System + Application level. Generally each crate is responsible for defining in Rust the things that are defined in hardware. For instance, the chip crate defines the arrangement of peripherals and everything that connects them as well as the chip-specific peripherals; the board crate is where you define the clock (assuming you are using an off-chip oscillator), LEDs, buttons, and things like mappings between logical pins and named I/O pins on a header. On top of that, if you have an external shield or are embedding the board into a larger system, it may make sense to have a system crate that handles connecting those external peripherals to the board-level I/Os and then to the internal chip peripherals.

I expect that there will be (or maybe exists already) a crate defining an Arduino-like API as a set of traits to be implemented by a board crate or in some cases as an adapter for a board crate. An embedded developer would be free to build an application using solely those traits and have it run correctly with a recompile on any of those board crates simply by using Cargo features.


#110

I’ve posted the source code of a demo application that drives a 24x WS2812B
LED ring and can be controlled via a serial interface, remotely if you use a
Bluetooth adapter. It uses the DMA to reach redrawing speeds of up to 160 FPS
while keeping the CPU use relatively low (15% @ 8 MHz).

cc @juleskers

I also have posted some thoughts about a CircularBuffer abstraction to use
with the DMA and peripherals like the ADC. If you have opinions on that comment
on the thread.


@therealprof

I still fail to see why that would be necessary.

Like I said that’s one way.

HAL would take care of mapping that to the required registers to implement

That would be the role of the configuration layer I mentioned. As I said no
concrete proposal for something like that for the RTFM framework.

However there’s nothing preventing anybody from just addressing GPIOs using
the physical names (or pin numbers) to implement complete custom solutions
just using the chips.

Unlike the AVR Cortex-M chips usually have remappable peripherals. For
example, the PA2 on the stm32f103 could be used as an analog pin (ADC2), serial
pin (TX2), a PWM pin (T2C2), an input capture pin (T2C2) or a quadrature encoder
interface pin (T2C2).

I meant that the Arduino, AFAIK, doesn’t solve this configuration problem: “how
is functionality distributed across all the available pins (without overlap)”.
Arduino doesn’t have to deal with this problem on the AVR because each pin has a
single (alternate) function (apart from GPIO).

Remapping is also why the blue-pill uses peripheral names like USART1 in the
API, the USART1 could be using pins PA9 (TX) and PA10 (TX) or PB6 (TX)
and PB7 (RX), instead of something like write(PA9, b'H'), what if PA9 was
configured as a PWM pin (T1C2)? - that shouldn’t compile but it’s better, IMO,
if you can make the mistake in the first place.

No need to say, oh, for this board I need to reserve this register and this
register because I want to use GPIO A1, and then initialise the GPIO with the
correct registers and clutter the whole program with board specific details…

There’s no proposal for implicit initialization (as in it gets taken care of
something / someone other than the application writer) for the RTFM framework.
It’s highly related to the configuration problem I mentioned.

@thejpster

I’d argue that it’s the function of the chip crate to define the peripherals,
and BSP to map peripherals to pins based on the board layout, and to implement
the HAL (relying on chip crate functionality wher e it’s common across
boards).

I suppose we could store the configuration as crates – that sounds a lot like
the “#define” / C model. I wonder if that’s the most flexible solution. We do
have unstable compiler plugins but I’d rather avoid something that breaks often
(cf. Zinc’s demise).

The application should, where possible*, be BSP/board agnostic.

That sounds like a rather reduced set of applications. But I suppose there’s
value in having a “Hello, world” application to test your whole toolchain /
development setup.

OTOH if you application targets a set of very similar boards then it may make
more sense to build the application against a single board crate and deal with
the small differences via features / #[cfg]s.

@jcsoo

That’s a nice hierarchy.

An embedded developer would be free to build an application using solely those
traits and have it run correctly with a recompile on any of those board crates
simply by using Cargo features.

Hmm, Cargo features are not the right … feature for this I think since they
are binary. For example, it doesn’t make much sense to compile an application
with both the features “board_a” and “board_b” enabled. You want something
that resembles an open ended enum.

There’s also the problem of extern crate;
if you are using traits since the trait implementation will be in a different
crate for each board. We probably want something better than:

#[cfg(board = "A")]
extern crate board_a as board;

#[cfg(board = "B")]
extern crate board_b as board;

// ..

#[cfg(board = "Z")]
extern crate board_z as board;

to make an application compilable for several boards …

Kinda makes you wish that it was possible to link to a crate using a command
line argument but without having to explicitly add the extern crate to the
source code. I think someone may have already proposed / asked for something
like that on rust-lang/rfcs.


#111

It’s okie! Whatever is most efficient and easiest for you :slight_smile:

Of course you know you can just do something like:

fn parse (bytes: [u8; 6]) -> Foo {
  let x: u16 = bytes.pread(0).unwrap();
  let y: i32 = bytes.pread(2).unwrap();
  Foo { x, y }
}

And it should work fine, which will read at the host machines and endianness - doesn’t get much easier imho! That’s why I made it, cause I’m lazy! :smiling_imp: And yea for fixed arrays I sometimes do the same thing when forcing invariants this way.


#112

This concept is a bit more complex than the with the AVR chips, but not by much. Alternate GPIO allows you also to do almost the same. But then again there’re (Cortex-M) chips like the Cypress PSoC which are even more complicated than the average Cortex-M implementation because they also have programmable logic blocks and freer routing of functions to pretty much any pin.

That is an incorrect interpretation. Check out e.g. this little bugger: http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-2586-AVR-8-bit-Microcontroller-ATtiny25-ATtiny45-ATtiny85_Datasheet-Summary.pdf, for instance PB0 can have the following functions: Digital Out and In (In also as an IRQ source), SPI, AREF, analog, Timer/PWM output, I2C (2 and 3-wire mode).

This is exactly why the reservation should happen at a physical resource level and not at a register level.

Also you’re still missing the fact that the Arduino is not just about former Atmel AVR (now Microchip AVR) anymore, it works with pretty much any MCU and CPU right now, also with Cortex-M MCUs, including (but not limited to) the blue-pill: http://www.stm32duino.com/. A lot of Arduino code even compiles without modification and just “configuration”.

That was not what I was suggesting, in fact something like write(PA9, b'H') never even occurred to me as useful. I want to say give me the USART on <PB7, PB6> and it’s the job of the HAL to determine whether this is a valid configuration and whether the resources are allocated allocated to another function. Then there could be a BSP like @thejpster suggested, providing a ready made configuration for that per board so you can just say: led::green instead of GPIO<PC13>. I think I’m pretty much on the same page as @thejpster here.


#113

You’re right, features are not the right way to do it, but Cargo doesn’t really have anything better at the moment, and as you point out it’s a pain to deal with. Also, I think all variations using the same target end up with the same output binary, which is not necessarily desirable if you want to build all the variants at once for distribution or testing.

In practice I’ve been keeping things in different crates and breaking out the main application logic into a separate crate.

@therealprof

I think you and @japaric may be talking past each other a bit. The way I see it, he’s talking about working strictly within the bounds of the MCU - really the core + memory + peripherals. Configuration of the MCU is a separate concern that takes place “outside” - during initialization or by the bootloader.

On the other hand, Arduino is essentially a specification for what the end results of that initialization should be, and a set of mappings from physical pins and peripherals to consistent logical pins and peripherals. It’s effective but very limited: once you want to go beyond, you immediately have to drop down to the vendor libraries, and there is no checking or assistance at that point - you really have to go to the schematic for your specific board and start from there.

The current trend is for vendors to fill the gap by providing proprietary configuration “wizards” such as Processor Expert, basically GUI driven code generation tools that internally have knowledge about their MCUs that go far beyond what they provide in SVDs, along with opaque driver code that gets injected into your project. Of course once you start using this, you have basically locked in your project to that vendor’s chip family, plus you will probably have a difficult time debugging any kinds of problems you might have with that code. I’m sure you can agree that this is pretty much the opposite of what we want for Rust.

There’s nothing sadder to me than doing a Google search for code examples on how to work with some peripheral and having the top result being a blog post that consists of a long sequence of screen grabs showing the exact steps needed to get a wizard to get it to spit out the right code. Even more so when that version of the wizard is no longer available, or if all of the images are missing. At least you can go to the Arduino source code and learn from it.


#114

I would not say it’s very limited because there’s really no ultimate restriction on what you can actually do. If you’re comfortable with low level register banging (or even using assembly mnemonics) then there’s really nothing preventing you from doing it. The important thing here is (and I think that’s what you’re calling “effective”) is that it is really easy to get started and good results right off the bat and on a huge amount of completely different hardware (even on custom boards). Sure, at some point you might have to deep dive into schematics and the manual but with pretty much every other cross-vendor environment (like mbed) you pretty much have to start like this! For me as someone who also does workshops on such topics it is absolutely crucial to have robust examples that work on a large number of “configurations” with only a few and obvious changes that also allow for easy modification and extension.

I’m a bit curious as to why you think I’m vouching for IDEs here since I hate them and do not use them if I can avoid it (including but certainly not limited to the Arduino IDE ;)).

What we really need is an easy to use but at the same time incredibly flexible solution to trigger the curiosity of the people and lock them in play and experimentation mode. In terms of ease of use and capability @japaric’s RTFM crate is pure joy and ingenuity; never has it been so easy, safe and quick to write multitasking code for MCUs; I can totally work with that.

The hal on the other hand is quite meh: I was optimistic when it came out that I would write plenty of HAL and “BSP” crates to support some of the hardware I have here and people might want to use. But quite frankly, as long as the ability to write truly portable example applications is not even a design goal of this hal, I am so not interested in going down this rabbit hole…


#115

Maybe I should be more specific so that you can see I’m mostly agreeing with you. The Arduino API is fairly limited, but the Arduino environment allows you to step outside of it to do what you need. At that point you are typically using a vendor API or directly with the hardware. I don’t think this is necessarily a bad approach; just an observation about the scope of API itself, compared to the vendor API which you expect to be comprehensive, at least for some level of functionality.

I didn’t intend to imply that you were pro-IDE; just making an observation about the world that we live in.


#116

Awesome! I’m a bit swamped right now, but it’s on my to-do list! Thanks!

I agree, and I sense a lot of frustration when reading between the lines. @japaric, @therealprof, please take a moment to feel eachothers shoes! You’re both adding valid points to the discussion, and I’d hate if this miscommunication would sour the thread.

In my understanding, japaric is talking about a “bottom-up HAL”, building the really low-level stuff, before we can even start considering the higher levels (“build the foundation before starting on the roof”), whereas therealprof is reasoning “top-down”, high-level HAL, and believes that the high level must influence the design of the low level to get the best results (“don’t paint ourselves in a corner”).
Both valid concerns!


#117

A good portion of the “low-level stuff” exists already (also written by @japaric) in the form of svd2rust, cortex-m-rt and cortex-m-rtfm crates and it’s exceptionally great and useful code!

I’m not really advocating a “high-level HAL” nor influence of HAL into the low-level design here, in fact I rather think that the HAL should always be in the lower middle of things: The purpose of a HAL is to provide an API that abstracts the low level aspects of discovering, setting up, addressing and using hardware capabilities and maps that into functional blocks so that developers can focus on functionality rather than having to worry about the low level hardware details all the time. And here’s where I think the HAL approach sketched by @japaric is a bit lacking because it tries not only to be a HAL but also a BSP at the same time, which I think should ultimately be nothing more than a hair thin layer saying: this board is (by default) using this hardware capability for this specific function. To make things worse, even the BSP part is still requiring intimate hardware knowledge because instead of saying give me the GPIO pin B12 you (e.g. for bluepill) have to say: I want to use GPIOs, here’s the Port B resource and here’s the RCC resource and this will configure pins B12 to B15 and make them automagically available…

The way I would like to see the stack-up (from high to low level) is this:

  • Application
  • (optionally) Application frameworks like cortex-m-rt and/or cortex-m-rtfm and higher level libraries
  • (optionally) Board specific BSP (Mapping board specific functions onto HAL provided functions, e.g. assigning LEDs to GPIOs (including important details like low-active, high-active) and assigning names , declaring hard wired functions like UART…)
  • MCU specific HAL (Mapping functions into hardware specifics)
  • svd2rust (Register description)

#118

As suggested by @juleskers I would like to announce clerk to you, a hardware agnostic HD44780 LCD library I am currently working on.

Here is the link to the original announcement.

The hardware layer trait definitely needs some improvement and there are some other things I would like to get some feedback. Let me know if it would be OK to post questions here or to move the discussion to the announcement topic.


#119

Probably best to keep Clerk-specific things in the clerk thread itself. Mentioning here was mostly for awareness among the bigger embed crowd.
I see you’ve also mentioned clerk on the github library tracking issue. Good going!


#120

How do you plan to use Rust in your embedded application?

I want to write a pure Rust application over an Adafruit Feather M0 which ATSAMD21G18 ARM Cortex M0 (https://www.adafruit.com/product/3178)

And, what’s preventing you from using Rust? Some possible reasons:

I would have liked to use rust over an ESP8266 but rust didn’t support it. The feather should arrive tomorrow so i’ll find out how successful I am at getting it going :).

Also, just feedback, i’m a little bit confused overall in how Rust embedding should work. Searches tend to pick up old blog posts and whatnot and its not clear how much of that is relevant now. I think something like an Embedded Cookbook would be helpful.

Also, it would be nice to be able to reuse some of the arduino libraries out there, for me specifically to be able to use all of the feather hardware that is available.