Rust for embedded development: Where we are and what's missing


#81

@japaric
Thank you for your very informative answer.

Have you seen the Real Time For the Masses (RTFM) framework?

I haven’t studied it in depth, but I’ve read your blog posts on the subject. I like the concept (especially the static verification at compile time). However I feel like it could become somewhat limiting, if your project approaches a certain complexity, so there are still types of applications that could probably benefit from using a normal RTOS.

The Rust language is not tied to any particular concurrency model; instead it has two marker traits, Send and Sync, that can be used to build concurrency models. Those two plus the borrow checker has been enough to build a memory safe thread-based concurrency model in the standard library, and to build a memory safe task-based concurrency model in the RTFM framework.

Thanks, I made the usual mistake of conflating the Rust language and its standard library :wink:
I had a quick look at the implementation of std::thread and especially the constraints of the closure that thread::spawn() takes and that cleared things up a little as to how it could potentially work (although I imagine the devil’s in the details…)

OTOH adding support for one of these RTOSes to the standard library sounds less likely.

Yeah, I think that would not make sense given the sheer number of RTOSes there exist, in addition to the points you mention…


#82

Many thanks for making embedded development much easier in Rust, @japaric. Awesome work!

–> “I want to write pure Rust applications”

I’m developing software for a living, mostly webservices (unfortunately not in Rust yet). I’m personally following Rust development since 0.7 or so and love to do personal projects in Rust from time to time. Low level or bare-metal coding is kind of a hobby and I’m fascinated about pure Rust programming on microcontrollers.

Actually nothing except lack of time :wink:

–> “How do I manage several build configurations (different boards, different devices, etc.)?”

This is exactly what I was thinking about for some time. It would be nice to be able to write firmware that compiles for different boards. Rust offers enough ways to abstract hardware-related stuff into own modules or own crates, but building for multiple boards seems hard. I’m unsure about how one should select the target board (some boards may use the same target triple but different peripheral crates, some boards may use a completely different target triple). Everything I came up with felt like either bloated cfg statements, a bloated build script or bloated customized target definitions.

There’s one other thing I’m not sure about how to use. Std has mpsc channels to pass data between different threads. In bare-metal, there are no threads, but different contexts. E.g. timing-critical data gathering typically runs in a periodic interrupt handler, adds gathered data to a ring buffer and leaves dequeuing and processing the data to the main loop. Sounds like a generic spsc channel (atomic bounded ring buffer) could be useful. But I wonder about how to handle such a channel since there’s no way to “move” something to the interrupt context like Send makes possible on real processors. Task priorities/thresholds in cortex-m-rtfm are very interesting, but they’re Cortex-M only, right? (I’m mainly used to Cortex-M, but I just wonder about others)


#83

@zargony

any thanks for making embedded development much easier in Rust, @japaric. Awesome work!

Thanks!

Everything I came up with felt like either bloated cfg statements, a bloated build script or bloated customized target definitions.

Yeah, my conclusion has been that Cargo features (due to their binary-ness) and custom targets (due to their “you must build a sysroot” requirement) are not a good solution to this. I think a tool is need to manage “profiles” that describe boards and these profiles have some sort of hierarchy and inheritance between them. I have yet to write down how this would look like; I still need to sort out my thoughts.

But I wonder about how to handle such a channel since there’s no way to “move” something to the interrupt context like Send makes possible on real processors.

Send works fine; it’s not tied to threads. You can use it to mean send data between different interrupts. In RTFM Send means “can be transferred across tasks (which are = interrupts)”. In fact, RTFM resources can be used a channels to send stuff across tasks; I had to patch some memory unsafety related to this before v0.1.0 release because it was “safe” to send a task token from task A to B which breaks the Local invariant of being only accesible from a single task.

Task priorities/thresholds in cortex-m-rtfm are very interesting, but they’re Cortex-M only, right?

Not really. cortex-m-rtfm is an implementation of the RTFM framework for the Cortex-M architecture but core of the RTFM framework, tasks and resources, can be implemented for other architectures. It just happens that task and resources can be implemented very efficiently for the Cortex-M architecture because of the hardware features it has (interrupts with configurable priorities and the BASEPRI functionality). But I have seen an implementation of the RTFM scheduler (the Immediate Ceiling Priority Protocol) for the MSP430 architecture; I can’t find the link right now though.


#84

I think a tool is need to manage “profiles” that describe boards and these profiles have some sort of hierarchy and inheritance between them. I have yet to write down how this would look like; I still need to sort out my thoughts.

Have you looked at device trees and device tree overlays? They are associated in most people’s minds with embedded Linux, but I think either device trees or a very similar system could be adapted to an MCU context nicely.


#85

So I got bored and felt like messing with generics again (thanks to Mutabah on irc for the final help with the Indexing generics :smiley: fun), and implemented this api for a more core-friendly version. So nothing allocates, and no results are returned (it will panic if the buffer is too small, no other way around this afaik). Basically you have your choice of the default ctx version (cread), which will read at the host machine’s endianness automatically, or with an explicit ctx/endianness (cread_with).

Here is a sketch, if the trait is too much, just check out the cread_api test function (using it is easy):

/// Core-read - core, no_std friendly trait for reading basic traits from byte buffers. Cannot fail unless the buffer is too small.
pub trait Cread<Ctx = super::Endian> : Index<usize> + Index<RangeFrom<usize>>
 where
    Ctx: Copy + Default + Debug,
{
    #[inline]
    /// Reads a value at `offset` with `ctx` - or those times when you _know_ your deserialization can't fail.
    fn cread_with<'a, N: FromCtx<Ctx, <Self as Index<RangeFrom<usize>>>::Output>>(&'a self, offset: usize, ctx: Ctx) -> N {
        N::from_ctx(&self[offset..], ctx)
    }
   fn cread<'a, N: FromCtx<Ctx, <Self as Index<RangeFrom<usize>>>::Output>>(&'a self, offset: usize) -> N {
       let ctx = Ctx::default();
        N::from_ctx(&self[offset..], ctx)
    }
}

impl<Ctx: Copy + Default + Debug, R: ?Sized + Index<usize> + Index<RangeFrom<usize>>> Cread<Ctx> for R {}

#[test]
fn cread_api() {
    use scroll::Cread;
    let bytes = [0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00, 0xef,0xbe,0x00,0x00,];
    let foo = bytes.cread::<usize>(0);
    let bar = bytes.cread::<u32>(8);
    assert_eq!(foo, 1);
    assert_eq!(bar, 0xbeef);
}

Importantly, if you implement FromCtx for your type, you can then just read out your type as well, e.g.:


#[repr(packed)]
struct Bar {
    foo: i32,
    bar: u32,
}

impl scroll::ctx::FromCtx for Bar {
    fn from_ctx(bytes: &[u8], ctx: scroll::Endian) -> Self {
        use scroll::Cread;
        Bar { foo: bytes.cread_with(0, ctx), bar: bytes.cread_with(4, ctx) }
    }
}

    use scroll::Cread;
    let bytes = [0xff, 0xff, 0xff, 0xff, 0xef,0xbe,0xad,0xde,];
    let bar = bytes.cread::<Bar>(0);
    assert_eq!(bar.foo, -1);
    assert_eq!(bar.bar, 0xdeadbeef);

A writer would be similarly (and easily) implemented. Now I just have to decide if Cread/Cwrite is an ok name, or think of something else :thinking:


#86

I was kind of wondering since you mentioned board support crates: Are we supposed to create and maintain those for every possible MCU/board combination? For instance I do have a Nucleo-F303RE board here and I’m not sure your stm32f30x crate fully applies while I’m very sure your f3 crate does not.

It would be great to have most of the higher level abstraction of the f3 crate to be automatically created from the svd file (potentially even automatically during build time) and merely requiring special Rust initialisation to specify which of the alternate functions are used in a specific implementation.


#87

An update on the HAL front.

I have now published (on GitHub) the cortex-m-hal crate which contains a
Hardware Abstraction Layer (HAL), in the form of a bunch of traits, for the
following peripherals / functionality:

  • Input Capture
  • Pulse Width Modulation
  • Quadrature Encoder Interface
  • Serial Peripheral Interface (SPI)
  • Serial (UART) interface
  • Timer / timeouts

Along with a reference implementation in the form of the blue-pill crate,
which I believe has the most complete API build on top of a device crate
generated via svd2rust. That crate also contains a bunch of examples.

The key points of this HAL are:

It’s fully non-blocking but it’s not tied to a particular asynchronous
model. You can easily adapt it to operate in blocking mode, or to work with
futures, or to work with an async / await model. All this magic is done via
the nb crate so check out that crate documentation too. That nb crate even
has an await! implementation, which doesn’t work right now because generators
have not landed in the language, but that macro lets you do cooperative
multitasking
in a cleaner way than if you would have used the futures
crate
.

It’s minimal to make it easy to implement and keep it as zero cost as
possible. The main idea is have enough of an API to erase device specific
details like registers but let you build higher level abstractions
with minimal overhead. Want a blocking read with timeout semantics? Just compose
the Serial and Timer abstractions using a generic function.

The ultimate goal of this HAL is code reuse. I know people have different
opinions about how an embedded program should be structured (e.g. cooperative
tasks vs event-driven tasks vs threads) and will want to use different
higher level abstractions tailored for their needs. What I want to see is that
those abstractions get built on top of this HAL instead of seeing everyone
rewrite very similar register manipulation code to build those abstractions.

I’d love to get some feedback. I have opened a bunch of issues in the
cortex-m-hal issue tracker where you can leave comments about each
particular trait. The HAL also needs more testing across devices to make sure
it’s generic enough to be implemented for devices from different vendors so let
me know if can or can’t implement this HAL for some device.


#88

@m4b

Thanks for working on this. I will try out scroll for real this weekend. I have some (de)serialization to do this time.

@therealprof

Are we supposed to create and maintain those for every possible MCU/board combination?

Not necessarily. Though I expect board support crates will become smaller and easier to implement in the future.

For instance I do have a Nucleo-F303RE board here and I’m not sure your stm32f30x crate fully applies

Yes you can use that crate with your board but it only has an API that operates at the register level. That crate works with STM32F300, STM32F301, etc. microcontrollers.

while I’m very sure your f3 crate does not.

You probably want to use a stm32f30x-hal-impl crate (which doesn’t exist right now) which is an implementation of this HAL for the stm32f30x crate and that provides a high level API like the one in the f3 crate and that also works with any STM32F30x microcontroller but that doesn’t tie a Serial abstraction to a single USART1 peripheral; instead that Serial abstraction can be used with USART1, USART2, etc.

It would be great to have most of the higher level abstraction of the f3 crate to be automatically created from the svd file

I’m probably misunderstanding what you are saying here, but that seems impossible to do in the general case unless we create a tool that can read reference manuals. Reason being that, for example, to write a Serial.write method you have to write to the data register but that register may be called different names on different chips so you or the tool has to check which of the many registers in the USART peripheral of each device is the data register.

merely requiring special Rust initialisation to specify which of the alternate functions are used in a specific implementation.

Figuring out how to do this generically is on my TODO list. My current plan is to have everyone agree on a struct that, for example, represents the configuration of serial interface; maybe something like struct SerialConf { data_bits: u8, stop_bits: u8, start_bits: u8, baud_rate: u32, ... }. Then we have every device crate (e.g. stm32f30x) implement some routine to initialize a certain peripheral using this configuration struct. That would be the basic glue to erase low level details about registers. Then we have something (a tool or a macro?) that takes some input (external file or Rust code?) that describes all the HAL interfaces that the application will use and then checks that e.g the pins of each interface don’t overlap and that e.g. the clock configurations are achievable, and then produces an init routine that does all the initialization of the described interfaces. Or something like that; I’m a bit fuzzy about how much of that can be implemented (sanely).

@awygle

Another useful library for developing embedded tooling that seems to be missing is one for working with device trees.

I meant to ask: are you proposing using device trees as a general purpose configuration format for e.g. microcontrollers? Or you mentioned them in the context of embedded Linux?


#89

@japaric

I meant to ask: are you proposing using device trees as a general purpose configuration format for e.g. microcontrollers? Or you mentioned them in the context of embedded Linux?

Both, although the former is just a thought that I haven’t pursued to its logical conclusions yet. I think the ability to work with device trees for embedded Linux purposes is necessary. I also agree with this statement:

I think a tool is need to manage “profiles” that describe boards and these profiles have some sort of hierarchy and inheritance between them. I have yet to write down how this would look like; I still need to sort out my thoughts.

Basically, I think we need an SVD that covers boards rather than processors. Device trees are the only semi-relevant example I’m aware of, so my hope is that they can be used for this in a drop-in manner, but I’ve been too busy lately to do much of anything so I haven’t spent the time to see how pie-in-the-sky that hope might be.


#90

@japaric

Yeah, I’m looking at the -hal crate right now. I tried to adapt the code from the f3 crate to the mentioned Nucleo board but for some reason horribly failed to change the frequency of the timer, not even sure why you’re using the very basic TIM7. :wink:

I do wonder though why https://github.com/posborne/cmsis-svd.git has lots of more and less specific versions for the f3 series…

Sounds like a really good idea. I’m looking at that right now…

Sorry, my bad. I was only briefly looking at the svd files before and thought they also had semantic information in them but now that I looked more closely I see that the specific meaning of the functions is only coded as sort of a naming convention; that’s probably not usable.

Something like this is what I had in mind. Well, let me check whether I can make your hal work for this nucleus, and once it works I’ve dozens of other STM32 boards here as well as other Cortex-M implementations to try whether they’ll fly as well…


#91

Cool! You will likely want to try git master as it has the Cread/Cwrite traits whose signature is:

offset -> T

Compared to Pread which is:

offset -> Result<T>

Although as i said latter uses non allocating errors.

They also seem to be faster in benchmarks.

The tracking issue is here: https://github.com/m4b/scroll/issues/7

I’ll likely publish with new traits on crates probably soon.

If you have any comments, insight, opinion, or troubles please let me know :slight_smile: and have fun!


#92

The stm32f30x crate needs an update to be useful for a hal, otherwise it will require the use of completely names than the stm32f103xx variants.


#93

I’m not too happy with that implementation yet. For “fun” I “ported” the blue-pill HAL to the rather similar Nucleo-F103RB and just to change the LED to the GPIO the Nucleo uses was a major PITA and required not only changes to src/led.rs but also to every single example I wanted to run due to the resource declaration in the application source which need to match the HAL. The GPIO implementation can also not be used becaus it doesn’t support the A port at all. Implementing the full HAL over and over again for any tiny change in hardware will be no fun at all…


#94

I have published the code for my remote controlled wheeled robot. The repo includes both the firmware and host code to remotely control the robot. The firmware is built on top of the RTFM framework and the implementation of the HAL that I mentioned in one of my previous comments. If you wanted to see a non-trivial embedded Rust project then I think that ^ is a good example.

(BTW, sorry @m4b. I ended up writing my serialization code by hand out of habit. Actually using e.g. &[mut] [u8; 6] as the input type for my (de)serialization functions helped me catch some bugs at compile time where I misdeclared the size of some statically allocated buffers – I think scroll wouldn’t have caugh that because it works with dynamically sized slices)

As a byproduct of writting the robotic application I created some DMA-based APIs which have not made its way to the HAL crate. I’ve opened a thread to discuss those APIs before we land them in the HAL crate as traits. Feedback would be appreciated!

@therealprof

Implementing the full HAL over and over again for any tiny change in hardware will be no fun at all

Well, implementing HALs in general is not fun at all; it’s quite a bit of work and worst part is that you have to actually read the reference manual. Also never expect two HAL implementations to look (exactly) the same.

The GPIO implementation can also not be used becaus it doesn’t support the A port at all

On the F103 all the GPIO peripherals look the same so I don’t see a major problem porting the GPIO API from one port to another.

only changes to src/led.rs but also to every single example I wanted to run due to the resource declaration in the application source which need to match the HAL

Huh? I’d expect resource declaration to look the same except perhaps from renaming a GPIOC peripheral to e.g. GPIOA. Unless you are using a different, possibly wrong, device crate (?). The device crate should be same as the one used in the blue-pill crate.


#95

With MCUs the situation is a bit special because there are tons of just slightly different MCUs multiplied by a huge number of slightly different implementation so in my opinion it is an absolute MUST HAVE that a HAL framework caters for that and allows for maximum code reuse.

True, just pointing out that it’s not even close to complete yet. :wink:

Well, since I need to pass in the GPIO peripheral to the LED initialisation I had to change the init signature:

-pub fn init(gpioc: &GPIOC, rcc: &RCC) {
+pub fn init(gpioa: &GPIOA, rcc: &RCC) {

which means that I not only had to change the peripherals declaration to GPIOA but also access that and pass it to the led which means I had to change every example using the built-in LED.

But isn’t the idea of a HAL to exactly get rid of such specifics? I would expect that this simple example is fully portable by simply compiling against correct HAL, i.e. by simply declaring that I’d like to use the LED in the peripherals and the HAL ensures that it uses the correct resources for that whatever they may be.


#96

Some people here expressed their desire for wanting to do embedded development
on stable Rust so I have started a survey to identify the most used unstable
features that relegate embedded development to the nightly channel. This
information will be used to inform the Rust teams about prioritization of
language features to consider for stabilization.


@therealprof

With MCUs the situation is a bit special because there are tons of just
slightly different MCUs multiplied by a huge number of slightly different
implementation so in my opinion it is an absolute MUST HAVE that a HAL
framework caters for that and allows for maximum code reuse.

We can’t tell vendors to name registers the same and design them to behave them
the same across their devices families but we can have more sharing across SVD
files
to increase code reuse.

True, just pointing out that it’s not even close to complete yet.

The f3 crate is not even implementing the embedded-hal at this point. I’m
postponing the implementation for that crate on purpose because it’s already
published on crates.io and I’d rather avoid continuously making minor releases
since the embedded-hal traits are not stable.

Well, since I need to pass in the GPIO peripheral to the LED initialisation I
had to change the init signature

You can remove all those references / arguments from that function if you are
fine with doing the synchronization (critical section or similar) within the
function implementation.

But isn’t the idea of a HAL to exactly get rid of such specifics?

init is not part of the embedded-hal traits, and initialization /
configuration stuff is out of scope for that crate. We do want some device
agnostic way to do initialization / configuration but no one has written a
concrete proposal at this point.


#97

I’m not implying that vendors need to do anything but if you look at already existing frameworks like Arduino then you will notice that they do exactly what many people expect from an HAL, namely abstracting the hardware in a way that the specific implementation does not really matter. And those architectures and hence the abstractions actually do share a lot of commonalities across vastly different hardware, e.g. GPIOs are typically named {bank}{number}. Which registers are needed to address those GPIOs are exactly what the device specific HAL should implement and abstract from the user.

I’m talking about blue-pill here, not the older f3 crate.

That’s not what I had in mind. Instead of focusing on hardware blocks or specific register sets the reserved resources should rather be concrete implementations provided by the HAL, i.e. instead of GPIOA and RCC I would simply specify the use of pin A1 as a peripheral resource; if the specific HAL knows about the resource and implements the required traits for it: great, if not then we’ll get an compiler error. The RCC is really a global and implementation specific resource which should simply be occupied by the specific HAL implementation and not exposed to the user of the specific HAL implementation.

It is not part of the traits but it really should be IMHO. And you’re right, it is completely out-of-scope for embedded-hal how a specific HAL actually implements that.


#98

@therealprof

namely abstracting the hardware in a way that the specific implementation does
not really matter.

And embedded-hal does that:

extern crate embedded_hal as hal;
#[macro_use]
extern crate nb;

fn app(serial: &S) where S: hal::Serial {
    loop {
        let byte = block!(serial.read());

        match byte {
            b'+' => /* application specific logic */,
            /* .. */
            _ => {},
        }
    }
}

no device specific code, no registers, no magic addresses / values. Generic code
is device agnostic.

resources should rather be concrete implementations provided by the HAL

I get the feeling that you want to see something like this?

extern crate embedded_hal as hal;
#[macro_use]
extern crate nb;

use hal::Serial;

fn main() {
    loop {
        let byte = block!(Serial.read());

        match byte {
            b'+' => /* application specific logic */,
            /* .. */
            _ => {},
        }
    }
}

However, this can’t be sensibly implemented in Rust because Rust generics don’t
work like that. (You could implement this as a crate by having one Cargo feature
per supported device which may be OK for a few devices but doesn’t scale very well because
you would end with N implementations in a single repository)

Even if you go and implement this the choice of a global singleton (Serial in
the above example) is rather limiting because it implies (a) forbidding
interrupts in the application, or (b) having each method implicitly use a
critical section in its implementation, unless © you are fine with marking all
the API as unsafe (but then you may as well be writing C). Both (a) and (b),
limit the usability of the HAL for applications that involve multiple tasks
and/or deadlines / priorities.

It is not part of the traits but it really should be IMHO

At the end it may or may not make sense to put configuration traits in the
embedded-hal crate but without having a clear idea of what the whole
configuration / initialization stack should look like it’s better to keep the
concerns separated.


#99

Nope. Serial is also not the best of all examples to illustrate this. So real world example, here’re the differences for one of the most simple programs one could write using the hal between blue-pill and a Nucleo F103RB (same hardware really but LED is connected differently:

 #![feature(used)]
 #![no_std]

-extern crate blue_pill;
+extern crate nucleo_f103rb;

 // version = "0.2.3"
 extern crate cortex_m_rt;
@@ -13,13 +13,13 @@ extern crate cortex_m_rt;
 #[macro_use]
 extern crate cortex_m_rtfm as rtfm;

-use blue_pill::led::{self, Green};
-use blue_pill::stm32f103xx;
+use nucleo_f103rb::led::{self, Green};
+use nucleo_f103rb::stm32f103xx;
 use rtfm::{P0, T0, TMax};

 // RESOURCES
 peripherals!(stm32f103xx, {
-    GPIOC: Peripheral {
+    GPIOA: Peripheral {
         ceiling: C0,
     },
     RCC: Peripheral {
@@ -29,10 +29,10 @@ peripherals!(stm32f103xx, {

 // INITIALIZATION PHASE
 fn init(ref prio: P0, thr: &TMax) {
-    let gpioc = &GPIOC.access(prio, thr);
+    let gpioa = &GPIOA.access(prio, thr);
     let rcc = &RCC.access(prio, thr);

-    led::init(gpioc, rcc);
+    led::init(gpioa, rcc);
 }

No problem whatsoever with the use use lines. But in the peripherals you need to know HAL implementation details namely which registers and/or MMIO addresses the LED implementation needs to control the LED on one hand and the exact port the LED is connected to on the other hand.

What I really would expect to being able to use instead is:

peripherals!(stm32f103xx, {
    Green: Peripheral {
         ceiling: C0,
    },
});

fn init(ref prio: P0, thr: &TMax) {
    Green::init();
}

fn idle(_prio: P0, _thr: T0) -> ! {
  Green.on();

  loop {
      rtfm::wfi();
  }
}

Now LEDs are really the most basic example but this is something that pretty much all of the HALs provide. I can take a blinking LED example written for Arduino and simply compile the exact same code for vastly different MCUs (say ATtiny85 and STM32F103) or even the same MCU on different boards (like blue-pill and Nucleo F103RB) and it will just work…

But more important for me (rather than assigning useful aliases like LED1 or Green to board specific resources) would be the ability to simply address all available resources like the GPIO pins in a generic way without jumping through hoops and or requiring changes all over the map.


#100

In Rust, we would strive to use the type system’s generics and other features to our advantage, while Go users don’t have that option, but I just figured I would post this link here in case anyone else finds it interesting to see what another HAL might look like, among other things that are interesting on that link.