Memory mapping SoC registers?

After what seems like most of the day spent chasing documentation around and around and trying all manner of ways to do a simple memory map I'm totally bewildered. So here I am.

What I want to do is map a 32 bit Raspberry Pi SoC register to users space, the system timer, and see it move.

I finally ended up with this code that does at least build and run:

use std::fs;

extern crate memmap;
use memmap::{MmapOptions};

pub fn map_system_timer() {
    // All peripherals can be described by an offset from the Peripheral Base Address, which starts at:
    // 0x20000000 on the Raspberry Pi model 1
    // 0x3F000000 on the models 2 and 3.
    const PERIPHERAL_BASE_ADDRESS: u64 = 0x3F000000;

    // The System Timer is a hardware clock that can be used to keep time and generate interrupts after a certain time.
    // It is located at offset 0x3000 from the peripheral base.
    const SYSTEM_TIMER_OFFSET: u64 = 0x3000;

    let f = fs::OpenOptions::new().read(true)
                                  .write(true)
                                  .open("/dev/mem")
                                  .unwrap();

    // Create a new memory map builder.
    let mmap = unsafe {
        MmapOptions::new()
                    .offset(PERIPHERAL_BASE_ADDRESS + SYSTEM_TIMER_OFFSET + 4)
                    .len(4096)
                    .map_mut(&f)
                    .unwrap()
    };

    let bytes = mmap.get(0..4).unwrap();
    loop {
        println!("{:?}", bytes);
    }
}

Problem is only one of the four bytes printed ever move. The system timer counts at 1MHz.

I made a version of this in C just to check I had the right address. So I hope that is correct.

If that were working as expected my next issue is how to access it as a 32 bit integer in one go?

Have you read The Embedded Rust Book and made contact with the Rust Embedded WG?

I have seen the Embedded Rust book and was aware of the WG,

But they are all about not using Linux, rather smaller micro-controllers programmed bare-metal.

What I want to do here is a simple memory mapping operating under Linux. I can do it in C so I'm sure it can be done from Rust.

I never read embedded rust book, but I suppose it contains note about ptr read_volatile.
And I suppose your problem in exactly this.
Compiler optimize your memory reading and in fact your program just read once from register.

1 Like

Yes, volatile, sounds like exactly like the sort of thing I need.That and reading 4 bytes in a single read operation.

Sadly I'm too dumb to make it compile let alone work. The documentation is making no sense to me. I have been chasing this around in circles for hours...

Could somebody spell this out

Can you post the working C version?

Sure:

//
// Access the System Timer registers directly on the Raspberry Pi
//
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stdint.h>

// #define PERIPHERAL_BASE 0x20000000   // For Pi 1 and 2
#define PERIPHERAL_BASE 0x3F000000      // For Pi 3
#define SYSTEM_TIMER_OFFSET 0x3000
#define ST_BASE (PERIPHERAL_BASE + SYSTEM_TIMER_OFFSET) 

// Sytem Timer Registers layout
typedef struct {
    uint32_t control_and_status;
    uint32_t counter_low;
    uint32_t counter_high;
    uint32_t compare_0;
    uint32_t compare_1;
    uint32_t compare_2;
    uint32_t compare_3;
} system_timer_t;

// Get access to the System Timer registers in user memory space.
void* get_system_timer() {
    system_timer_t *system_timer;
    int  fd;

    if ((fd = open("/dev/mem", O_RDWR | O_SYNC) ) < 0) {
        printf("can't open /dev/mem \n");
        exit(-1);
    }

    system_timer = mmap(
        NULL,
        4096,
        PROT_READ | PROT_WRITE,
        MAP_SHARED,
        fd,
        ST_BASE
    );

    close(fd);

    if (system_timer == MAP_FAILED) {
        printf("mmap error %d\n", (int)system_timer);  // errno also set!
        exit(-1);
    }
    return system_timer;
}

int main(int argc, char **argv) {
    volatile system_timer_t* system_timer = get_system_timer();
    int32_t t0, t1;

    while (1) {
        t0 = system_timer->counter_low;
        sleep(1);
        t1 = system_timer->counter_low;
        printf ("Elaspsed = %d\n", t1 - t0);
        t0 = t1;
    }
    return 0;
}

The counter is clocked at 1MHz so the output is:

$ sudo ./pi_system_timer
Elaspsed = 1000136
Elaspsed = 1000093
Elaspsed = 1000094
...

It should be pretty straightforward, something like this:

  let mmap = unsafe {
        MmapOptions::new()
                    .offset(PERIPHERAL_BASE_ADDRESS + SYSTEM_TIMER_OFFSET + 4)
                    .len(4096)
                    .map_mut(&f)
                    .unwrap()
    };

    let ptr = mmap.as_ptr() as *const u32;
    loop {
        let data: u32 = unsafe { ptr.read_volatile() };
	println!("{}", data);
    }

I managed to get a volatile pointer to my memory mapped timer register and can new see the time advancing. With this at the end of the code I showed above:

    let ptr = mmap.as_ptr();
    let volatile_ptr = ptr as *mut Volatile<i32>;
    let one_second = time::Duration::from_secs(1);

    loop {
        unsafe {
            println!("{:?}", &*volatile_ptr.offset(1));
        }
        thread::sleep(one_second);
    }

With output:

$ sudo ./target/debug/pi-system-timer
Volatile(564682713)
Volatile(565682874)
Volatile(566683029)
...

Now my problem is I can't for the life of me find a way to deference that pointer and get the time into an i32. Obvious things like:

    let mut t0: i32;
    t0 = *volatile_ptr.offset(1);

don't work.

I'm lost again.

Dushistov,

Ah thanks. I took your suggestion and ran with it a bit:

struct SystemTimer {
    control_and_status: u32,
    counter_low: u32,
    counter_high: u32,
    compare_0: u32,
    compare_1: u32,
    compare_2: u32,
    compare_3: u32,
}
...
    let mut t0: i32;
    let mut t1: i32;

    let system_timer = mmap.as_ptr() as *const SystemTimer;

    loop {
        t0 = unsafe { system_timer.read_volatile().counter_low  } as i32;
        thread::sleep(one_second);
        t1 = unsafe { system_timer.read_volatile().counter_low  } as i32;
	    println!("Elapsed: {}", t1 - t0);
    }

Works like a charm:

$ sudo ./target/debug/pi-system-timer
Elapsed: 1000094
Elapsed: 1000132
Elapsed: 1000092
...

Didn't have any luck with that volatile_ptr thing...

OK, so I got this thing to work with a volatile_ptr as well:

#[derive(Debug, Copy, Clone)]
struct SystemTimer {
    control_and_status: u32,
    counter_low: u32,
    counter_high: u32,
    compare_0: u32,
    compare_1: u32,
    compare_2: u32,
    compare_3: u32,
}
...
...
let system_timer = mmap.as_ptr() as *const Volatile<SystemTimer>;

loop {
    let t0 = unsafe { (*system_timer).read().counter_low };
    thread::sleep(one_second);
    let t1 = unsafe { (*system_timer).read().counter_low };
    println!("Elapsed: {}", t1 - t0);
}

I'm not sure I like it...

It required adding the Copy trait to my SystemTimer, which seems a bit against the grain as there is only ever one of those in a system.

It required dereferencing the pointer, (*system_timer), which is ugly and also seems a bit odd because using read_volatile() did not require a dereference.

Problem is that neither of these approach compiles as --release using the nightly, failing with a SEGV:

$ cargo build --release
   Compiling libc v0.2.62
error: Could not compile `libc`.

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.62/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=e86e46f59313e545 -C extra-filename=-e86e46f59313e545 --out-dir /home/pi/pi-system-timer/target/release/build/libc-e86e46f59313e545 -L dependency=/home/pi/pi-system-timer/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)

What's the best way to report the issue?

I recommend wrapping your struct in a type with dedicated functions and hiding those calls to read_volatile away.

Alice,

Yes, that is a great idea and on my TODO list.

don't forget to use #[repr(C)] for the structure, otherwise the compiler is allowed to reorder fields, which could cause quite some havoc here !

another suggestion would be to have the individual fields be volatile, instead of the struct—right now, to access one field it will copy the entire structure (that's why the need for the Copy trait at that level)

for additional inspiration you could take a look at svd2rust output for RegisterBlock, for example for the e310x (it's fairly difficult to read, though)
it uses vcell::VolatileCell<u32> (crate vcell) for the individual registers fields to abstract the volatile accesses

1 Like

alice, medusacle,

wrapping your struct in a type - Check.

use #[repr(C)] for the structure - Check.

have the individual fields be volatile - Check.

I had started to wonder about the scope of the volatile I had there. Certainly don't want to be hitting all the registers all the time.

That e310x thing is interesting. Perhaps the ugliest code I have seen for ages. Well, it is the output of a code generator. Just now I'm in the mood to do things for myself, from the ground up, with the minimum of support from ready made crates. Just to get more familiar with the language and the compiler error messages :slight_smile:

Anyway here is how my System Timer interface looks now:

pub struct SystemTimer {
    mmap: MmapMut,
}

#[repr(C)] 
struct Registers {
    control_and_status: Volatile<u32>,
    counter_low: Volatile<u32>,
    counter_high: Volatile<u32>,
    compare_0: Volatile<u32>,
    compare_1: Volatile<u32>,
    compare_2: Volatile<u32>,
    compare_3: Volatile<u32>,
}

impl SystemTimer {
    fn map_registers() -> MmapMut {
        // All peripherals can be described by an offset from the Peripheral Base Address, which starts at:
        // 0x20000000 on the Raspberry Pi model 1
        // 0x3F000000 on the models 2 and 3.
        const PERIPHERAL_BASE_ADDRESS: u64 = 0x3F000000;

        // The System Timer is a hardware clock that can be used to keep time and generate interrupts after a certain time.
        // It is located at offset 0x3000 from the peripheral base.
        const SYSTEM_TIMER_OFFSET: u64 = 0x3000;

        let f = fs::OpenOptions::new().read(true)
                                    .write(true)
                                    .open("/dev/mem")
                                    .unwrap();

        // Create a new memory map builder and build a map.
        let mmap: MmapMut = unsafe {
            MmapOptions::new()
                        .offset(PERIPHERAL_BASE_ADDRESS + SYSTEM_TIMER_OFFSET)
                        .len(4096)
                        .map_mut(&f)
                        .unwrap()
        };
        mmap
    }

    pub fn read_counter_low(&self) -> u32 {
        unsafe {
            let registers = self.mmap.as_ptr() as *const Registers;
            (*registers).counter_low.read()
        }
    } 
}

pub fn take_system_timer() -> SystemTimer {
    let mmap = SystemTimer::map_registers();
    SystemTimer {
        mmap: mmap,
    }
} 

Next up is to make it a singleton as described for peripherals in the embedded Rust book.

Thanks for the tips.

If you're not reading this frequently, perhaps you can just seek and read in /dev/mem rather than mapping it.

I wonder how seeking and read/writing in /dev/mem like that actually works. My fist guess is that, as that is a file operation, whole file blocks are getting read/written at some point rather than hitting individual 32 bit registers. In general this does not sound like what we want for peripherals. Probably OK for just reading this timer though. Then there is the matter of accuracy, with memory mapping the latency in reading the timer is only a pointer indirection.

Anyway, this is all part a some experiments in another direction. By isolating a core from the Linux kernel so that it cannot schedule any processes there and by disabling any interrupts on that core, we basically have a bare-metal environment to run in. We have a means of doing high speed/real-time things with no kernel scheduling jitters. We can experiment with our own task scheduling and so on. To that end direct access to peripherals via memory mapping is the way to keep the kernel out of things.

And, well, I'm here to learn some Rust. This has been another great exercise, which again confirms to me that Rust is great for systems programming, it offers such a lot as a language whilst also matching the performance of C at that same task.