I've been working on a rust-based bootloader for the raspberry pi 4 .
I'm not sure what I'm missing as I'm not well-versed with Linux but I assume folks who are, may be able to provide some pointers. So, here what I've done so far - follow the Booting AArch64 Linux guide and from what I was able to gather, there are 3 parts to booting Linux
-
-
device tree modification:
- Linux uses a device tree for hardware discovery and we use this to pass boot parameters (i.e. kernel cmd line parameters).
- We also use the DT to provide basic information about the address and size of the physical memory.
- So, rather than write my own device tree for rpi4, I decompiled an existing one, edited it with the required info and recompiled it. Its in the dts folder
-
pre-boot checks: The kernel expects a known HW state. So, we'll need to do a few additional things like
- jump to a lower privilege level (Linux calls this EL1 or exception level 1). By default, the raspberry pi starts executing at EL2 but I've transitioned into the more appropriate EL1.
- disable the memory management unit MMU. (we never turn it on, so good here).
- disable interrupts (we never enable interrupts i.e. GICv2 or the global interrupt controller is never turned on, so good again)
-
jump to kernel instruction:
- After we're done with all of the above, we load the kernel and dtb into RAM and simply jump to the kernel's entry point, passing the device tree as an argument.
- I've 2 impls for this. Both of them don’t work. Although, the generated assembly (below) checks out.
0000000000089f90 <boot_to_kernel>:
89f90: f81f0ffe str x30, [sp, #-16]!
89f94: aa0003e8 mov x8, x0
89f98: aa0103e0 mov x0, x1
89f9c: aa1f03e1 mov x1, xzr
89fa0: aa1f03e2 mov x2, xzr
89fa4: aa1f03e3 mov x3, xzr
89fa8: d63f0100 blr x8
89fac: d503205f wfe
89fb0: 17ffffff b 89fac <boot_to_kernel+0x1c>
My code manages to make it to the final jump after loading the kernel and dtb but after this point nothing happens i.e. I see no serial output or any indication that the kernel has booted (or failed to boot).
Full output:
[ 0.000027] EMMC: reset card.
[ 0.000154] Divisor = 63, Freq Set = 396825
[ 0.407636] CSD Contents : 00 40 0e 00 32 5b 59 00 00ee 9d 7f 80 0a 40 00
[ 0.411465] cemmc_structure=1, spec_vers=0, taac=0x0E, nsac=0x00, tran_speed=0x32,ccc=0x05B5, read_bl_len=0x09, read_bl_partial=0b, write_blk_misalign=0b,read_blk_misalign=0b, dsr_imp=0b, sector_size =0x7F, erase_blk_en=1b
[ 0.431096] CSD 2.0: ver2_c_size = 0xEFFD, card capacity: 32026132480 bytes or 32.03GiB
[ 0.439001] wp_grp_size=0x0000000b, wp_grp_enable=0b, default_ecc=00b, r2w_factor=010b, write_bl_len=0x09, write_bl_partial=0b, file_format_grp=0, copy=0b, perm_write_protect=0b, tmp_write_protect=0b, file_format=0b ecc=00b
[ 0.458726] Divisor = 1, Freq Set = 25000000
[ 0.463437] EMMC: Bus width set to 4
[ 0.466463] EMMC: SD Card Type 2 HC, 30542Mb, mfr_id: 27, 'SM:EB2MW', r3.0, mfr_date: 8/2017, serial: 0xc8e6576d, RCA: 0x59b4
[ 0.477748] EMMC2 driver initialized…
[ 0.481484] rpi4 version 0.1.0
[ 0.484437] Booting on: Raspberry Pi 4
[ 0.488086] Current privilege level: EL1
[ 0.491908] Exception handling state:
[ 0.495469] Debug: Masked
[ 0.498597] SError: Masked
[ 0.501724] IRQ: Masked
[ 0.504852] FIQ: Masked
[ 0.507980] Architectural timer resolution: 18 ns
[ 0.512584] Drivers loaded:
[ 0.515277] 1. BCM GPIO
[ 0.518144] 2. BCM PL011 UART
[ 0.521532] Chars written: 1426
[W 0.524575] wait duration smaller than architecturally supported, skipping
[ 0.531349] create new emmc-fat controller...
[ 1.309897] Listing root directory:
[ 1.311514] Found: DirEntry { name: ShortFileName("boot"), mtime: Timestamp(2021-05-07 16:06:08), ctime: Timestamp(2021-05-07 16:06:08), attributes: FV, cluster: Cluster(0), size: 0, entry_block: BlockIdx(16290), entry_offset: 0 }
[ 1.331106] Found: DirEntry { name: ShortFileName("BCM271~1.DTB"), mtime: Timestamp(2022-01-08 18:31:30), ctime: Timestamp(2022-01-08 18:35:56), attributes: FA, cluster: Cluster(65538), size: 25713, entry_block: BlockIdx(16290), entry_offset: 96 }
[ 1.353084] Found: DirEntry { name: ShortFileName("KERNEL8.IMG"), mtime: Timestamp(2022-01-08 18:39:00), ctime: Timestamp(2022-01-08 18:39:36), attributes: FA, cluster: Cluster(154740), size: 131192, entry_block: BlockIdx(16290), entry_offset: 128 }
[ 1.375238] Found: DirEntry { name: ShortFileName("VMLINUZ"), mtime: Timestamp(2022-01-08 18:37:12), ctime: Timestamp(2022-01-08 18:38:16), attributes: FA, cluster: Cluster(97567), size: 29272576, entry_block: BlockIdx(16290), entry_offset: 160 }
[ 1.401246] Found: DirEntry { name: ShortFileName("CONFIG.TXT"), mtime: Timestamp(2022-01-04 11:46:38), ctime: Timestamp(2021-05-07 15:07:00), attributes: FA, cluster: Cluster(1017), size: 1846, entry_block: BlockIdx(17103), entry_offset: 352 }
[ 1.420111] Found: DirEntry { name: ShortFileName("FIXUP4.DAT"), mtime: Timestamp(2021-04-30 14:01:38), ctime: Timestamp(2021-05-07 15:07:00), attributes: FA, cluster: Cluster(1036), size: 5446, entry_block: BlockIdx(17103), entry_offset: 480 }
[ 1.444596] Found: DirEntry { name: ShortFileName("START4.ELF"), mtime: Timestamp(2021-04-30 14:01:38), ctime: Timestamp(2021-05-07 15:07:02), attributes: FA, cluster: Cluster(59166), size: 2228768, entry_block: BlockIdx(54532), entry_offset: 224 }
[ 1.465552] Found: DirEntry { name: ShortFileName("SYSTEM~1"), mtime: Timestamp(2021-09-21 13:57:30), ctime: Timestamp(2021-09-21 13:57:28), attributes: DHS, cluster: Cluster(97564), size: 0, entry_block: BlockIdx(105365), entry_offset: 192 }
[ 1.486331] Get handle to `dtb` file in root_dir...
[ 1.491266] load `dtb` into RAM...
[ 1.579706] loaded dtb: 25713 bytes, starting at addr: 0x3bc33e8
[ 1.582924] EOF
[ 1.584576] Get handle to `kernel` file in root_dir...
[ 1.590823] load `kernel` into RAM...
[ 94.896272] loaded kernel: 29272576 bytes, starting at addr: 0x3bcfbe8
[ 94.900010] first 40 kernel bytes: [77, 90, 64, 250, 255, 63, 79, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 201, 1, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[ 94.914952] EOF
So, I'm kind out of ideas (until I find a HW debugger) but if anyone has any pointers on what I might be missing here, that would be much appreciated.