Relocation out of range when calling panic

I'm writing firmware for bare-metal x86 (x86_64-unknown-none) and keep hitting relocation out of range errors that I haven't been able to solve by messing with the linker script, code generation options or linker options.

The specific error I'm seeing seems to be related to sections prefixed with .rodata..Lanon:

rust-lld: error: /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/597092d75453df74/out/linker_panic_codegen.a4oktnzon0hpo8l8o9gb7g9f2.0r37x4l.rcgu.o:(function reset: .text.reset+0x4): relocation R_X86_64_32S out of range: 2147483648 is not in [-2147483648, 2147483647]; references section '.rodata..Lanon.ab7aeef485f60b46b136ed38406e9087.0'
          >>> referenced by main.rs:14 (src/main.rs:14)

          rust-lld: error: /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/597092d75453df74/out/linker_panic_codegen.a4oktnzon0hpo8l8o9gb7g9f2.0r37x4l.rcgu.o:(function reset: .text.reset+0xb): relocation R_X86_64_32S out of range: 2147483704 is not in [-2147483648, 2147483647]; references section '.rodata..Lanon.ab7aeef485f60b46b136ed38406e9087.2'
          >>> referenced by main.rs:14 (src/main.rs:14)

And viewing those sections with objdump they seem to be somewhat related to panic messages:

> rust-objdump --all-headers --full-contents target/x86_64-unknown-none/debug/build/linker_panic_codegen/597092d75453df74/out/linker_panic_codegen.a4oktnzon0hpo8l8o9gb7g9f2.0gdqlaw.rcgu.o
...
Contents of section .rodata..Lanon.ab7aeef485f60b46b136ed38406e9087.0:
 0000 696e7465 726e616c 20657272 6f723a20  internal error:
 0010 656e7465 72656420 756e7265 61636861  entered unreacha
 0020 626c6520 636f6465                    ble code
Contents of section .rodata.str1.1:
 0000 7372632f 6d61696e 2e727300           src/main.rs.
Contents of section .rodata..Lanon.ab7aeef485f60b46b136ed38406e9087.2:
 0000 00000000 00000000 0b000000 00000000  ................
 0010 0e000000 05000000                    ........
Contents of section .rela.rodata..Lanon.ab7aeef485f60b46b136ed38406e9087.2:
 0000 00000000 00000000 01000000 05000000  ................
 0010 00000000 00000000                    ........
...

I've found a few different ways to fix the error, none of which work for my use case unfortunately:

  1. Commenting out the single unreachable!() expression (I'd like to be able to panic!)
  2. Changing the codegen backend to cranelift, which seems to generate these sections under .rodata instead of .rodata..Lanon (cranelift doesn't support the x86-interrupt ABI that I'm trying to use)
  3. Changing the ORIGIN of the memory region in the linker script to below 2048M (the code is loaded just under 4096M according to the x86 spec)

I have the feeling it's an issue with my linker script, but running -Clink-arg=--print-map seems to show everything being within a range of a couple hundred bytes, so I'm not sure where the relocation error is coming from. If anyone knows what might be causing this, or anything else I could try, I would very much appreciate it :slight_smile:


Source code
> cargo build --target x86_64-unknown-none -Zbuild-std=core
// src/main.rs

#![no_std]
#![no_main]

use core::panic::PanicInfo;

#[panic_handler]
fn panic(_panic_info: &PanicInfo) -> ! {
    loop {}
}

#[unsafe(no_mangle)]
fn reset() -> ! {
    // Changing this to `loop {}` fixes the error
    unreachable!()
}
/* src/link.ld */

MEMORY {
    /* Lowering the origin any further fixes the error (but the firmware needs to be loaded just under 4GiB in this case) */
    ROM : ORIGIN = 2048M, LENGTH = 512K
}

ENTRY(reset);

SECTIONS {
    .eh_frame     : { *(.eh_frame);     *(.eh_frame.*);     } > ROM
    .eh_frame_hdr : { *(.eh_frame_hdr); *(.eh_frame_hdr.*); } > ROM
    .rodata       : { *(.rodata);       *(.rodata.*);       } > ROM
    .text         : { *(.text);         *(.text.*);         } > ROM
    .got          : { *(.got);          *(.got.*);          } > ROM
    .data.rel.ro  : { *(.data.rel.ro);  *(.data.rel.ro.*);  } > ROM
    .data         : { *(.data);         *(.data.*);         } > ROM
    .bss          : { *(.bss);          *(.bss.*);          } > ROM

    .reset : {
        KEEP(*(.reset));
    } > ROM
}

# .cargo/config.toml

[target.x86_64-unknown-none]
rustflags = [
    "-Clink-arg=-Tsrc/link.ld",
    "-Crelocation-model=static",
    # Changing this to cranelift fixes the error
    "-Zcodegen-backend=llvm",
]

Try using -Ccode-model=large.

(or medium)

Thank you so much, that's exactly what I was looking for! I thought I'd already tried that but must've gotten mixed up somewhere along the way :sweat_smile:

My understanding is that the large code model allows the program to address a much larger range, but I'm still a bit confused as to why it's needed in the first place? I'd expect the .rodata..Lanon sections would be caught by *(.rodata.*); in the linker script and placed with the rest of the code, so the relocation being so far away is quite confusing ...

With the default "small" model, all addresses (code and data) are expected to be in the first 2 GiB of address space; the compiler will choose instructions accordingly, and the chosen instructions are allowed to truncate addresses to 31 bits. The "large" model has no such assumption - addresses can be anywhere in address space, and will not be truncated.

There's also the "medium" model (code must be in first 2 GiB, data can be anywhere), and "kernel" (everything is in the last 2 GiB of address space).

Thanks, that makes sense! Apologies for not wording my question well, what I'm confused about is more how I'm seeing addresses outside that 2GiB range.

What I'd expect to happen with my linker script is that all the code/symbols/sections etc is placed contiguously, starting at the ROM origin. In the example from the original post I'd expect a range of a couple hundred bytes or so. Without the call to unreachable!, or using cranelift, everything is placed exactly as I'd expect. My understanding is that for whatever reason, the .rodata..Lanon sections don't follow this pattern and are placed far away from the rest of the binary (and seemingly contrary to the linker map).

What would I need to change in that above code to have these .rodata..Lanon sections be placed in that same small contiguous region of memory, alongside the rest of the binary? Or is there something else I could change to get rid of the linker errors while staying within the small/kernel code model?

You're forcing the static relocation model, which combines with the small model to say that the linker and compiler may use absolute addressing and assume that all addresses are below 2048M. You're also telling the linker that the base address of your ROM is 2048M, and that all addresses in here are thus above 2048M.

Choices here:

  1. Switch to the "large" model, which says that you cannot assume that addresses are below 2048M. You'll still have absolute addressing used in the code, but now the compiler and linker must use the full 64 bits of the address, not just 31 bits.
  2. Switch to the pic relocation model, which prohibits absolute addressing for symbols (like the panic machinery), and requires everything to be within a 2048M range (so that a 32 bit offset can be used), but allows the base to be something other than 0.
  3. Use panic=abort to shed the unwind machinery, and abort on panics instead.

Thanks for such a patient and thorough explanation, I really do appreciate it! I'm sorry for not communicating the intent of my question clearly enough, what I'm confused by is not so much how to allow these large relocations but why they're so large in the first place when I'd expect all addresses to be within a 512KiB range?

The firmware is executed starting at 0xFFFFFFF0 (4GiB - 16), so if the binary fits within 512KiB I'd like everything to be laid out in the range of (4GiB - 512KiB)..4GiB or 0xFFF80000..0xFFFFFFFF. I don't have any control over that starting address and with such a small binary I feel like there should be no problems with the size of relocations, and that's part of why I'm explicitly setting -Crelocation-model=static. Since I'm using the kernel code model (at least according to rustc --print target-spec-json --target x86_64-unknown-none -Zunstable-options) I'd expect everything to work fine.

The intent of my linker script is to ensure the entire binary is laid out within that range - addresses starting at 0xFFF80000 and ending at 0xFFFFFFFF. What confuses me so much about these .rodata..Lanon sections is that they don't seem affected by the linker script unlike the rest of my code, as if they were the size of the relocations wouldn't be so large.

My understanding is that a "relocation" is when one section/symbol references another, and the size of that relocation is the difference in addresses between those two sections/symbols. The error I'm seeing is that code in .text is referencing panic-related messages in .rodata..Lanon, and the difference between those suggests .rodata..Lanon is being placed somewhere below 2GiB. If .rodata..Lanon was placed according to the intent of my linker script, I'd expect the size of that relocation to be within 512KiB, as the whole binary should be in that range.

The most confusing thing of all is that when I add -Clink-arg=--print-map to my RUSTFLAGS, the linker map shows the VMA and LMA of those sections are exactly where I would expect them to be, inside .rodata and well within the 512KiB range.

I'm beginning to suspect this might be something to do with my usage of build-std, but I don't see any problems with other parts of core so that's just a hunch. If anyone has any suggestions I'd really appreciate it!

Linker script
MEMORY {
    ROM : ORIGIN = 0xFFF80000, LENGTH = 512K
}

ENTRY(reset);

SECTIONS {
    .eh_frame     : { *(.eh_frame);     *(.eh_frame.*);     } > ROM
    .eh_frame_hdr : { *(.eh_frame_hdr); *(.eh_frame_hdr.*); } > ROM
    /* I'd expect the following line to apply to those `.rodata..Ldata` sections, and place them with the rest of the binary
     (the linker map indicates that it IS doing what I'd expect??)
    */
    .rodata       : { *(.rodata);       *(.rodata.*);       } > ROM
    .text         : { *(.text);         *(.text.*);         } > ROM
    .got          : { *(.got);          *(.got.*);          } > ROM
    .data.rel.ro  : { *(.data.rel.ro);  *(.data.rel.ro.*);  } > ROM
    .data         : { *(.data);         *(.data.*);         } > ROM
    .bss          : { *(.bss);          *(.bss.*);          } > ROM

    .reset : {
        KEEP(*(.reset));
    } > ROM
}
Linker map
                       VMA              LMA     Size Align Out     In      Symbol
                  fff80000         fff80000       50     8 .rodata
                  fff80000         fff80000       28     1         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/7f27e90b0d4b0e27/out/linker_panic_codegen.bzen6lgzk8z8u7lbj60zjjx5g.0xtyxcy.rcgu.o:(.rodata..Lanon.d21f2c76e04b9f8640a60b7c54d578eb.0)
                  fff80028         fff80028        c     1         <internal>:(.rodata.str1.1)
                  fff80038         fff80038       18     8         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/7f27e90b0d4b0e27/out/linker_panic_codegen.bzen6lgzk8z8u7lbj60zjjx5g.0xtyxcy.rcgu.o:(.rodata..Lanon.d21f2c76e04b9f8640a60b7c54d578eb.2)
                  fff80050         fff80050       6b    16 .text
                  fff80050         fff80050        4     4         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/7f27e90b0d4b0e27/out/linker_panic_codegen.bzen6lgzk8z8u7lbj60zjjx5g.0xtyxcy.rcgu.o:(.text._RNvCsjHpjAFo4bi0_7___rustc17rust_begin_unwind)
                  fff80050         fff80050        4     1                 __rustc::rust_begin_unwind
                  fff80060         fff80060       19    16         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/linker_panic_codegen/7f27e90b0d4b0e27/out/linker_panic_codegen.bzen6lgzk8z8u7lbj60zjjx5g.0xtyxcy.rcgu.o:(.text.reset)
                  fff80060         fff80060       19     1                 reset
                  fff80080         fff80080        e    16         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/core/5b610cd027d6b15d/out/libcore-5b610cd027d6b15d.rlib(core-5b610cd027d6b15d.core.3e4e3192b8a8540c-cgu.12.rcgu.o):(.text.unlikely._RNvNtCs5lEkclErxmG_4core9panicking5panic)
                  fff80080         fff80080        e     1                 core::panicking::panic
                  fff80090         fff80090       2b    16         /home/finchie/Projects/Rust/linker_panic_codegen/target/x86_64-unknown-none/debug/build/core/5b610cd027d6b15d/out/libcore-5b610cd027d6b15d.rlib(core-5b610cd027d6b15d.core.3e4e3192b8a8540c-cgu.12.rcgu.o):(.text.unlikely._RNvNtCs5lEkclErxmG_4core9panicking9panic_fmt)
                  fff80090         fff80090       2b     1                 core::panicking::panic_fmt
... (this is followed by a bunch of (hopefully unrelated) debug info

you may have missed it, but it is pointed out by @farnz, the cause of the error is the COMBINATION of the code model, AND the high address.

that's inaccurate.

there are different types of relocation records. IIRC, R_X86_64_32S writes the absolute address of the symbol, while the more common R_X86_64_PC32 writes the relative distance between the target and the current instruction.

because the absolute address of the panic string is in the upper address space, it is out of range of signed 32 integer.

This paragraph is self-contradictory·

When you set -Crelocation-model=static, you tell the compiler that it can use absolute addressing for all symbols. When you choose the "small" model, you tell the compiler that absolute addresses are in the range (0..2 GiB) - for the "kernel" model, absolute addresses are in the range (16 EiB - 2 GiB)..16 EiB).

Your linker script puts symbols in the range 2 GiB..(2 GiB + 512 KiB). This is outside the reach of absolute addresses in either the "small" or "kernel" models, since it's above the upper limit for the "small" model, and below the lower limit for the "kernel" model.

Relocations don't have a size per-se; they're what the compiler puts into the code to say "I only know this by its symbol name - linker, please fill in the address". In this case, you're hitting up against there being four broad classes of relocations in x86-64:

  1. Relocations that supply an address as a constant 32-bit signed offset to the value of RIP, or another value (e.g. the location of the start of the GOT[1]) that the program will know when it tries to access this address.
  2. Relocations that supply an address as a constant 64-bit signed offset to the value of RIP, or to another value that the program will know when it tries to access this address. This, and 32-bit offsets, get bundled up as "relative" addressing.
  3. Relocations that supply an address as a sign-extended 32-bit constant. This inherently can only address 0..2 GiB and (16 EiB - 2 GiB)..16 EiB), because sign-extended 32-bit constants only produce 64-bit addresses in those ranges.
  4. Relocations that supply an address as a 64-bit constant. This, and 32-bit constants, get bundled up as "absolute" addressing.

The memory model you choose determines the size of the constant in the relocation - "small" and "kernel" require 32-bit constants, "medium" requires 32-bit constants for code and 64-bit constants for data, and "large" requires 64-bit constants for everything.

The relocation model you choose determines which relocation types are allowed: static tells the compiler that it can use absolute addressing, while pic tells the compiler that it can only use relative addressing.

In your case, you've told the compiler, via the static relocation model, that it can use absolute addressing. You've told it, via the "kernel" memory model (although you'd have the same problem with the "small" memory model), that it must use sign-extended 32-bit constants. When the compiler chooses an absolute addressing instruction, and tells the linker to fill in a relocation here, the linker gets in trouble - it needs to find a 32-bit constant that sign-extends to a 64-bit value between 231 and 231 + 219 to comply with your linker script. However, sign-extended 32-bit constants cover the range 0 to (231 - 1) and (264 - 231) to (264 - 1). It thus fails, because it cannot do what you've asked it to do - these ranges do not overlap.

Hence the suggestion to use the pic memory model, which fits your understanding as-is (since the linker meets the compiler's request if it can find an offset between RIP and the symbol, rather than needing to find a 32-bit constant that will sign extend to the right address) and thus resolves the problem.


  1. Global Offset Table. ↩︎

I think you had the wrong idea about the address space for the kernel code model?

kernel modal uses the same 32 bit offset just like the small model, except the code generator assume all symbols are located within the "negative" range of the 32-bit signed offsets in the 64-bit address space, in other words, it starts at 0xFFFF_FFFF_8000_0000.

but your ROM region starts at 2048M, which is 0x0000_0000_8000_0000, so absolute addressing mode in kernel or small code model will never work.

so, your options are:

  • give up absolute addressing mode, i.e. don't use -C relocation-model=static
  • use different code model that can encode absolute address in your ROM region
  • move the VMA to the kernel code modal range, which you said is out of your control

Thank you both, this is making a lot more sense! As @nerditation pointed out I think my main issue was thinking the kernel code model used an address range from 2GiB..4GiB, which wasn't true. I assume I didn't have any issues until now as I was only generating code that used relative addressing, and the panic machinery was just the first thing to break my incorrect setup.

Using -Crelocation-model=pic seems to fix the errors I was seeing, although it seems to break my inline assemly. If I add the following code, I get error: relocation R_X86_64_32 cannot be used against local symbol; recompile with -fPIC, even though I've set -Crelocation-model=pic. Is there some codegen/linker argument that I might be missing?

global_asm!(
    "global_descriptor_table:",
    "   .quad 0",
    ".global reset",
    "reset:",
    "   movl $global_descriptor_table, %eax",
    options(att_syntax)
);

the linker's suggestion of setting -fPIC is a bit misleading here. the PIC option is for the code generator of C or rust compiler frontend, but your R_X86_64_32 relocation record is originated from the inline assembly code.

the reason is this instruction:

movl $global_descriptor_table, %eax

it encodes the local symbol $global_descriptor_table using absolute addressing mode, so the assember will emit a record of type R_X86_64_32. to use rip relative addressing mode, change it to:

leal global_descriptor_table(%rip), %eax

Note, too, that if the GDT ends up too far from the instructions (e.g. RAM at 0, code at 4 GiB - 512 KiB), you may need leaq global_descriptor_table(%rip), %rax (64-bit offset) instead of leal. It's important, FWIW, that your linker script is accurate here, since you're on bare metal - if the linker knows that RAM is at 1 MiB upwards, and believes your code is at 2 GiB upwards when it's actually nearer 4 GiB, it may generate the wrong RIP-relative offset.

Similarly, if the GDT is going to be in a fixed location, but the firmware needs to be relocatable in address space, you may prefer to use movq $global_descriptor_table, %rax - thus loading a 64-bit constant, not a 32-bit constant. This should generate a full 64-bit relocation that doesn't depend on the offset between firmware code and RAM.

Since I've been messing around with x64 emit recently, is there any tricks with DS that could be pulled here? With my limited understanding, I think you could set that to 2GB and it would theoretically work, but I don't know if that trips up other code (especially heap allocated data)


I guess it should actually be CS, but I'm pretty sure that'd be a nightmare to get eg panic to use

You can't set CS, DS, ES or SS in x64 - they must be zero. See, for example, the Intel SDM, volume 1, section 3.3.4, where it says:

64-bit mode — Segmentation is generally (but not completely) disabled, creating a flat 64-bit linear-address space. Specifically, the processor treats the segment base of CS, DS, ES, and SS as zero in 64-bit mode (this makes a linear address equal an effective address). Segmented and real address modes are not available in 64bit mode.

That rules out tricks with DS or CS to make the small memory model and static relocations work for the range 2048M to 4096M; you can't use the segment registers to adjust the final address accessed.

You could probably reuse one of FS or GS to make some otherwise-impossible accesses work; Linux only uses FS for thread-local storage (reserving GS for userspace), while Windows reserves both and currently uses GS for access to userspace-visible OS data structures like the PEB and TEB, so this would be specific to bare metal ABIs.

Ah, that makes some sense: I think I had some vague memories of NaCl tricks still floating around in there, along with a few x64 disassemblers still printing the ds: prefix.

A bit of a pity when we still have cruft like retf and bare disp32 ModR/M encodings still hanging around, being able to use selectors for pointer compression would be quite handy!


Edit: it's pretty confusing, though, that the far call docs (ie. the FF/3 encoding) are pretty clear it does make use of a provided code segment "descriptor" even in 64-bit mode. I suspect I am just not understanding the difference here between, say, CS and the Code Segment Descriptor though. Probably not a great idea to be looking this up this late at night for me, though :sweat_smile:

Coming back to thinking about this - there's the R_X86_64_32 relocation, which is the same as R_X86_64_32S except that it uses a constant that is zero-extended, instead of sign-extended.

At least in theory, that could be used to create yet another code model, this one addressing 0 to 4 GiB instead of 0 to 2 GiB or 16 EiB - 2 GiB to 16 EiB.

Whether there's enough of a practical use case for this to make it worth the effort is a completely different matter, of course.

And it's very confusing in the docs; 3.2.4 in the Intel SDM volume 3 is the clearest explanation, where it says:

In IA-32e mode of Intel 64 architecture, the effects of segmentation depend on whether the processor is running in compatibility mode or 64-bit mode. In compatibility mode, segmentation functions just as it does using legacy 16-bit or 32-bit protected mode semantics.

In 64-bit mode, segmentation is generally (but not completely) disabled, creating a flat 64-bit linear-address space. The processor treats the segment base of CS, DS, ES, SS as zero, creating a linear address that is equal to the effective address. The FS and GS segments are exceptions. These segment registers (which hold the segment base) can be used as additional base registers in linear address calculations. They facilitate addressing local data and certain operating system data structures.

Note that the processor does not perform segment limit checks at runtime in 64-bit mode.

In other words, while the descriptor may be similar to 32-bit mode, and still contains a base and limit, the processor just ignores it.

yeah, I think it has very little real world use case.

the R_x86_64_32S relocation is consistent with instruction encoding in 64bit mode, where a 32bit immediate address/displacement is sign extended during decoding to form the 64bit address.

but to support zero extended addresses, either a 0x67 prefix needs to be emitted, which forces the address to be calculated in 32bit mode, or the immediate value needs to be loaded into a register first, which could potentially increase register pressure, given the x86 architecture's already scarce general registers, compared to typical RISC machines.

for normal application code, if only 4GiB address space is needed, the program can just be compiled as 32bit code (runs in compatible mode). however, for a kernel, compatible mode is practically impossible (note, I'm talking about "compatible" mode, not the "protected" mode), because even if it's possible for the kernel "executive" layer to run in compatible mode, the interrupts must be delievered to "long" mode handlers.