I've spent the last while building SLATE, a no_std, no-alloc key-value engine for the setting most storage engines quietly assume away: bare NOR flash, no filesystem underneath, no battery-backed cache, well under 100 KiB of RAM, and a supply rail that can die in the middle of a page program. Think ESP32-class microcontrollers up through small Linux boards. I wanted to share it here because the embedded category is exactly the crowd that runs into this problem, and I'd like feedback from people who've fought it before.
The problem I kept running into
Every project I'd shipped on flash eventually asked the same question: what does the device actually contain after a crash? Filesystems like LittleFS answer this reasonably well; raw flash with a hand-rolled log usually doesn't, or answers it by assertion rather than proof. I wanted an engine where the answer isn't "probably fine" but a statement I could check.
What SLATE actually guarantees
Three properties, each with a written proof in docs/specification.md and an empirical run backing it:
- Prefix durability. After any number of power failures at any instant, recovery returns exactly the state produced by some prefix of the acknowledged write sequence — every write acknowledged before the last crash is there, nothing acknowledged-but-not-durable sneaks in. I ran 20,000 power-loss trials with the cut landing at a uniformly random byte offset; zero violations.
- Rollback resistance. If an attacker with physical access swaps the flash for an older authentic image, recovery detects it — accepting a stale epoch requires forging a MAC. This is per-epoch (a rollback within the current epoch isn't distinguishable, bounded by a parameter Θ), and I say so rather than overclaim. 5,000 splice attacks, all rejected.
- Erasure tolerance. Segment parity is MDS (Reed–Solomon over GF(2⁸)), so any
n-kdeclared block erasures reconstruct byte-exactly. I exhaustively checked every erasure pattern for RS(12,8): all 794 patterns within the code's distance recover exactly, all 792 beyond it are correctly refused — 1,586 patterns, zero wrong bytes.
On top of that there are closed forms that actually help you size a device: constant-time index lookup, recovery time linear in the post-checkpoint tail only (not total stored volume), steady-state write amplification of 1/(1-u), and an optimal commit-batch size B* = sqrt(2·λ·A/c) that landed within 5.35% of the measured optimum in my sweep.
The DX bits I cared about
- Commit markers are the acknowledgement point, not the record write itself. A
putbuffers into the open batch; nothing is acknowledged untilcommit()makes the marker durable. That's the mechanism that makes "recovered state is a prefix" true by construction, not by convention — andb_commit(batch size) is your one dial between throughput and durability latency. - Heapless
no_stdcore,#![forbid(unsafe_code)], zero dynamic allocation — every buffer is a fixed array or a caller-supplied slice. This was non-negotiable for the ESP32 target. - Async-native, blocking-projected. Every algorithm is written once as
async fnover anAsyncFlashtrait, and the blocking API is a one-line projection over it — so you're not maintaining two implementations that drift. - A partial-key cuckoo index: fixed arena, small stash, exactly
2b+s= 16 slot probes per lookup regardless of load factor, at ~4.2 bytes of RAM per key.
Minimal usage:
use slate_kv::db::{Db, KeySource, Options, Profile};
let opts = Options { profile: Profile::Pi, ..Options::default() };
let mut db = Db::open(
std::path::Path::new("./slate_db.bin"),
KeySource::Bytes([0x42u8; 32]),
opts,
)?;
db.put(b"sensor_1", b"23.5 C")?;
db.commit()?; // acknowledged here, not before
if let Some(val) = db.get(b"sensor_1")? {
println!("sensor_1 = {}", String::from_utf8_lossy(&val));
}
There's also a stable C ABI (slate-kv-ffi) with a generated header, and a thin Go/cgo binding, if you need it outside Rust.
What's not solved yet
This is the part I'd rather lead with than bury, since a proof is only as good as what it doesn't cover:
- Reclaimed space isn't reusable yet. The log head can't wrap into freed segments, so a long-running device eventually halts with most of its segments free. This is the biggest open item.
- RAM is over budget on ESP32. The shipped config needs ~81 KiB resident against a 64 KiB target, mostly a checkpoint buffer holding the full serialized index.
n_buckets = 1024is the largest config that actually fits 64 KiB. - Mount replay doesn't yield. Recovery is still on the blocking flash trait, so replaying 8,192 records is one uninterruptible ~4.6 s span.
- Sequential keys hurt the index. Keys like
sensor_000123push the collision rate to ~5.7x the theoretical bound; well-mixed keys stay under it. - Rollback protection is only as strong as your counter. Hardware monotonic counter gives you the full guarantee, a flash-backed counter is best-effort, and with neither the engine tells you it has no protection rather than pretending it does.
Where it runs today
slate-kv-core / -crypto / -erasure / -hal are the no_std engine crates; slate-kv is the std wrapper for Linux/Pi-class boards; targets/esp32 is bare-metal esp-hal firmware with a QEMU crash-injection suite and a Wokwi hardware scenario running in CI. Repo layout and all the crates are in the README.
Repo: GitHub - ja7ad/slate: SLATE is a key-value storage engine for edge computing, from bare-metal microcontrollers like the ESP32 up to boards like the Raspberry Pi. It's built around four goals that usually fight each other: a tiny memory footprint, good performance, low energy use, and real at-rest security. · GitHub
Spec with every proof and every conformance number: slate/docs/specification.md at main · ja7ad/slate · GitHub
I'd genuinely like pushback — on the epoch-bounded rollback model, on the RAM budget, on whatever else looks off. If you've built something in this space I'd also like to compare notes on where the log-structured + parity approach falls down against what you've tried.