Sled: pre-alpha lock-free embedded database


#1

For the last few months I’ve been spending most of my time implementing sled: a BW tree written in Rust. This is a lock-free persistent b-link tree which has extremely promising theoretical performance, with faster reads than LSM trees and faster writes than traditional B+ trees. The goal is to have a reliable, high performance embedded database, pagecache, and log that can outperform existing systems built for spinning disks.

Today I’d like to announce that the project has gotten to a place where it’s relatively safe for newcomers to contribute, as the automated testing should now catch most leaks, races, and logical errors.

I’d love to work with more people now to:

  • improve performance
  • improve the C API (this needs a total rework, and eventually some language bindings)
  • improve the testing, like adding ALICE for testing crash safety, and property based testing of the LockFreeLog and PageCache
  • and of course, documentation! anything that is not clear, please open an issue or PR, and I’d love to make it more easy to understand!

I’m happy to spend quite a lot of time with anyone who is interested in contributing to this effort to get up to speed :slight_smile:

This is a great project for people who are interested in databases, lock-free programming, high-reliability systems, performance tuning, FFI, etc… There’s a ton of stuff to do at all skill levels, and I’m willing to provide mentorship for folks who would like to get involved.

Looking forward to working with more people on this!

Also: if anyone has the nerve to try out an infantile database, let me know what you think!