I am evaluating various embedded databases for use in a toy project and so far I have narrowed it down to sled and rocksdb. But I have a hard time deciding one or the other, besides obvious factors like maturity. I have only used sqlite as an embedded database and for this case I think it will be too slow (also I am not excited about having to deal with SQL). Probably both sled and rocksdb will be "fast enough", but I have hard time figuring out what other things should I consider.
My guess is sled will be more ergonomic by supporting more rust datatypes, but on the other hand it has practically no high-level documentation and I will suffer from more bugs and is probably less future proof (will have to update my code when updating the db?)
I would be happy if you can share thoughts on the topic. Basically I am trying to find excuses to select sled over rocksdb on a project that prefers ease of use and maintainability :). Probably rocksdb is the smarter choice, still I am eager to have my mind changed.
That is not how performance optimization works. Did you make measurements with toy data that is similar to what your real data will look like? SQLite is fast, and correctly guessing about the performance of complex code on modern hardware is basically not possible.
Sled's API, as fas as I can tell, operates on raw blobs of bytes. If you want a database supporting high-level types, then you are probably looking for an ORM, such as Diesel.
It will for sure be too slow if every time I get large parts of the (numeric) data, wrangle it in memory, then write back everything. I have worked with sqlite enough to know that it's not the right tool for the job. Even a csv file is better for the job than it (although for a smaller part of the data I need DB features, so I need a DB one way or the other)
Now that I think of it, duckdb might be an option as well.
What sort of operations do you anticipate? Do you anticipate large datasets? Or is it more a question of latency (being able to access a small amount of data quickly )?
I am a big fan of LMDB: Lightning Memory-Mapped Database Manager (LMDB) .
There is a go port GitHub - boltdb/bolt: An embedded key/value database for Go. , which I have never used in production.
There is also a Rust port https://crates.io/crates/jammdb , which I have also never used in production.
Depending on the definition of "small", yes. It fits in memory, but it's still a large part of the whole database. And the program will do this constantly. Read a large portion, do calculations, update the data en bulk, write everything back. Sqlite will present too big overhead and acid guarantees that I do not need. Still I need some kind of persistence and I do not want to lose data every day - so I need some kind of database.
SQLite has a purely in-memory mode, which can be serialized to disk after the fact.
I don't know about an API in Sled that has a purely in-memory equivalent, which would basically be a
BTreeMap<Vec<u8>, Vec<u8>> anyway – but that can hardly be considered a database.
Hmm, yeah, some kind of hybrid (in-memory and serialize from time to time) mode might work...
I'm happy user of sled on my private project
Concerning "friendliness towards Rust", sled is using bytes as keys and values, and RockDB is the same. So it's up to you to serialise/deserialize any complex structured. I'm using utf8 strings as keys and bincode for serializing values.
I did not make any measurements, but it feels quite fast. If you set enough size for memory cache, everything can happen in memory.
Features I like:
- ordered iterator - helped a lot in couple of usecases
- transactions across tables
- batch operations
- otherwise it's just a Map interface
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.