Lol: A new Raft library

[akiradeveloper]

This is an announcement of my new project "lol" (this name is not a joke. one of the two L means log) which is a generic Raft library based on tokio. The design is mostly fixed and it is good time to share.

akiradeveloper/lol

Raft is a consensus algorithm in distributed environment: Nodes in cluster make a consensus by consistent log replication. This algorithm is used in famous application like etcd.

This library is useful if you want to make a distributed application that share the same log (or state) in the same order strictly. To make an application you need to implement RaftApp trait but it has only 4 functions and easy to implement. (kvs/kvs_server.rs is an example)

For detail, please read these documents:

• User Guide: Design and features are explained.
• Doc: Doc.rs

Thank you for reading.

[zicklag]

Hey, looks cool. I might try that some day if I ever get to making one of my distributed orchestration tools.

I like that you've already got a guide up for it.

[akiradeveloper]

Thank you. I am improving this library day by day so design would still change a bit but the document-level concept is fixed. Please keep watching.

[akiradeveloper]

Announcement of version update: v0.5.0 is released

The biggest change in this release is introducing a new concept SnapshotTag.

SnapshotTag is an indirection to actual snapshot resource. A SnapshotTag is just a sequence of byte which is able to express any identity pointing to a resource. For example, if the resource is a local file it could be a file path and if the resource is on AWS S3 it could be an object key.

The SnapshotTag is maintained in RaftStorage which is an abstraction of backend storage to store log, vote, etc. lol already provides two tested implementations: in-memory and persistent. Persistent one is backed by RocksDB. If you have an idea of more sophisticated implementation PR is welcome.

For more information, please refer to doc.rs and project wiki.

Thanks.

[akiradeveloper]

Hi Rust guys. Today, v0.6.0 is released.

After v0.5.0, I went through 0.5.3 with minor fixes / improvements. During v0.5 period, my company project started to run our application in test environment and it seems everything is going well.

v0.6.0 includes breaking design changes. The design in v0.6 becomes less coupled with tonic infrastructure so the library doesn't care about the actual transport being used. Below is the example of how you can build your own distributed application in Raft. Thank you for reading.

// Implement RaftApp for YourApp!
struct YourApp { ... }
impl RaftApp for YourApp {
    ...
}
let app = YourApp { ... };
let storage = ...; // Choose a backend from lol_core::storage
let core = RaftCore::new(app, storage, config, ...);
let service = lol_core::make_service(core);
tonic::transport::Server::builder()
  .add_service(service)
  .serve(socket).await;

[akiradeveloper]

This is an announcement of the new release.

Since 0.6.0, especially these tasks are done:

• 0.6.2
  • Use adaptive failure detection algorithm to detect leader failure.
• 0.7.0
  • Based on Tokio 1.0 and Tonic 0.4.
  • Implement Pre-vote phase for better stability.
  • Say goodbye to messagepack. Say hello to Bincode.
  • Document is now mdBook-based.
• 0.7.1
  • Code coverage is now measurable (using LLVM's instrument-coverage)
  • Started a performance analysis project named lol-perf (using cargo-flamegraph and cargo-profiler)

I am thinking of Multi-Raft as the next step. Anyway, I will keep improving it.

[akiradeveloper]

0.7.2 is released.

In 0.7.2, I have received the first PR. Implement Tune message by fteychene · Pull Request #173 · akiradeveloper/lol · GitHub

I think receiving PRs is the best part of the OSS. I am happy with this. Thank you.

lol's development is very slow now but there are lot of ideas in Issue page. Multi-Raft is one of them. I will appreciate you leave some comments on them because I want to make a decision based on the community feedback.