I've always wanted to learn distributed systems and now I have a toy project that can benefit from this. My question is very general in the sense it asks for all kinds of ideas, algorithms to use, crates available etc.
I want to write an app that is mostly able to work without the central server (think a mobile app that is functional offline), although it does use it to sync, for backup etc. (I am not writing a P2P network). Users (or several devices of the same user) may collaborate but consistency is a very relaxed requirement - conflicts are rare (you don't need to edit your data from 2 devices simultaneously), easily resolved and errors (as in inconsistent data) are acceptable to some extent (of course, except for a tiny amount of data, which in the extreme case can be read-only in offline mode).
I do not have the requirement for "hard" offline mode (eg. days), my main goal is speed, the "client" needs to have the data locally, both to offload the server and to increase analytical speed (will need to scan large parts of the data per operation) and for the purpose of sharding - one user does not need the data of other users. The central (backup) storage can be slow (no need to support analytical queries). Also the central server does not need to be able to read user data (privacy).
To make things more complicated, the total amount of data does not fit in a single web request - probably many megabytes, maybe gigabytes. So it's not acceptable to sync everything always, it has to have granularity (and the data naturally does provide this property)
I haven't done anything like this. Neither in Rust nor in any other language.
I would prefer to use ready libraries if available, or ready databases/services.
So all kinds of thoughts are very welcome!