Distributed, atomic key value store solution wanted!

Hi,
I need some advice/pointers/suggestions. I have been searching for an atomic key-value store system such that:

  • typical search queries are not possible (so nobody should be able to search the table)
  • only a person/user that knows the key can access the stored data
  • access to data only supports get & update functions

Something like IPF(N)S but with a limited number of updatable keys. Does anything like this exists ?

thnx

Sounds like you need a modified version of a hashmap. A hashmap satisifes these two of your requirements:

1) typical search queries are not possible (so nobody should be able to search the table)
2) only a person/user that knows the key can access the stored data

You can make typical search queries virtually impossible by having a large key with a flood-safe hash function like SipHash (used by default in std::collections::HashMap) or something like Murmurhash3 (used by Oracle). The virtual impossibility arises due to the fact that you will need an exact hasher-output to get the n-th index. 128-bit hashers might do the job, although, you may need to write a custom hasher depending on your desired outcome.

As for requirement 3...

3) access to data only supports get & update functions

HashMaps have get as well as get_mut (which allows for an update or replacement by dereferencing the mutable reference obtained from get_mut)

There are plenty of hashers in crates.io that will do what you need and more.

There are many ways to accomplish what you want, but all it'l take is some creativity

1 Like

Creating a new type wrapper around hashmap sounds fun something like this

struct KVStore<K, V>(HashMap<K, V>);

impl<K, V> KVStore<K, V>
where
    K: Eq + Hash,
{
    fn new(vals: Vec<(K, V)>) -> Self {
        let mut inner = HashMap::new();
        for (k, v) in vals {
            inner.insert(k, v);
        }
        KVStore(inner)
    }
    fn get(&self, key: &K) -> Option<&V> {
        self.0.get(key)
    }
    
    fn update<F: Fn(&mut V)>(&mut self, key: &K, f: &F) {
        // just to update or you could use entries api
        if let Some(mut val) = self.0.get_mut(key) {
            f(&mut val)
        }
    }
}

you can always override the default hash function FNV

1 Like

It's not Rust, but I think you would be interested in https://www.foundationdb.org/ .

1 Like