I have a hashmap which gets updated after every 15mins. Then I'm doing hashmap lookup in multiple separate threads. i want that threads which starts after hashmap update should do lookup from updated version of it.
I don't want to use Arc mutex . Rwlock or any locks because threads waits for lock and slows down the process and reduce overall performance a lot when we have suppose 60 threads or more.
I have tried below approaches but nothing worked:
if we create global hashmap variable to reflect the update changes in other threads also -- 2 ways mainly to make global -- static mut(can't initialize the map because it only takes constant functions) and lazy static (can't update this one later because it can't be mutable)
Tried Thread.local with Refcell -- but if i do update in refcell variable in main thread, other threads can't access the updated version. so either i do update in each thread which is costly or if i try updating in thread.local itself then only one of the threads reflect the update and later again start using the older thread objects.
if i create hashmap in main thread and clone these in separate thread -- it works but cloning again makes the overall process very slow when no of threads increases.
can anyone suggest, what is the fastest way to update global hashmap in main thread and it gets relfected in all other threads for lookup?
What you want is fundamentally impossible: if you have concurrent readers and writers in multiple threads, then you must synchronize.
Maybe instead of a full mutex or RwLock, you could try using a lock-free data structure, which ensures synchronization using atomics. These should have significantly smaller overhead than locks.
One option that sounds good for your use case is the arc-swap crate. It has the cost that you have to copy the entire map every time you change it, but it is maximally efficient on the reader side. Since you change it very rarely, it sounds suitable for you.
Or, to elaborate @Ayyy: it is a misconception that only multiple writers need to be synchronized. If a reader and a writer access the same value from two threads, then they must also be synchronized, because otherwise, the reader wouldn't know when not to read a partially-written value (or, equivalently, the writer wouldn't know when not to write a value currently being read).
Yes, Agree.
but there should be some way by which we could make it work, either by keeping a global flag which will be set 1 while writing and again set 0 one write done. then while flag is 0 then only other threads will read from updated hashmap else read from the old map. (but there should be some way we can access the same map in multiple threads except Arc mutex and Rwlock)
If you keep following that train of thought, you'll have reinvented Mutex or RwLock. That's basically all they are -- atomic flags to control access to shared data.
If the locks in std (based on system primitives like pthreads) are not fast enough for you, consider an alternative like parking_lot. We're also exploring the possibility of rewriting the std types: https://github.com/rust-lang/rust/issues/93740