For read-mostly workloads, there are also ways to do without reader locks altogether, if you can afford the overhead of an occasional copy/fresh write (or can make copies cheap, as in linked data structures).
The general idea is to have a publicly visible pointer to the current version of the data. A writer proceeds by building a modified copy of the data, then CASing it in place of the previous version. At this point, there are two Hard Problems to take care of:
- Make sure that CAS is a reliable indicator that the data was not modified. If there are multiple writers and storage can be reused, this is not guaranteed. Various specialized memory management schemes exist to resolve this issue.
- Figure out a way to tell when readers are done with the old data. There are multiple schemes for that, which can be broadly split in two categories: those that count readers looking at the data (easy but can get expensive) and those that track when readers go idle (much harder to get right, but may scale more).
User-space RCU is one popular example of this strategy at work. As a shameless plug, I also had some fun with different algorithms for the single-writer case last year, when I wrote triple-buffer and spmc-buffer, and explained the design a bit here.
For the cases where a copy is unaffordably inefficient, even accounting for the relative rarity of writes, and that cannot be improved upon, I agree that a distributed reader-writer lock can be the right choice.
It comes with its own expenses though (more atomic operations and memory barriers on the reader's happy path, writer path is very expensive), and that is a relatively narrow use case, so I am not sure if the pattern is general enough to warrant a generic implementation. What do you think?