One of the key points of the borrowing system is its profound simplicity.
I'm sure one can come up with exceptions where shared mutability is technically not wrong. But my *checks calendar* several-year experience with people wanting to give up the RWLock pattern consistently shows that basically everyone says "oh yeah, I never really needed shared mutability in the first place" after getting the hang of Rust and learning how to write idiomatic code.
Let's just keep the borrowing rules as-is. Piling subtle yet complicated exceptions on top of one another never worked out well in programming language design. There's no reason to believe we would magically get it right this time.