Just think about it – this isn't something that should be possible.
The RefCell needs to keep track of whether it is borrowed (and if so, whether it is borrowed mutably or immutably). A plain reference can't do that. It can't call back to its owner, it has no knowledge whatsoever as to any wrapper types it might point inside.
This is exactly why Ref(Mut) exists: it's an RAII guard that signals to its originating RefCell upon being dropped that the borrow is ended. If you could obtain a borrow that is independent of the lifetime of any Ref(Mut)s, then it would be unsound. You could have some outstanding immutable borrows, drop all Refs, and borrow the inner value mutably, creating an aliasing &mut, which is insta-UB.
Thus, any returned reference must have its lifetime tied to the existence of Ref(Mut), and so the only way to get such a (short-lived) reference is to go through the guard object itself, because that's how the compiler can ensure, by way of lifetime annotations, that it doesn't outlive the guard object.