It might be because the closure is potentially long (many lines), and leaving the default hanging below it could be visually disconnected from the function call.
Fwiw it also makes sense to me to have arguments in this order since it's what they would be in a language like haskell where each argument is partially applied.
e.g. I'd have want to have map_or_404() and map_or_timeout() etc. by calling let map_or_404 = map_or_else(Error::404)...
Doesn't apply so much to Rust, or at least map_or_else() specifically, but since Rust borrows some of its ideas from other functional languages maybe this was a bit of influence too?