Does making a reference mutable change the lifetime semantics?

I stumbled upon this when trying to change

 struct MacroVisitor<'a> {
     context: &'a Context<'a>
 }

to

struct MacroVisitor<'a> {
    context: &'a mut Context<'a>
}

The second version yields a cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements error, so I am wondering what impact changing the mutablility of the reference has on the lifetime checks? From the lifetime perspective, it seems like nothing has changed to me.

Here is a link to the complete code complete code

You can have multiple immutable references to a value but mutability is exclusive.

One issue is the difference in accessing & or &mut behind a reference of shorter lifetime than the stored lifetime, well explained on SO.