Borrow checker issues when using enums and structs

The previous post addresses the specific code problems; this post is mostly about match patterns / binding modes more generally.


The details of match binding modes (originally called "match ergonomics") are not simple and even has some long-standing bugs. I personally find it easier to start from a pre-binding-modes understanding of pattern matching. This is a good article on the topic of patterns, written before binding modes existed. Before binding modes, you had to explicitly "peel back" references like &mut, just like you have to explicitly match on things like tuple variants.

The main thing the article doesn't cover are reference binding modifiers:

let ref a = a;     // like `let a = &a`
let ref mut a = a; // like `let a = &mut a`

Which are required because you're often matching some place nested within a larger data structure, where you can't just take a reference on the right-hand side.

The main benefit of pre-binding mode patterns are their (relative) simplicity -- the dual nature of the pattern and the expression being matched. Without binding modes, the pattern and expression must be "in sync". If you want to get your own matches back to a pre-binding-mode state, for example to aid understanding what is going on with the types involved, you can use clippy and deny the pattern_type_mismatch lint. However, it won't necessarily help if there are other errors that mask the clippy lint, without taking it step by step anyway.


Starting from an understanding of pre-binding-mode patterns, the main idea of binding modes is to enable rewriting (pre-binding-mode) matches like this:

let some_borrow = &mut something;
// ...far away...
match some_borrow {
//  vvvv Have to "peel back" the `&mut`
    &mut SimpleToken::Identifier(ref mut a) => {
// Have to not move the `String` ^^^^^^^

To something like this:

match some_borrow {
    // No "peeling back" required
    SimpleToken::Identifier(a) => {
    // No `ref mut` required

Where by not explicitly "peeling back" the &mut,[1] you enter a "binding mode" where new bindings like a are implicitly ref mut.

It's pretty straight-forward for simple cases like this once you get used to it, but the rules for when the binding modes change are complicated, the documentation incomplete, and the implementation buggy. So any attempt to explain it tends to either also be incomplete or to spiral in complexity.

Things that impact the binding mode or are otherwise taken into consideration include matching references against non-references, matching references against references, and binding modifiers like ref, ref mut, and mut. The rules are there as an attempt to "simply" "always do what you want", but as is typical with such attempts, the result is something with a lot of actual complexity.

Personally I find it convenient enough that I do make use of binding modes to a point, but try to avoid them and be explicit once my patterns get much more complicated than the simple example above.

Citations

Back to your code. This is mostly redundant with @kpreid's answer, but I want to point out that[2] you usually can find out what the compiler did by doing something like this:

        match &mut self.token {
            SimpleToken::Identifier(ref mut a) => {
                let a: () = a;
                return Expression::VarDefinition(mem::take(&mut a), 2)
            }
17 |                 let a: () = a;
   |                        --   ^ expected `()`, found `&mut String`
   |                        |
   |                        expected due to this

It's let you know that the a binding is a &mut String. And that's the same if you had done either of

        // Pre-binding-mode version
        match self.token {
            SimpleToken::Identifier(ref mut a) => {
        // Binding mode version
        match &mut self.token {
            SimpleToken::Identifier(a) => {

Which reflects some of the special rules mentioned before -- the ref mut qualifier while in ref mut binding mode is redundant.


  1. i.e. matching a non-reference pattern against a &mut value ↩︎

  2. even when it's a pain to get back to a non-binding mode state with clippy ↩︎