Borrow is out of scope, but struct still borrowed

[AlexSherb]

Hello everyone. While writing own compile time reflection crate I came with interesting issue.

My code:

pub struct IfBlock<'a> {
    reflection_scope: &'a mut ReflectionScope<'a>,
    code: TokenStream,
}

// impl<'a> IfBlock<'a>
pub fn else_block<F>(mut self, block: F)
    where
        F: FnOnce(&mut ReflectionScope),
    {
        let else_block_code = {
            let mut reflection_scope = self.reflection_scope.child_scope(); // borrow of `*self.reflection_scope` occurs here argument requires that `*self.reflection_scope` is borrowed for `'a`
            block(&mut reflection_scope);
            quote! {
                else {
                    #reflection_scope
                }
            }
        };

        self.code.extend(else_block_code);

        self.end(); // Error: cannot move out of `self` because it is borrowed. Why it fails if borrow out of scope?
    }

    pub fn end(self) {
        self.reflection_scope.code.extend(self.code);
    }

pub struct ReflectionScope<'a> {
    pub(crate) code: TokenStream,
    kind: ScopeKind<'a>
}   

enum ScopeKind<'a> {
    Root {
        ident_counter: u64
    },
    Child {
        parent: &'a mut ReflectionScope<'a>
    }
}

// impl<'a> ReflectionScope<'a>
pub fn child_scope(&'a mut self) -> ReflectionScope<'a> {
        ReflectionScope {
            code: TokenStream::default(),
            kind: ScopeKind::Child { parent: self }
        }
    }

[steffahn]

There is definitely something very wrong with the type signature of

impl<'a> ReflectionScope<'a> {
    pub fn child_scope(&'a mut self) -> ReflectionScope<'a> {
        …
    }
}

The self argument is of type &'a mut ReflectionScope<'a> which has the effect that this mutable borrow must exist for the entire time the ReflectionScope<'a> value exists.

This signature is certainly wrong. Why you chose this signature is then the relevant follow-up question. These often come about as a reaction to compiler suggestions to “fix” your code because you did something else wrong… a wait, look here

enum ScopeKind<'a> {
    Root {
        ident_counter: u64
    },
    Child {
        parent: &'a mut ReflectionScope<'a>
    }
}

it’s another instance of this dreaded &'a mut ReflectionScope<'a> type. I’m not sure what exactly the data structure you had in mind here is

pub struct ReflectionScope<'a> {
    pub(crate) code: TokenStream,
    kind: ScopeKind<'a>
}   

enum ScopeKind<'a> {
    Root {
        ident_counter: u64
    },
    Child {
        parent: &'a mut ReflectionScope<'a>
    }
}

but it looks a lot like you’re abusing references. Rust references are generally meant to be used for short-term borrows, and rarely come up in data structures in the first place. Furthermore, a reference in a field named “parent” suggests you might be trying to store back-references which is hard to do in Rust, and definitely impossible using ordinary references. (Back references are usually avoided at all cost, and if absolutely necessary, you’d pull out Rc and Weak references possibly need RefCell, too.)

Even if “parent” is no form of “back reference”, it’s still clear that this data structure hasn’t got its ownership story well-defined. Perhaps you could shed some light on how exactly you want to link up these structs and enums (I’ll also try if me re-reading the code once more makes the story become more clear).

[steffahn]

On second read, it looks like you might at least not be trying anything cyclic. I mean, if they are and I’m misinterpreting this, then the following thoughts will be irrelevant.

So what I’m thinking is: possibly… if e.g. the code were to work with immutable references &'a ReflectionScope<'a> your problems might go away, since then covariance can help allow such a type to be a shorter borrow, of an original ReflectionScope<'b> with 'b outliving 'a. Perhaps give it a try to see what happens if you wrap the parts that need mutability into Cell or RefCell^[1] (it’s unclear from the code snippet alone where the mutable access is needed… perhaps e.g. for the ident_counter? Presumably code, too?) and switch to shared references. No guarantees that this will succeed though, and I don’t know how large your actual code is and how much would need to be refactored ^^

Also feel free to provide or link the complete code in case you want to share the whole context. While I appreciate the summary highlighting the important parts, it can still be faster to answer questions like “what does this part do” or even more importantly “would it compile if I changed such-and-such” with the full code at hand.

---

1. admitted that would be a bit of an uncommon case for using interior mutability if the references are still truly unique, and you just want to fix the variance, but I’m not aware of too good alternatives (there’s some super-powers that trait objects can offer, but it isn’t free either, both in boilerplate and run-time overhead); and if you went for Rc instead, the interior mutability would become necessary anyways ↩︎

[AlexSherb]

Did you mean something like?:

pub struct IfBlock<'a, 'b: 'a> {
    reflection_scope: &'a mut ReflectionScope<'b>,
    code: TokenStream,
}

[steffahn]

Separating the lifetime parameters works for IfBlock, but not for the (mutually) recursive ReflectionScope/ScopeKind (since for distinguishing them all recursively, you’d need an unbounded number of parameters, essentially).

[AlexSherb]

As I understood you I should to wrap some parts of ReflectionScope and ScopeKind with RefCell to fix that, but not understand what concrete parts to wrap. Can you write their definition?

[steffahn]

To demonstrate my explanations, here’s a compiling version using shared references, and here’s even a version using the trait object trickery^[1] I’ve hinted at in my footnote.

Of course I could only incorporate the code that you showed, so it’s not impossible that these solutions don’t work with some other parts of your code or need to be adjusted. In particular the former approach might need more things mutable, e.g. perhaps the ident_counter wants to be mutated somewhere, too. (For simple types like u64 using Cell can be nicer than RefCell, by the way.) Also in case you use this with multi-threading, RefCell could be limiting.

---

1. which working based on the fact that they can encapsulate and hide lifetimes behind a single common lower-bound ↩︎

[AlexSherb]

But why it have compiled? There is no changes except change of mutable references to non-mutable. The reference in part:

let reflection_scope = self.reflection_scope.child_scope();

Is still borrowed for 'a, and then moved when calling self.end()

[steffahn]

To be fair, one would probably also want to re-write

impl<'a> ReflectionScope<'a> {
    pub fn child_scope(&'a self) -> ReflectionScope<'a> {
        ReflectionScope {
            code: Default::default(),
            kind: ScopeKind::Child { parent: self },
        }
    }
}

into

impl<'a> ReflectionScope<'a> {
    pub fn child_scope<'b>(&'b self) -> ReflectionScope<'b> {
        ReflectionScope {
            code: Default::default(),
            kind: ScopeKind::Child { parent: self },
        }
    }
}

or equivalently, with elision

impl ReflectionScope<'_> {
    pub fn child_scope(&self) -> ReflectionScope<'_> {
        ReflectionScope {
            code: Default::default(),
            kind: ScopeKind::Child { parent: self },
        }
    }
}

but it works, too, without this change.

The reason is that with immutable references, ReflectionScope<'a> becomes covariant in its lifetime argument 'a. This means that the &'a ReflectionScope<'a>-type value passed to child_scope can come from borrowing a ReflectionScope<'b> of a different lifetime 'b that can be larger than 'a. This is because then the ReflectionScope<'b> could be borrowed for a lifetime of 'a, as a &'a ReflectionScope<'b> reference, and due to covariance, this reference can then implicitly be coerced into &'a ReflectionScope<'a>. These kind of coercions can happen in a lot of place in Rust implicitly, including the call to self.reflection_scope.child_scope() in the linked code example.

With the signature changed as indicated above, this kind of coercion would instead happen inside of the body of child_scope. Whereas, with mutable references, doing the same kind of adjustment to the lifetimes in the signature of child_scope makes the compilation error simply move from the body of else_block over to the body of child_scope.

[AlexSherb]

Thanks for help!