Can not return a reference to local variable when using RefCell?

help
1

I don't understand why the Rust compiler is complaining here. Can I please get some help? I don't see anyway how there is a "reference" to the local variable "matcher" in the below example... and yet, I can't get Rust to stop complaining. Help please!

Full code here: GitHub - th317erd/rust-adextopa-core: Advanced Extensible Token Parser (core) for Rust
Relevant file: rust-adextopa-core/debug.rs at main · th317erd/rust-adextopa-core · GitHub

Data structures:

pub struct DebugPattern {
  matcher: Option<Rc<RefCell<Box<dyn Matcher>>>>,
  debug_mode: usize,
}

#[derive(Debug, PartialEq, Clone)]
pub enum MatcherSuccess<'a> {
  Token(TokenRef<'a>),
  ExtractChildren(TokenRef<'a>),
  Skip(isize),
  Break((&'a str, Box<MatcherSuccess<'a>>)),
  Continue((&'a str, Box<MatcherSuccess<'a>>)),
  None,
  Stop,
}

#[derive(Debug, PartialEq, Clone)]
pub enum MatcherFailure<'a> {
  Fail,
  Error(&'a str),
}

pub type ParserContextRef<'a> = Rc<RefCell<ParserContext<'a>>>;

pub struct ParserContext<'a> {
  debug_mode: usize,
  matcher_reference_map: Rc<RefCell<HashMap<String, Box<dyn Matcher>>>>,
  variable_context: Rc<RefCell<HashMap<String, String>>>,
  pub offset: SourceRange,
  pub parser: ParserRef,
  pub name: &'a str,
}

Screenshot-2022-03-20-12-08-48
Screenshot-2022-03-20-12-08-481739×1409 378 KB

2

Because MatcherSuccess and MatcherError are annotated with a lifetime, they represent borrowed data. Due to Rust’s lifetime elision rules, the signature of matcher.exec() indicates that its return value borrows from self. In this case, that is the local variable matcher.

2 Likes
3

Thank you much! Now that I understand the issue though... I am not sure how to go about fixing it. I tried the following:

fn exec<'b, 'c>(
    &'b self,
    context: ParserContextRef<'c>,
  ) -> Result<MatcherSuccess<'b>, MatcherFailure<'b>>

...and many other iterations as well... and now that you have explained it, I understand why my attempts aren't working, but I don't know what to do about it. How can I fix this?

4

Okay... I think after some head-banging against the keyboard I finally understand what is wrong. Let me know if I've got this correct:

I have an essential three ingredients in my code:
Pattern structs (that contain information on how a match should happen)
Matchers trait (which define how patterns should be implemented)
Tokens structs, which are generated and output by Pattern Matcher implementations

Pattern instances have names, and Token instances also have names. What I am doing right now is referencing the name of the Pattern inside the generated Token. This is forcing me to tie the lifetimes of Patterns and Tokens together, which means that my result will always be the lifetime of the matcher that the result was generated by.

To fix this issue, I believe that I simply need to turn my name: &'a str inside both structs (or at least the Token struct) into name: String, and do a name copy when I generate the Token result inside the Pattern Matcher implementation. This will "untie" the two lifetimes (because, in reality, Tokens can and will generally live longer than their Patterns that generated them).

Did I understand this situation correctly? Thank you for all your help here.

1 Like
5

Shouldn't it be Result<MatcherSuccess<'c>, MatcherFailure<'c>> ? If I understand your code correctly, MatcherResult borrows data from context but not from matcher.

1 Like
6

The problem is that you are mixing up the lifetime of &self with the lifetime of all the names. The most straightforward solution is to add a lifetime parameter to the Matcher trait, to indicate the name lifetime.

diff --git a/src/matcher.rs b/src/matcher.rs
index acb08ea..0e54f3c 100644
--- a/src/matcher.rs
+++ b/src/matcher.rs
@@ -27,2 +27,2 @@ pub enum Pattern<'a> {
-  Matcher(&'a dyn Matcher),
-  Func(&'a dyn Fn(&'a ParserContext) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>>),
+  Matcher(Box<dyn Matcher<'a>>),
+  Func(Box<dyn Fn(&ParserContext<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>>>),
@@ -31,3 +31,3 @@ pub enum Pattern<'a> {
-pub trait Matcher {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure>;
-  fn get_name(&self) -> &str;
+pub trait Matcher<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>>;
+  fn get_name(&self) -> &'a str;
diff --git a/src/matchers/break.rs b/src/matchers/break.rs
index b577e60..63aa99b 100644
--- a/src/matchers/break.rs
+++ b/src/matchers/break.rs
@@ -14,2 +14,2 @@ impl<'a> BreakPattern<'a> {
-impl<'a> Matcher for BreakPattern<'a> {
-  fn exec(&self, _: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for BreakPattern<'a> {
+  fn exec(&self, _: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -22 +22 @@ impl<'a> Matcher for BreakPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/debug.rs b/src/matchers/debug.rs
index 6eb77a5..298bbd1 100644
--- a/src/matchers/debug.rs
+++ b/src/matchers/debug.rs
@@ -6,2 +6,2 @@ use std::rc::Rc;
-pub struct DebugPattern {
-  matcher: Option<Rc<RefCell<Box<dyn Matcher>>>>,
+pub struct DebugPattern<'a> {
+  matcher: Option<Rc<RefCell<Box<dyn Matcher<'a>>>>>,
@@ -11,2 +11,2 @@ pub struct DebugPattern {
-impl DebugPattern {
-  pub fn new(matcher: Option<Box<dyn Matcher>>) -> Self {
+impl<'a> DebugPattern<'a> {
+  pub fn new(matcher: Option<Box<dyn Matcher<'a>>>) -> Self {
@@ -22 +22 @@ impl DebugPattern {
-  pub fn new_with_debug_mode(matcher: Option<Box<dyn Matcher>>, debug_mode: usize) -> Self {
+  pub fn new_with_debug_mode(matcher: Option<Box<dyn Matcher<'a>>>, debug_mode: usize) -> Self {
@@ -33,2 +33,2 @@ impl DebugPattern {
-impl Matcher for DebugPattern {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for DebugPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -72 +72 @@ impl Matcher for DebugPattern {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/discard.rs b/src/matchers/discard.rs
index a5952de..8f91b45 100644
--- a/src/matchers/discard.rs
+++ b/src/matchers/discard.rs
@@ -6,2 +6,2 @@ use crate::token::{StandardToken, TokenRef};
-pub struct DiscardPattern {
-  matcher: Box<dyn Matcher>,
+pub struct DiscardPattern<'a> {
+  matcher: Box<dyn Matcher<'a>>,
@@ -10,2 +10,2 @@ pub struct DiscardPattern {
-impl DiscardPattern {
-  pub fn new(matcher: Box<dyn Matcher>) -> Self {
+impl<'a> DiscardPattern<'a> {
+  pub fn new(matcher: Box<dyn Matcher<'a>>) -> Self {
@@ -35,2 +35,2 @@ fn collect_errors<'a, 'b>(error_token: TokenRef<'a>, walk_token: TokenRef<'a>) {
-impl Matcher for DiscardPattern {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for DiscardPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -71 +71 @@ impl Matcher for DiscardPattern {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/error.rs b/src/matchers/error.rs
index f426bed..46866fd 100644
--- a/src/matchers/error.rs
+++ b/src/matchers/error.rs
@@ -16,2 +16,2 @@ impl<'a> ErrorPattern<'a> {
-impl<'a> Matcher for ErrorPattern<'a> {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for ErrorPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -30 +30 @@ impl<'a> Matcher for ErrorPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/fatal.rs b/src/matchers/fatal.rs
index e4985ac..ddf081c 100644
--- a/src/matchers/fatal.rs
+++ b/src/matchers/fatal.rs
@@ -14,2 +14,2 @@ impl<'a> FatalPattern<'a> {
-impl<'a> Matcher for FatalPattern<'a> {
-  fn exec(&self, _: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for FatalPattern<'a> {
+  fn exec(&self, _: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -19 +19 @@ impl<'a> Matcher for FatalPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/flatten.rs b/src/matchers/flatten.rs
index a2b18fa..8bc7025 100644
--- a/src/matchers/flatten.rs
+++ b/src/matchers/flatten.rs
@@ -5 +5 @@ pub struct FlattenPattern<'a> {
-  matcher: Box<dyn Matcher>,
+  matcher: Box<dyn Matcher<'a>>,
@@ -10 +10 @@ impl<'a> FlattenPattern<'a> {
-  pub fn new(matcher: Box<dyn Matcher>) -> Self {
+  pub fn new(matcher: Box<dyn Matcher<'a>>) -> Self {
@@ -17 +17 @@ impl<'a> FlattenPattern<'a> {
-  pub fn new_with_name(matcher: Box<dyn Matcher>, name: &'a str) -> Self {
+  pub fn new_with_name(matcher: Box<dyn Matcher<'a>>, name: &'a str) -> Self {
@@ -22,2 +22,2 @@ impl<'a> FlattenPattern<'a> {
-impl<'a> Matcher for FlattenPattern<'a> {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for FlattenPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -48 +48 @@ impl<'a> Matcher for FlattenPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/matches.rs b/src/matchers/matches.rs
index 8a15443..1183bc2 100644
--- a/src/matchers/matches.rs
+++ b/src/matchers/matches.rs
@@ -33,2 +33,2 @@ impl<'a> MatchesPattern<'a> {
-impl<'a> Matcher for MatchesPattern<'a> {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for MatchesPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -63 +63 @@ impl<'a> Matcher for MatchesPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/not.rs b/src/matchers/not.rs
index 2e72dbe..30d7db2 100644
--- a/src/matchers/not.rs
+++ b/src/matchers/not.rs
@@ -4,2 +4,2 @@ use crate::parser_context::ParserContextRef;
-pub struct NotPattern {
-  matcher: Box<dyn Matcher>,
+pub struct NotPattern<'a> {
+  matcher: Box<dyn Matcher<'a>>,
@@ -8,2 +8,2 @@ pub struct NotPattern {
-impl NotPattern {
-  pub fn new(matcher: Box<dyn Matcher>) -> Self {
+impl<'a> NotPattern<'a> {
+  pub fn new(matcher: Box<dyn Matcher<'a>>) -> Self {
@@ -14,2 +14,2 @@ impl NotPattern {
-impl Matcher for NotPattern {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for NotPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -43 +43 @@ impl Matcher for NotPattern {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/optional.rs b/src/matchers/optional.rs
index fb26c27..f98a4d1 100644
--- a/src/matchers/optional.rs
+++ b/src/matchers/optional.rs
@@ -4,2 +4,2 @@ use crate::parser_context::ParserContextRef;
-pub struct OptionalPattern {
-  matcher: Box<dyn Matcher>,
+pub struct OptionalPattern<'a> {
+  matcher: Box<dyn Matcher<'a>>,
@@ -8,2 +8,2 @@ pub struct OptionalPattern {
-impl OptionalPattern {
-  pub fn new(matcher: Box<dyn Matcher>) -> Self {
+impl<'a> OptionalPattern<'a> {
+  pub fn new(matcher: Box<dyn Matcher<'a>>) -> Self {
@@ -14,2 +14,2 @@ impl OptionalPattern {
-impl Matcher for OptionalPattern {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for OptionalPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -28 +28 @@ impl Matcher for OptionalPattern {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/program.rs b/src/matchers/program.rs
index f814ce6..62d21fa 100644
--- a/src/matchers/program.rs
+++ b/src/matchers/program.rs
@@ -32 +32 @@ pub struct ProgramPattern<'a> {
-  patterns: Vec<Box<dyn Matcher>>,
+  patterns: Vec<Box<dyn Matcher<'a>>>,
@@ -65 +65 @@ impl<'a> ProgramPattern<'a> {
-  pub fn new_program(patterns: Vec<Box<dyn Matcher>>, stop_on_first: MatchAction) -> Self {
+  pub fn new_program(patterns: Vec<Box<dyn Matcher<'a>>>, stop_on_first: MatchAction) -> Self {
@@ -79 +79 @@ impl<'a> ProgramPattern<'a> {
-  pub fn new_loop<T>(patterns: Vec<Box<dyn Matcher>>, r: T) -> Self
+  pub fn new_loop<T>(patterns: Vec<Box<dyn Matcher<'a>>>, r: T) -> Self
@@ -92 +92 @@ impl<'a> ProgramPattern<'a> {
-    patterns: Vec<Box<dyn Matcher>>,
+    patterns: Vec<Box<dyn Matcher<'a>>>,
@@ -105 +105 @@ impl<'a> ProgramPattern<'a> {
-    patterns: Vec<Box<dyn Matcher>>,
+    patterns: Vec<Box<dyn Matcher<'a>>>,
@@ -120 +120 @@ impl<'a> ProgramPattern<'a> {
-  pub fn add_pattern(&mut self, pattern: Box<dyn Matcher>) {
+  pub fn add_pattern(&mut self, pattern: Box<dyn Matcher<'a>>) {
@@ -219,2 +219,2 @@ fn add_token_to_children<'a>(
-impl<'a> Matcher for ProgramPattern<'a> {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for ProgramPattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -720 +720 @@ impl<'a> Matcher for ProgramPattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/matchers/sequence.rs b/src/matchers/sequence.rs
index ba16733..f115ca4 100644
--- a/src/matchers/sequence.rs
+++ b/src/matchers/sequence.rs
@@ -58,2 +58,2 @@ impl<'a> SequencePattern<'a> {
-impl<'a> Matcher for SequencePattern<'a> {
-  fn exec(&self, context: ParserContextRef) -> Result<MatcherSuccess, MatcherFailure> {
+impl<'a> Matcher<'a> for SequencePattern<'a> {
+  fn exec(&self, context: ParserContextRef<'a>) -> Result<MatcherSuccess<'a>, MatcherFailure<'a>> {
@@ -126 +126 @@ impl<'a> Matcher for SequencePattern<'a> {
-  fn get_name(&self) -> &str {
+  fn get_name(&self) -> &'a str {
diff --git a/src/parser_context.rs b/src/parser_context.rs
index 9c99552..3d5a9ac 100644
--- a/src/parser_context.rs
+++ b/src/parser_context.rs
@@ -10 +10 @@ pub struct ParserContext<'a> {
-  matcher_reference_map: Rc<RefCell<HashMap<String, Box<dyn Matcher>>>>,
+  matcher_reference_map: Rc<RefCell<HashMap<String, Box<dyn Matcher<'a>>>>>,
@@ -23 +23 @@ impl<'a> Clone for ParserContext<'a> {
-      name: self.name.clone(),
+      name: self.name,
@@ -57 +57 @@ impl<'a> ParserContext<'a> {
-  pub fn clone_with_name<'b>(&'b self, name: &'b str) -> ParserContextRef<'b> {
+  pub fn clone_with_name(&self, name: &'a str) -> ParserContextRef<'a> {

Personally, I don't see the point of using &'a strs everywhere for the names. In all use cases I can imagine, either the names are always string literals, in which case &'a str should be replaced with &'static str, or the names are dynamically generated, in which case it would be better to own them as Strings and copy them as necessary. Currently, it makes little sense to clone the entire source string into the Parser, then meticulously avoid copying any names.

7

Thank you all! This was a lot of help for me, and I learned a lot more about lifetimes and feel I understand them a bit better (which is huge!). This is my final fix, which is more in-line with what LegionMammal978 suggested.

8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.