How to own a &str so a struct depending on it can be passed around?

help
1

The background is I tried Logos to lex, and I need to repeatedly create lexers for newly generated user input, stored as a String. That library interface only accept a &str.

Since newly created Strings are called inside a loop somewhere else, they will go out of scope.

Here's what I attempt:

pub struct Lexer<'input> {
    input_str: Rc<str>,
    token_stream: SpannedIter<'input, Token>,

    pub state: LexerState,
}

impl<'input> Lexer<'input> {
    pub fn new(input: &'input str) -> Self {
        let rc_str: Rc<str> = String::from(input).into();
        let borrowed: &str = rc_str.clone().borrow();
        Self {
            state: LexerState::default(),
            token_stream: Token::lexer(borrowed).spanned(),
            input_str: rc_str,
        }
    }
}

But this won't work because the clone of Rc will go out of scope.

What am I suppose to do here to fix the lifetime? Thanks a lot!

EDIT: here's the function that keeps update the Lexer:

impl Iterator for Lexer<'_> {
    type Item = Spanned<Token, usize, LexicalError>;

    fn next(&mut self) -> Option<Self::Item> {
        loop {
            match self.token_stream.next() {
                Some((token, span)) => {
                    self.state.borrow().request_input = true;
                    break Some(Ok((span.start, token.unwrap(), span.end)));
                }
                None => {
                    let mut state = self.state.borrow();
                    match (state.request_input, &mut state.reader) {
                        (true, Some(reader)) => {
                            match reader.readline(">> ") {
                                Ok(line) => {
                                    //state.rl_result_holder = line.clone();
                                    self.token_stream =
                                        Token::lexer(state.rl_result_holder.as_str()).spanned();
                                }
                                Err(_) => (), // starts over on error.
                            }
                        }
                        _ => break None,
                    }
                }
            }
        }
    }
}
1 Like
2

The owned version of strings in Rust is String. If you are dealing with an interface that only accepts string slices (&str), then you can pass a reference to the owned string and it will be coerced to a string slice.

3

Yes. I know it, but as I already said, in this scenario, String is created repeated and dynamically. I need the ownership tied to the struct itself so I can store the lexer inside the struct.

4

The problem here is that you are trying to store some data (the string) and another value that borrows that data (SpannedIter<'input, Token>) in the same struct (a “self-referential struct”). Rust does not support this; there is no possible combination of types and lifetime annotations that will make it work. You must either:

  • Organize your code so that you do not need such a struct; that is, finish lexing the input within the scope of one function call that takes the &'input str.
  • Use a library which implements such self-reference, such as ouroboros. This is risky because it’s using unsafe code to add something the language doesn’t understand.
1 Like