Lifetime puzzle

[Geobomatic]

Hi there,

I'm writing an interpreter for an in house tool and everything went very smoothly until I decided to add an "include another file" feature to it.

I have this function:

fn parse_script<'a>(filename: &str, content: &'a str) -> Result<Box<Script<'a>>>

which calls other parse_* function according to pest parser and I have this particular one:

fn parse_use<'a>(pair: Pair<'a, Rule>, ast: &mut Script<'a>, span: &Span) -> Result<()>

inside, I'm reading a file into a String, and calling parse_script on the content of it.

Of course, it's not allowed because this new String will be dropped at the end of parse_use. I know that so I tried to keep this String alive alongside the new Box<Script<'a>>:

fn parse_use<'a>(pair: Pair<'a, Rule>, ast: &mut Script<'a>, span: &Span) -> Result<()> {
     // ... skip
    let mut sub_script = SubScript {
            script: None,
            content: std::fs::read_to_string(&path)?,
    };
    let script = parse_script(&file_name, &sub_script.content)?;
    sub_script.script = Some(script);
    ast.sub_scripts.insert(sub_script.id.clone(), sub_script);
}

but it seems this is not sufficient for Rust to garantee safety here:

error[E0597]: `sub_script.content` does not live long enough                                      
  --> src\parser.rs:69:43
   |
52 | fn parse_use<'a>(pair: Pair<'a, Rule>, ast: &mut Script<'a>, span: &Span) -> Result<()> {    
   |              -- lifetime `'a` defined here
...
69 |     let script = parse_script(&file_name, &sub_script.content)?;
   |                                           ^^^^^^^^^^^^^^^^^^^ borrowed value does not live long enough
...
78 |     ast.sub_scripts.insert(sub_script.id.clone(), sub_script);
   |     --------------------------------------------------------- argument requires that `sub_script.content` is borrowed for `'a`
79 |     Ok(())
80 | }
   | - `sub_script.content` dropped here while still borrowed

error[E0505]: cannot move out of `sub_script` because it is borrowed
  --> src\parser.rs:78:51
   |
69 |     let script = parse_script(&file_name, &sub_script.content)?;
   |                                           ------------------- borrow of `sub_script.content` 
occurs here
...
78 |     ast.sub_scripts.insert(sub_script.id.clone(), sub_script);
   |                     ------                        ^^^^^^^^^^ move out of `sub_script` occurs 
here
   |                     |
   |                     borrow later used by call

I'm pretty sure this is feasible in Rust, I just don't know how… yet?

Thanks for reading!

EDIT:
forgot to give the def of SubScript:

pub struct SubScript<'a> {
    pub script: Option<Box<Script<'a>>>,
    pub content: String,
}

[alice]

You are trying to create a self-referential struct, i.e. script references content. This is impossible in safe code, and very difficult even in unsafe code.

The main challenge has to do with the fact that if you move the struct containing all the fields, references from one field to another are suddenly invalidated.

[Geobomatic]

Thanks for your answer @alice!

It's working fine with the root file because I read the content and pass it to parse_script without any struct.

What I really need, is to make the content of the sub script to survive as long as the root content. Is there a pattern to do so?
Or a way to say to Rust that this new content and this new Box<Script<'a>> will live together?

[alice]

You could pass some sort of string storage as an argument and put the string in there, and then construct your references by referencing that store, e.g.

fn parse_use<'a>(pair: Pair<'a, Rule>, ast: &mut Script<'a>, span: &Span, string_store: &'a mut String) -> Result<()> {
     // ... skip
     *string_store = std::fs::read_to_string(&path)?;
    let script = parse_script(&file_name, string_store)?;
    ast.sub_scripts.insert(sub_script.id.clone(), script);
}

Now the caller has the responsibility of ensuring the provided spot the string is placed in lives long enough.

[Geobomatic]

Stupid me, of course… Thank you very much!

[Geobomatic]

Hm, of course it not that simple, as I want multiple String, I tried passing mut content_storage: &'a mut Vec<String> but when immutably borrowing from it to parse a String, I can't borrow it mutably anymore (in parse_use) to add a new String.

Makes sense, nothing tell Rust that I won't destroy a String in the Vec.

Is there something that can store multiple String and only add new String without ever destroying any until the end of program?

[alice]

If you know how many you need in advance, you can create a vector of empty strings and use the iter_mut() function to create an iterator that gives you a mutable reference on each call to next().

[Geobomatic]

I don't but I think I'll parse de the file once to know how may include it contains and forbid nested include, should be sufficient for this internal tool

Thank you again