Somewhat complex lifetime issues

Hi all,

I'm struggling to get a piece of code working which involves graphql_parser and stubborn lifetime issues. Graphql_parser takes a str reference and returns a document with references to the input &str, thus the corresponding document struct needs a lifetime.

It also supports using String copies so there's no requirement for the input string reference to live as long as the resulting document struct and the crate provides a function to convert the lifetime of such a document to 'static, into_static().

This caused all kinds of issues because the document is an element in a stuct of my own and 'static means it lives as long as the process so I replaced that bit with a similar call to hack the lifetime to the one of the struct, 'a.

I got most of it working but can't seem to get the final issue resolved in line 72 of the sample code. You'll need graphql-parser = "0.4.0" in you Cargo dependencies section.

Thanks in advance for any insight!
--Christina

use std::io;
use std::fs;
use std::sync::Mutex;

use graphql_parser::schema;
use graphql_parser::query;

#[derive(Debug)]
struct Generator<'a, D> {
  schema:  Mutex<schema::Document<'a, String>>,
  data:    D
}

pub trait GeneratorTrait<D> {
  fn new() -> D;
  fn query<'b>(self: &Self, gen: &'b Generator<'b, D>, query: &query::Query<'b, &'b str>);
  fn mutate<'b>(self: &Self, gen: &'b Generator<'b, D>, mutate: &query::Mutation<'b, &'b str>);
  fn subscribe<'b>(self: &Self, gen: &'b Generator<'b, D>, subsribe: &query::Subscription<'b, &'b str>);
}

#[derive(Debug)]
struct GeneratorTestData {
}

pub type GeneratorTest<'a> = Generator<'a, GeneratorTestData>;

impl GeneratorTrait<GeneratorTestData> for GeneratorTestData {
  fn new() -> Self {
    GeneratorTestData {}
  }
  fn query<'b>(self: &Self, gen: &'b GeneratorTest, query: &query::Query<'b, &'b str>) { }
  fn mutate<'b>(self: &Self, gen: &'b GeneratorTest, query: &query::Mutation<'b, &'b str>) { }
  fn subscribe<'b>(self: &Self, gen: &'b GeneratorTest, query: &query::Subscription<'b, &'b str>) { }
}

impl<'a, D> Generator<'a, D> where D: GeneratorTrait<D>{
  pub fn new(fname: &str) -> io::Result<Self> {

    /* parse schema file into a document */
    let s = fs::read_to_string(fname)?;
    let schema: schema::Document<String> = match schema::parse_schema(&s) {
      Ok(doc) => doc,
      Err(why) => return io::Result::Err(io::Error::new(io::ErrorKind::InvalidData, why.to_string()))
    };

    /* Since we're using 'String' copies for the parser's AST elements, we can remove the lifetime
     * dependency to the &str passed to 'parse_schema()'. The author of graphql_parser provides the
     * function 'into_static()' for this purpose but we don't really need 'static, just living as
     * long as the struct would be ebough for our purposes, so we'll hack this manually and avoid
     * issues with rustc complaining that a struct member variable can't really be 'static further
     * down.
     */
    let schema = unsafe { std::mem::transmute::<_, schema::Document<'a, String>>(schema) };

    /* return completed struct */
    let this = Self {
      schema: Mutex::new(schema),
      data:   D::new()
    };
    io::Result::Ok(this)
  }

  pub fn parse<'b>(self: &'b Self, query: &'b str) ->
    Result<query::Document<'b, &'b str>, query::ParseError> {
    query::parse_query(query)
  }

  pub fn query<'b>(self: &Self, query: &query::Document<'b, &'b str>) {
    /* go through top-level operations in the given query */
    for d in &query.definitions {
      match d {
        query::Definition::Operation(op) => {
          match op {
            query::OperationDefinition::SelectionSet(s) =>  (), 
            query::OperationDefinition::Query(q) =>         self.data.query(self, q),
            query::OperationDefinition::Mutation(m) =>      self.data.mutate(self, m),
            query::OperationDefinition::Subscription(s) =>  self.data.subscribe(self, s),
          };
        },

        query::Definition::Fragment(f) => (),  /* ignored for now */
      }
    }  
  }
}

fn main() -> std::io::Result<()> {
  /* load schema */
  let gen: GeneratorTest = match GeneratorTest::new("../assets/test_schema.graphql") {
    Ok(g) => g,
    Err(err) => {
      println!("Failed to read GraphQL schema file: {}", err.to_string());
      return Err(err);
    }
  };
  println!("schema: {:#?}", gen);
  println!("\n\n-------------------------------------------------------------\n");

  /* load query */
  let qs = fs::read_to_string("../assets/test_query.graphql")?;
  let query = match gen.parse(&qs) {
    Ok(doc) => doc,
    Err(why) => {
      println!("Failed to read GraphQL query file: {}", why.to_string());
      return std::io::Result::Err(std::io::Error::new(std::io::ErrorKind::InvalidData,
                                                      why.to_string()));
    }
  };
  println!("query: {:#?}", query);
  println!("\n\n-------------------------------------------------------------\n");

  /* execute GraphQL query */
  gen.query(&query);

  Ok(())  
}

Your comment before the transmute call seems to reveal a misunderstanding about lifetimes. Fundamentally, a lifetime is not how long a value lives. Lifetimes are the duration of borrows, but your Generator struct doesn't actually borrow from anything.

If the library provides a way to get an schema::Document<'static, String>, then you should do that and store that type in your struct. This lets you remove all of the lifetime annotations from the rest of your program.

No. A type being "'static" does not mean "values of this type live forever". Rather, it means "values of this type will never contain a dangling reference no matter how long they live". There's no requirement that they actually live forever.

For example, an &'static str doesn't have to live forever. It's just a reference - it can go out of scope at any time. However, the reference is still said to be 'static, because the thing it points at lives forever, so keeping the &'static str around for a long time will not result in the thing it points at being destroyed.

If a type contains no references, then it is also 'static because if you never have any references at all, then you also don't have dangling references.

4 Likes

Hmm... when using into_static(), rustc complained further down in the GeneratorTrait functions that the Generator reference doesn't live long enough to satisfy the 'static lifetime associated to the graphql::Document.

This seemed to make sense to me, so I replaced the hack in graphql::Document::into_static() with a similar hack to give the parsed document the same lifetime as the Generator struct. This did solve the problems in the trait functions...

I'm happy to go back to using 'static for the document struct - whatever works :slight_smile: - but that would shift the problem to the trait functions and another method in the Generator struct which I had not included in the sample code because it's unrelated to the current problem.

It definitely sounds to me that you should use into_static, remove all lifetimes from all type definitions, and try to tackle whatever issue you ran into instead of what you're trying to do now.

1 Like

That's actually what I've been doing - remove lifetime parameters and qualifiers wherever I can - but it seems as if there's nothing else to remove.

At this point, I'm stuck and would really like to understand why line 72 causes an error.

But thanks for all the help and tips so far!

Something to note is that all these lifetime structs are invariant due to the trait bound -- that is, their lifetimes can't be automatically shrunk like a reference's lifetime can.

With that in mind, let's spell things out with more lifetimes. You're in

impl<'a, D> Generator<'a, D> where D: GeneratorTrait<D> {
    pub fn query<'b>(self: &'s Self, query: &'d query::Document<'b, &'b str>) {
// added for discusssion    ^^               ^^

Trying to call something like

<D as GeneratorTrait>::query<'q>(&self.data, self, &query.definitions[0].0)

which has the signature

fn query<'q>(
    self: &Self,
    gen: &'q Generator<'q, D>,
    query: &'x query::Query<'q, &'q str>
    //      ^^ added for discussion
)

Whenever you have a &'outer Whatever<'inner>, 'inner has to outlive 'outer, or you'd have a reference to invalid data. Let's write this as 'outer <= 'inner. Then from the signatures involved alone, we have

's <= 'a // &'s Self, Self = Generator<'a, D>
'd <= 'b // &'d query::Document<'b, &'b str>
'x <= 'q // &'x query::Query<'q, &'q str>

And if we line up the things you're passing to GeneratorTrait::query with what you have in the method, we see

fn query<'q>(
    self: &Self,                          // &self.data
    gen: &'q Generator<'q, D>,            // &'s Generator<'a, D> (self)
    query: &'x query::Query<'q, &'q str>  // &'d query.... 'b
    //      ^^ added for discussion
)

Which imposes that

'x <= 'd
'q <= 's
'q == 'a // due to invariance of Generator (due to invariance of Document)
'q == 'b // due to invariance of Query

And because we now have 's <= 'a == 'q and 'q <= 's, we also have 's == 'q. Putting it all together:

'x <= 'd <= 'q == 'b == 's == 'a

Which means the only way you could actually make the call is if your method signature was

-pub fn query<'b>(self: &Self, query: &query::Document<'b, &'b str>) {
+pub fn query(self: &'a Self, query: &query::Document<'a, &'a str>) {

(Or if you change something else, like the trait / trait method signature.)

2 Likes

Thanks for the detailed explanation. Changing the signature in the trait is not all that easy because the query documents are definitely not living as long as the Generator instance - they are on the stack of a request processing thread while the Generator instance is in some kind of context, or even a static.

But the point about lifetime shrinking not working with trait bounds in this case would explain my issue: just not calling any of the trait functions in query() resolves the issue.

I've been trying to mimic object-oriented patterns via generics and traits, which makes things more complex than I like to begin with. I'll rewrite the code to use a base struct, Generator, with the base functions and a macro which takes care of embedding the base Generator struct into the implentation structs.

Thanks,
--Christina

I've given the idea of more precise lifetimes for the trait functions another stab and noticed two errors on my side:

  1. I didn't properly distinguish between struct and function parameter lifetimes
  2. I had the order of lifetime dependencies in the 'where 'x: 'y" clause backwards. The longer-living lifetime is on the left side.

The trait functions now look like this:

pub trait GeneratorTrait<'a, I, CV> {
  fn new() -> I;
  fn query<'b>(self: &'b Self, gen: &'a Generator<'a, I, CV>, query: &'b query::Query<'b, &'b str>);
  fn mutate<'b>(self: &'b Self, gen: &'a Generator<'a, I, CV>, mutate: &'b query::Mutation<'b, &'b str>);
  fn subscribe<'b>(self: &'b Self, gen: &'a Generator<'a, I, CV>, subsribe: &'b query::Subscription<'b, &'b str>);
}

I initially had 'self' with lifetime 'a but that makes no sense in this context because the trait's backing struct only has to live as long as the function call, thus I demoted it to 'b as all other paramerets, except for 'gen'.

The generator, being the base struct and, further down, being used to return references to data stored within, needs lifetime 'a.

Very confusing. I believe rustc should really handle lifetime management by itself but that's apparently a long way to go...

Thanks again for all the input, @alice and @quinedot

Thanks,
--Christina