And yet another lifetime puzzle

1

I am attempting, completely unsuccessfully, to use Rust to write a verifier for a database that's part of a personal financial management system I've written. I've run into a lifetime puzzle. Here's a very cut-down version of my code that exhibits the issue:

  1 extern crate rusqlite;                                                                                             
  2                                                                                                                    
  3 use std::env;                                                                                                      
  4 use rusqlite::Connection;                                                                                          
  5 use rusqlite::Statement;                                                                                           
  6                                                                                                                    
  7 // Types                                                                                                           
  8 struct Globals<'a> {                                                                                               
  9     new_guid: &'a mut Statement<'a>,                                                                               
 10 }                                                                                                                  
 11                                                                                                                    
 12 fn main() {                                                                                                        
 13     const DB_FILE_INDEX: usize = 1;                                                                                
 14                                                                                                                    
 15     // Open the database                                                                                           
 16     let db = Connection::open(env::args().nth(DB_FILE_INDEX).unwrap()).unwrap();                                   
 17                                                                                                                    
 18     // Prepare queries we will need later                                                                          
 19     let mut new_guid_stmt = db.prepare("select lower(hex(randomblob(16)))").unwrap();                              
 20                                                                                                                    
 21     let mut globals = Globals {                                                                                    
 22         new_guid: &mut new_guid_stmt,                                                                              
 23     };                                                                                                             
 24 }

The compiler (1.12.0) says:

error: new_guid_stmt does not live long enough
--> main.rs:22:24
|
22 | new_guid: &mut new_guid_stmt,
| ^^^^^^^^^^^^^
|
note: reference must be valid for the block suffix following statement 1 at 16:80...
--> main.rs:16:81
|
16 | let db = Connection::open(env::args().nth(DB_FILE_INDEX).unwrap()).unwrap();
| ^
note: ...but borrowed value is only valid for the block suffix following statement 2 at 19:85
--> main.rs:19:86
|
19 | let mut new_guid_stmt = db.prepare("select lower(hex(randomblob(16)))").unwrap();
| ^

error: aborting due to previous error

The block suffix following lines 16 and 19 are the same! Can someone please explain what this thing is complaining about?

A general observation: it has been apparent for a very long time that the explanations of lifetime provided by the Rust Book and other sources are simply not adequate. They explain the syntax of how to annotate code with explicit lifetimes, but they do not explain the semantics; what the annotations mean is just not explained. A little searching of this forum and some googling reveals how much confusion has existed and continues to exist about this topic and for how long. I don't understand why the documentation problem hasn't been fixed after all this time that people have been clearly baffled by this. Perhaps the underlying subject itself is so arcane that it's extremely difficult to explain?

I hope someone can provide a satisfactory explanation for the issue I've run into above. Unless and until that happens, I have to put the use of Rust on my project on hold.

2

The block suffix following lines 16 and 19 are the same!

No they're not the same, the one starting at 16 ends after the one starting at 19. So new_guid_stmt is dropped before db is dropped at the end of the function. This is what the notes say.

I just can't see why the reference must be valid as long as db, but I don't know those functions from this crate, so maybe someone else can chime in there. I'm pretty sure it's connected to your moving the value in line 19, but I can't wrap my head around it.

Btw you could enclose your code by three backticks, which would make it a lot easier to read.

3

Yes, you are correct that new_guid_stmt dies before db, so my statement about the block suffixes not being the same is wrong. But that does not explain the core issue, which you summarized well in your second paragraph.

I'll fix my post to make it more readable.

Thanks.

4

@carols10cents and I are currently re-writing the book, and we're working on the initial lifetimes section right now:

(there will be a more complex, in-depth chapter later in the book itself, this is just the introduction)

There is so, so, so much work to do. We only have one full-time docs person, and a few contributors. If I'd spent all my time working on lifetimes docs, people would ask why the standard library's documentation is so bare.

1 Like
5

The solution to your problem is:

struct Globals<'a, 'b: 'a> {
    new_guid: &'a mut Statement<'b>,
}

That is, when you say &'a mut Statement<'a>, you're saying that both the Statement and the reference to it have the same lifetime. But this is impossible; the Statement is dropped after the reference to it.

This syntax is a bit complicated, and covered in the nomicon.

Anyway, the first step is to give the Statement a different lifetime than its reference:

struct Globals<'a, 'b> {
    new_guid: &'a mut Statement<'b>,
}

This will error with

   Compiling repro v0.1.0 (file:///home/steve/tmp/repro)
error[E0491]: in type `&'a mut rusqlite::Statement<'b>`, reference has a longer lifetime than the data it references
 --> src/main.rs:9:5
  |
9 |     new_guid: &'a mut Statement<'b>,                                                                               
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
note: the pointer is valid for the lifetime 'a as defined on the struct at 8:0
 --> src/main.rs:8:1
  |
8 | struct Globals<'a, 'b> {                                                                                               
  | ^
note: but the referenced data is only valid for the lifetime 'b as defined on the struct at 8:0
 --> src/main.rs:8:1
  |
8 | struct Globals<'a, 'b> {                                                                                               
  | ^

The key is reference has a longer lifetime than the data it references, which means we need to inform Rust of the relationship. So changing <'a, 'b> to <'a, 'b: 'a> says "There's a relationship between 'b and 'a, 'b must live at least as long as 'a will." And that makes sense: we need the Statement to outlive the reference to the Statement. This solves the error.

2 Likes
6

I'm glad to hear that this subject is getting your attention now and look forward to seeing the result, hopefully soon.

7

Thanks for the good explanation. You explain what the notation means, which is what I think is lacking in the Lifetime section in the current Book.

I actually tried

struct Globals<'a, 'b> {
new_guid: &'a mut Statement<'b>,
}

because after KillTheMule's message, I realized that the reference and the Statement have different lifetimes. Naturally, I got the error you mention. At that point, I had no idea how to say what needed to be said, that the Statement needs to live at least as long as the reference.

8

More precisely, you're saying that the lifetimes can be coerced to be the same. If the Globals struct was defined with an immutable reference:

struct Globals<'a> {
    new_guid: &'a Statement<'a>,
}
...
let mut globals = Globals {
    new_guid: &new_guid_stmt,                   
};

... the program would compile, semantic correctness notwithstanding, since the lifetime of the Statement could be shortened to match that of the reference.

With a mutable reference that shortening isn't possible, as explained in the nomicon under "Variance" (the Statement must be invariant.) That's not to say that the desired combination of lifetimes is impossible: the solution with separate annotations and the explicit relationship between them demonstrates that.

It might be worth mentioning somewhere in the book that mutable references are rather unforgiving, and can require one to be much more explicit, as seen in this example.

9

So what makes this work:

#![allow(unused_variables)]
#![allow(dead_code)]
#![allow(unused_mut)]

struct Statement<'b> {
    id: &'b mut Vec<i32>,
}

struct Globals<'a> {
    new_guid: &'a mut Statement<'a>,
}

fn main() {
    let mut new_guid_stmt = Statement{ id: &mut vec![5i32] };

    let mut globals = Globals {
      new_guid: &mut new_guid_stmt,
     };
}

I'm trying to de-noise the problem and its solution, but as far as I can see, the above is the same, I only added an implementation for statement.