Baffled again by lifetime issue


#1

In a post here some months ago, I mentioned that I couldn’t see the cost/benefit proposition for Rust and that part of the reason I felt the costs were too high relative to benefit was the quality of the documentation. I did say that I might revisit Rust as, hopefully, things improve, and I can see that the New Book is growing. So I thought I’d try again and so I resurrected some code I’d written in an earlier attempt that resulted in a pitched battle with the compiler.

Unfortunately, the experience now has been similar to the experience then. I include a snippet of code here that I want to discuss. It is intended to be a report generator that is part of a personal financial mangement suite I have written. The ugly magic numbers in the code below are actually generated by m4. I have my reasons for this that I won’t discuss here, since they are not relevant to the Rust issue I want to discuss. Here’s the code:

fn build_account_tree(parent:& mut Account, db:&Connection,
                        mut marketable_asset_value_stmt:Statement, 
                        mut non_marketable_asset_and_liability_value_stmt:Statement, 
                        mut income_and_expenses_value_stmt:Statement,
                        julian_end_date_time:f64) {
    fn get_account_value<'l>(child:&'l Account, marketable_asset_value_stmt:&'l mut Statement, 
                        non_marketable_asset_and_liability_value_stmt:&'l mut Statement, 
                        income_and_expenses_value_stmt:&'l mut Statement,
                        julian_end_date_time:f64) -> f64 {
        let error_message = format!("Account named {}, guid {} had no flag bits indicating account type",
                            child.name, child.guid);
        let stmt = (if (child.flags & 4) != 0 {
            (if  (child.flags & 1) != 0 {
                Result::Ok(marketable_asset_value_stmt)
            } else {
                Result::Ok(non_marketable_asset_and_liability_value_stmt)
            })
        } else {
            (if  (child.flags & 32) != 0 {
                Result::Ok(non_marketable_asset_and_liability_value_stmt)
            } else {
                (if  (child.flags & (64 |
                            128)) != 0 {
                    Result::Ok(income_and_expenses_value_stmt)
                }  else {
                    Result::Err(error_message)
                })
            })
        }).unwrap();
        stmt.bind(1,child.guid.as_str());
        stmt.bind(2,julian_end_date_time);
        newcash_sqlite::one_row(stmt, newcash_sqlite::get_f64)
    };
    let mut account_children_stmt =  db.prepare(constants::ACCOUNT_CHILDREN_SQL).unwrap();
    let child_accessor = |stmt:&mut Statement| {
                        let mut child = Account {
                                name: stmt.read(0).unwrap(),
                                guid: stmt.read(1).unwrap(),
                                value: 0.0,
                                flags: stmt.read(2).unwrap(),
                                children: Vec::new()
                            };
                            child.value = get_account_value(&mut child, marketable_asset_value_stmt, 
                                                                non_marketable_asset_and_liability_value_stmt, 
                                                                income_and_expenses_value_stmt,
                                                                julian_end_date_time);
                            child.flags = child.flags | (parent.flags & (1 |
                                        4 |
                                        16 |
                                        32 |
                                        64 |
                                        128 |
                                        256));
                            parent.children.push(child);
                    };
    account_children_stmt.bind(1,parent.guid.as_str());
    while let State::Row = account_children_stmt.next().unwrap() {
        child_accessor(&mut account_children_stmt);
    }
    account_children_stmt.reset().unwrap();

    for child in parent.children.iter() {
        build_account_tree(&mut child, db,
            &mut marketable_asset_value_stmt,
            &mut non_marketable_asset_and_liability_value_stmt, 
            &mut income_and_expenses_value_stmt, julian_end_date_time);
        parent.value += child.value;
    }
}

Here is the error that is plaguing me:

error[E0308]: if and else have incompatible types
   --> src/main.rs:100:13
    |
100 | /             (if  (child.flags & 32) != 0 {
101 | |                 Result::Ok(non_marketable_asset_and_liability_value_stmt)
102 | |             } else {
103 | |                 (if  (child.flags & (64 |
...   |
108 | |                 })
109 | |             })
    | |______________^ lifetime mismatch
    |
    = note: expected type `std::result::Result<&mut sqlite::Statement<'_>, _>`
               found type `std::result::Result<&mut sqlite::Statement<'_>, _>`
note: the anonymous lifetime #2 defined on the function body at 87:5...
   --> src/main.rs:87:5
    |
87  | /     fn get_account_value<'l>(child:&'l Account, marketable_asset_value_stmt:&'l mut Statement, 
88  | |                         non_marketable_asset_and_liability_value_stmt:&'l mut Statement, 
89  | |                         income_and_expenses_value_stmt:&'l mut Statement,
90  | |                         julian_end_date_time:f64) -> f64 {
...   |
113 | |         newcash_sqlite::one_row(stmt, newcash_sqlite::get_f64)
114 | |     };
    | |_____^
note: ...does not necessarily outlive the anonymous lifetime #3 defined on the function body at 87:5
   --> src/main.rs:87:5
    |
87  | /     fn get_account_value<'l>(child:&'l Account, marketable_asset_value_stmt:&'l mut Statement, 
88  | |                         non_marketable_asset_and_liability_value_stmt:&'l mut Statement, 
89  | |                         income_and_expenses_value_stmt:&'l mut Statement,
90  | |                         julian_end_date_time:f64) -> f64 {
...   |
113 | |         newcash_sqlite::one_row(stmt, newcash_sqlite::get_f64)
114 | |     };
    | |_____^

There are other errors in this code fragment, some a result of my trying to fix the above error. But let’s focus on this. As you can see, get_account_value receives mutable references to three prepared Sqlite statements and the function tries to select from them based on the characteristics of the account it’s dealing with. The error is complaining about a lifetime mis-match. If I change all the OK expressions to return the same statement (obviously nonsensical), the error goes away. So apparently the compiler is upset about differences in the lifetimes of the three statements, which are owned by the outer function. But I have provided lifetime annotations. My reading of Chapter 10 of the new book suggests to me that when you assert that multiple references have the same lifetime, if the shortest of their lifetimes meets the compiler’s other correctness criteria, then the code will compile. But here, the compiler seems to me to be complaining about an immaterial difference in lifetimes. Either this is a bug, or I am missing something. I don’t doubt the latter and if so, an explanation would be appreciated.

/Don Allen


#2

Could you put together a short, self contained, compilable example of what you are seeing? This would make it so much easier for other people to help.


#3

The problem is that Statement itself has a lifetime and that’s not specified explicitly so compiler elides them to be different.

Also, please use [code]...[/code] blocks to make the code readable.


#4

I made one myself.

extern crate sqlite;
use sqlite::Statement;

fn get_account_value<'l>(
    flag: bool,
    a: &'l mut Statement,
    b: &'l mut Statement,
) {
    let _ = if flag { a } else { b };
}

fn main() {}

The problem is &mut Statment has two lifetimes, not one. One is the lifetime for which the Statement is mutably borrowed. The other is the (much longer) lifetime of the underlying sqlite3 state.

As @vitalyd wrote, your function signature only requires one of them to be consistent across the inputs, but the other lifetime can be totally different.

You need to equate them both with something like &'a mut Statement<'l>.


#5

Thank you both for your responses. Telling the compiler that the lifetimes of the three Statements are the same is the key to fixing this.

But my reaction is how anyone is supposed to figure this out from the compiler’s error message and from the documentation. Obviously the two of you understood this, but I’m guessing that somehow you both have knowledge of Rust that goes beyond what the compiler and the new and old Books say. If I’m wrong that both the compiler and the Books were unhelpful in this case, please enlighten me.


#6

Well in my case it was easy because the 1.21 compiler (currently beta) has a much better error message for this.

error[E0623]: lifetime mismatch
 --> src/main.rs:9:13
  |
6 |     a: &'l mut Statement,
  |                ---------
7 |     b: &'l mut Statement,
  |                --------- these two types are declared with different lifetimes...
8 | ) {
9 |     let _ = if flag { a } else { b };
  |             ^^^^^^^^^^^^^^^^^^^^^^^^ ...but data from `b` flows into `a` here

#7

I’m using the 1.20 compiler.


#8

Oh interesting, I didn’t know the error message changed- that’s nice!

In my case the original error message in the OP mentioned Statement<'_> which reveals that it has a lifetime too. From that point, knowing elision rules, it’s clear that the types are actually different. It’s important to remember that lifetime parameters are similar to type parameters - different ones change the concrete type.


#9

After fixing the error that we’ve discussed here, with the help of both of you and fixing errors introduced by my failed attempts to fix the original problem, I again found myself in lifetime hell and concluded that, for me, the cost-benefit proposition of Rust hadn’t changed.

It’s a combination of the complexity of the language and especially the state of the documentation, as well as my own requirements for performance. As I’ve said in earlier posts, I’m far more productive in Haskell and C than in Rust and the things I’m working on at the moment don’t require ultimate performance (so therefore in Haskell), nor would they suffer from the possibility of GC pauses (that actually never happen in practice with the applications I’m writing).

So thank you for your help. I’ll check back in a year or so to see the progress.


#10

I’m willing to help with whatever subsequent lifetime issue you ran into is. Feel free to post it here if you’ve not fully jumped off the bandwagon. I suppose even if you decided to take a break, posting the example here might help people discover better ways to document/teach the concepts.

It’s also possible that you might be missing something fundamental (and perhaps a small tidbit in the grand scheme of things) that’s preventing you from making progress. But, taking a break is certainly fine - I think many people have circled back to Rust multiple times before settling down.


#11

I appreciate your offer, but right now, I can’t devote any more time to this. I’ve preserved everything and when the opportunity presents itself, I’ll try to distill the latest problem into a simple example to present here. Who knows, maybe the compiler messages and documentation will have improved enough that the latest problem will be obvious to me.

By the way, I do not recall ever reading that the type of an object includes the lifetime of the object. This was at the root of the compiler’s complaint about the two legs of the ‘if’ in my original code. It may be in the documentation and I missed it, but if it’s not, it needs to be.

Thanks again –
/Don Allen


#12

In the second edition of the book, it’s here: https://doc.rust-lang.org/book/second-edition/ch10-03-lifetime-syntax.html#lifetime-annotations-in-struct-definitions. And, relatedly, for generics: https://doc.rust-lang.org/book/second-edition/ch19-02-advanced-lifetimes.html#lifetime-bounds

It seems the compiler message has already improved as @dtolnay pointed out :slight_smile:. It may still not be super obvious, but I think it’s moving in the right direction.


#13

You say "In the second edition of the book, it’s here: https://doc.rust-lang.org/book/second-edition/ch10-03-lifetime-syntax.html#lifetime-annotations-in-struct-definitions." That section talks about the need to provide lifetimes for references in the fields of structs. The title of section 10.3 is “Validating References with Lifetimes”. The whole section seems to me to be about the need to assure the compiler that a reference doesn’t point to something that has vaporized. That’s important, certainly, but I’m talking about something different. If you use ‘if’ as an expression, it certainly makes sense that the two legs of the if return the same type. The original complaint I got from the compiler was that I had an ‘if’ that didn’t satisfy that. Complaining of a mis-match, it then proceeded to print the type it was expecting and the type that was actually there. The two types were identical! But those types included lifetimes (under-bars) for the sub-types of the Result objects I was returning and the issue was that the subtypes were qualified by their lifetimes, although the compiler didn’t say that with anything approaching clarity. I simply don’t see where this issue is discussed in the section you cite or anywhere else in 10.3, for that matter. Again, it may be there and I haven’t read it carefully enough, or it may be there but in obscure form (this is an issue for me with both versions of The Book; it is not always clear; I think version 2 is an improvement in that regard, but it still is not always clear enough, or written as clearly as language documents that I consider the best examples of the art), or it may not be there at all.

As for improving compiler messages, yes, I saw the example given by @dtolnay from the 1.21 compiler and agree that it’s an improvement. In general, though not in the case we are discussing here, I’ve found the Rust compiler messages to be quite good. As a Haskell enthusiast, I’m used to getting good hints from the compiler about how to fix things, and the Rust compiler usually doesn’t disappoint.


#14

Ah, I see what you mean. The “typeness” of types with lifetime parameters is https://doc.rust-lang.org/book/second-edition/ch19-02-advanced-lifetimes.html#lifetime-subtyping. The nomicon also has a section on subtyping (which is purely lifetime based in Rust): https://doc.rust-lang.org/nomicon/subtyping.html. It’s best to think of lifetime parameters in types just like type parameters: Foo<String> is not the same type as Foo<i32> just like Foo<'a> is not the same type as Foo<'b>, unless 'a and 'b are related via lifetime constraints and one can be substituted for the other (where variance/subtyping is allowed). This is somewhat of a complex/advanced topic though.

Also, I think anyone who thinks they’ll read the Rust book (1st and/or 2nd edition) and understand nearly every nuance of the language is sorely misguided (this isn’t directed at you @donallen, just a general comment). In reality, with the current state of docs/availability of books, one has to consult multiple places, including forums such as this one. I’ve been recommending O’Reilly’s Programming Rust book in a few threads, and I do think it’s a great book. It’ll take one much further than the free Rust online books/docs, but by no means will one be an “expert” from just reading the book, nor will it cover every (dark) corner of the language.


#15

I’m sure you are right, but on the page https://www.rust-lang.org/en-US/documentation.html, we find this:

"Learning Rust

The Rust Programming Language. Also known as “The Book”, The Rust Programming Language is the most comprehensive resource for all topics related to Rust, and is the primary official document of the language."

It doesn’t say “If you want an introduction to Rust, this is the place to start.”

Regarding your statement that multiple resources are needed, that page does cite other documents. And References appear just below. But The Rust Reference “tends to be out-of-date”. This is a serious problem, in my opinion. Using C as an example, K&R gives you both a relatively friendly discussion of C in the first part of the book, with good use of examples, followed by a thorough reference to the language in the second part. Rust needs both a User Guide/Tutorial and a Language Reference that can be trusted to reflect the language as it is now, not as it was at some date in the past, maybe distant. I realize that Rust is much younger than C and that the documentation is a work in progress. But Rust is not much younger than Go, and, in my opinion, Go’s documentation is far superior.

I’ve had this discussion with Klabnik in the past, but he and I don’t seem to agree on the time of day, let alone how to document a big language and Rust is a big language (your paragraph above supports that assertion).

Thanks for an interesting discussion (your explanation of how lifetimes qualify types could go into the docs verbatim – excellent!).

Talk to you next year.

/Don


#16

We’ve changed this, thanks for bringing it up! It is indeed inaccurate.


#17

I am also having the same problem that @donallen was having. From https://www.rust-lang.org/en-US/documentation.html I came to know that there is an index maintained for Rust compiler errors https://doc.rust-lang.org/error-index.html. But the error number E0623 is not listed there.


#18

It looks like no explanation is provided for E0623 in here and thus it doesn’t appear in the index nor in rustc --explain.

cc @ekuber as he may know why that is.


#19

The diagnostics.rs file in the compiler lists E0623 as follows:

E0623, // lifetime mismatch where both parameters are anonymous regions

The specific error is generated in a module devoted to error reporting about different lifetimes.