RefCell and borrow() cause error[E0597]: borrowed value does not live long enough

I'm stumped. I was not getting this error until I had to modify data stored in an Rc, so I added RefCell and used borrow(). The code has further details. This is with Rust 1.81 (the version available on OpenBSD 7.6).

The code is:

use sqlx::{Postgres, Transaction};
use std::cell::{Ref, RefCell};
use std::rc::Rc;

//This minimally reproduces the problem, but the actual complete code base with
//this compilation error is at github.com/onemodel/onemodel in the branch "wip",
//in core/src/model/entity.rs around line 540 (https://github.com/onemodel/onemodel/blob/wip/core/src/model/entity.rs). It has many compilation errors that I can better
//fix after resolving this one (and the one around line 747).
//The complete code is in process of being converted from Scala (commented out) to Rust.
//
//I can't use into_inner() here because of the "dyn Database". These errors did not occur before I
//added the call to borrow(), which was necessary due to use of RefCell, inside the Rc, because 
//I added code that needs to write to the db struct's data. 
//
//To build this, one must first run "cargo update url@2.5.4 --precise 2.3.1".
fn main() {
    let pg = PostgreSQLDatabase {};
    let db = Rc::new(RefCell::new(pg));
    let entity = Entity { db, id: 0 };
    let result = entity.add_quantity_attribute2(None);
    println!(".");
    result.unwrap()
}
struct Entity {
    db: Rc<RefCell<dyn Database>>,
    id: i64,
}
impl Entity {
    fn add_quantity_attribute2<'a, 'b>(
        &'a self,
        //The compiler forced me to use this 'b and other generics due to reasons found in the full program
        //if it is not clear enough here.
        transaction: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
    ) -> Result<(), String>
    where
        'a: 'b,
    {
        let db: Ref<'b, dyn Database> = self.db.borrow();
        let id: i64 = db.create_quantity_attribute(
            transaction.clone(),
            self.id,
        )?;
        Ok(QuantityAttribute::new2(
            self.db.clone(),
            transaction.clone(),
            id,
        ))
    }
}
pub struct QuantityAttribute {}
impl QuantityAttribute {
    fn new2(
        _db: Rc<RefCell<dyn Database>>,
        _transaction: Option<Rc<RefCell<Transaction<Postgres>>>>,
        _id: i64,
    ) {
        ()
    }
}
trait Database {
    fn create_quantity_attribute<'a, 'b>(
        &'a self,
       transaction: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
        parent_id_in: i64,
    ) -> Result<i64, String>
    where
        'a: 'b;
}
struct PostgreSQLDatabase {}
impl Database for PostgreSQLDatabase {
    fn create_quantity_attribute<'a, 'b>(
        &'a self,
       _transaction: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
        _parent_id_in: i64,
    ) -> Result<i64, String>
    where
        'a: 'b,
    {
        Ok(0)
    }
}

The full error message is:

error[E0597]: `db` does not live long enough 
  --> src/main.rs:41:23
   |
31 |       fn add_quantity_attribute2<'a, 'b>(                    
   |                                      -- lifetime `'b` defined here
...                                                                                                                                  
40 |           let db: Ref<'b, dyn Database> = self.db.borrow();
   |               -- binding `db` declared here
41 |           let id: i64 = db.create_quantity_attribute(
   |                         -^                                                                                                      
   |                         |                                    
   |  _______________________borrowed value does not live long enough
   | |               
42 | |             transaction.clone(),
43 | |             self.id,   
44 | |         )?;  
   | |_________- argument requires that `db` is borrowed for `'b`
...                                                               
50 |       }
   |       - `db` dropped here while still borrowed

For more information about this error, try `rustc --explain E0597`.
error: could not compile `my_cargo_app` (bin "my_cargo_app") due to 1 previous error

Here is the Cargo.toml required:

[package]
name = "my_cargo_app"
version = "0.1.0"
edition = "2021"

[dependencies]
# sqlx = { version = "0.6.3", features = [ "runtime-tokio-rustls", "postgres", "uuid" ] }
sqlx = { version = "0.6.3", features = [ "runtime-tokio-rustls", "postgres"] }

This suggestion to add 'b seems problematic. That lifetime is an input to the function, so you necessarily can't have a temporary inside the function (dropped at the end of the function) with that same lifetime. Ref<'b, _> inside the function feels wrong to me.
I hope others can provide more insightful analysis.

1 Like

For what it's worth, the same error occurs if I replace that line with
let db: Ref<'_, dyn Database> = self.db.borrow();
or
let db = self.db.borrow();
.

1 Like

What is the real body of the create_quantity_attribute implementation of PostgreSQLDatabase? I.e. why did you introduce the lifetime parameters in the first place? I'm asking because if I elide all lifetimes from your snippet, it compiles just fine:

use sqlx::{Postgres, Transaction};
use std::cell::RefCell;
use std::rc::Rc;

//This minimally reproduces the problem, but the actual complete code base with
//this compilation error is at github.com/onemodel/onemodel in the branch "wip",
//in core/src/model/entity.rs around line 540 (https://github.com/onemodel/onemodel/blob/wip/core/src/model/entity.rs). It has many compilation errors that I can better
//fix after resolving this one (and the one around line 747).
//The complete code is in process of being converted from Scala (commented out) to Rust.
//
//I can't use into_inner() here because of the "dyn Database". These errors did not occur before I
//added the call to borrow(), which was necessary due to use of RefCell, inside the Rc, because
//I added code that needs to write to the db struct's data.
//
//To build this, one must first run "cargo update url@2.5.4 --precise 2.3.1".
fn main() {
    let pg = PostgreSQLDatabase {};
    let db = Rc::new(RefCell::new(pg));
    let entity = Entity { db, id: 0 };
    let result = entity.add_quantity_attribute2(None);
    println!(".");
    result.unwrap();
}
struct Entity {
    db: Rc<RefCell<dyn Database>>,
    id: i64,
}
impl Entity {
    fn add_quantity_attribute2(
        &self,
        //The compiler forced me to use this 'b and other generics due to reasons found in the full program
        //if it is not clear enough here.
        transaction: Option<Rc<RefCell<Transaction<'_, Postgres>>>>,
    ) -> Result<(), String> {
        let db = self.db.borrow();
        let id: i64 = db.create_quantity_attribute(transaction.clone(), self.id)?;
        Ok(QuantityAttribute::new2(
            self.db.clone(),
            transaction.clone(),
            id,
        ))
    }
}
pub struct QuantityAttribute {}
impl QuantityAttribute {
    fn new2(
        _db: Rc<RefCell<dyn Database>>,
        _transaction: Option<Rc<RefCell<Transaction<Postgres>>>>,
        _id: i64,
    ) {
    }
}
trait Database {
    fn create_quantity_attribute(
        &self,
        transaction: Option<Rc<RefCell<Transaction<'_, Postgres>>>>,
        parent_id_in: i64,
    ) -> Result<i64, String>;
}
struct PostgreSQLDatabase {}

impl Database for PostgreSQLDatabase {
    fn create_quantity_attribute(
        &self,
        _transaction: Option<Rc<RefCell<Transaction<'_, Postgres>>>>,
        _parent_id_in: i64,
    ) -> Result<i64, String> {
        Ok(0)
    }
}
1 Like

Here is the full code for that method, with the extra lifetimes removed. The error message it gets without the lifetimes, follows. As you see, I am still learning. All corrections welcome. I omitted but can provide the doc comments.

    fn create_quantity_attribute(
        //&'a self,
        &self,
        //transaction_in: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
        transaction_in: Option<Rc<RefCell<Transaction<Postgres>>>>,
        parent_id_in: i64,
        attr_type_id_in: i64,
        unit_id_in: i64,
        number_in: f64,
        valid_on_date_in: Option<i64>,
        observation_date_in: i64,
        sorting_index_in: Option<i64>, /*= None*/
    ) -> Result</*id*/ i64, anyhow::Error>
    //where
    //    'a: 'b,
    {
        //BEGIN COPY/PASTED/DUPLICATED BLOCK-----------------------------------
        // Try creating a local transaction whether we use it or not, to handle compiler errors
        // about variable moves. I'm not seeing a better way to get around them by just using
        // conditions and an Option (many errors):
        // (I tried putting this in a function, then a macro, but it gets compile errors.
        // So, copy/pasting this, unfortunately, until someone thinks of a better way. (You
        // can see the macro, and one of the compile errors, in the commit of 2023-05-18.
        // I didn't try a proc macro but based on some reading I think it would have the same
        // problem.)
        let local_tx: Transaction<Postgres> = self.begin_trans()?;
        let local_tx_option = Some(Rc::new(RefCell::new(local_tx)));
        let transaction = if transaction_in.clone().is_some() {
            transaction_in.clone()
        } else {
            local_tx_option
        };
        //END OF COPY/PASTED/DUPLICATED BLOCK----------------------------------

        let id: i64 = self.get_new_key(transaction.clone(), "QuantityAttributeKeySequence")?;
        let form_id = self.get_attribute_form_id(Util::QUANTITY_TYPE)?;
        self.add_attribute_sorting_row(
            transaction.clone(),
            parent_id_in,
            form_id,
            id,
            sorting_index_in,
        )?;
        let valid_on = match valid_on_date_in {
            None => "NULL".to_string(),
            Some(d) => format!("{}", d),
        };
        self.db_action(transaction.clone(),
                       format!("insert into QuantityAttribute (id, entity_id, unit_id, \
                                         quantity_number, attr_type_id, valid_on_date, observation_date) values ({},{},{},{},\
                                         {},{},{})", id, parent_id_in, unit_id_in, number_in, attr_type_id_in, valid_on, observation_date_in).as_str(),
                       false, false)?;
        if transaction_in.is_none() && transaction.is_some() {
            // see comments at similar location in delete_objects about local_tx
            // see comments in delete_objects about rollback
            let local_tx_cell: Option<RefCell<Transaction<Postgres>>> =
                Rc::into_inner(transaction.unwrap());
            match local_tx_cell {
                Some(t) => {
                    let unwrapped_local_tx = t.into_inner();
                    if let Err(e) = self.commit_trans(unwrapped_local_tx) {
                        return Err(anyhow!(e.to_string()));
                    }
                }
                None => {
                    return Err(anyhow!("Unexpectedly found None instead of Some<RefCell<Transaction<Postgres>>>. How?"));
                }
            }
        }
        Ok(id)
    }

Here is the interesting part again, but with the errors shown in neovim with rust-analyzer (there are 4 messages; I show it from cargo compilation farther below)

H  705     fn create_quantity_attribute(     _ consider introducing a named lifetime parameter and update trait if needed: `<'a>`, `'
   706         //&'a self,                                                                                                           
H  707         &self,     _ let's call the lifetime of this reference `'1`                                                           
   708         //transaction_in: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,                                                     
H  709         transaction_in: Option<Rc<RefCell<Transaction<Postgres>>>>,     __ has type `Option<std::rc::Rc<std::cell::RefCell<sql
   710         parent_id_in: i64,                                                                                                    
   711         attr_type_id_in: i64,                                                                                                 
   712         unit_id_in: i64,                                                                                                      
   713         number_in: f64,                                                                                                       
   714         valid_on_date_in: Option<i64>,                                                                                        
   715         observation_date_in: i64,                                                                                             
   716         sorting_index_in: Option<i64>, /*= None*/                                                                             
   717     ) -> Result</*id*/ i64, anyhow::Error>                                                                                    
   718     //where                                                                                                                   
   719     //    'a: 'b,                                                                                                             
   720     {                                                                                                                         
   721         //BEGIN COPY/PASTED/DUPLICATED BLOCK-----------------------------------                                               
   722         // Try creating a local transaction whether we use it or not, to handle compiler errors                               
   723         // about variable moves. I'm not seeing a better way to get around them by just using                                 
   724         // conditions and an Option (many errors):                                                                               725         // (I tried putting this in a function, then a macro, but it gets compile errors.                                     
   726         // So, copy/pasting this, unfortunately, until someone thinks of a better way. (You                                   
   727         // can see the macro, and one of the compile errors, in the commit of 2023-05-18.                                        728         // I didn't try a proc macro but based on some reading I think it would have the same                                 
   729         // problem.)                                                                                                          
   730         let local_tx: Transaction<Postgres> = self.begin_trans()?;                                                            
E  731         let local_tx_option = Some(Rc::new(RefCell::new(local_tx)));     _ lifetime may not live long enough  argument require
   732         let transaction = if transaction_in.clone().is_some() {                                                               
   733             transaction_in.clone()                                                                                            
   734         } else {                                                                                                              
   735             local_tx_option                                                                                                   
   736         };                                                                                                                    
   737         //END OF COPY/PASTED/DUPLICATED BLOCK----------------------------------  

Here is that part from cargo compilation:

2358 error: lifetime may not live long enough
    2359    --> src/model/postgres/postgresql_database3.rs:731:44
    2360     |
    2361 707 |         &self,
    2362     |         - let's call the lifetime of this reference `'1`
    2363 708 |         //transaction_in: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
    2364 709 |         transaction_in: Option<Rc<RefCell<Transaction<Postgres>>>>,
    2365     |         -------------- has type `Option<Rc<RefCell<Transaction<'2, Postgres>>>>`
    2366 ...
    2367 731 |         let local_tx_option = Some(Rc::new(RefCell::new(local_tx)));
    2368     |                                            ^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'2`
    2369     |
    2370 help: consider introducing a named lifetime parameter and update trait if needed
    2371     |
    2372 705 ~     fn create_quantity_attribute<'a>(
    2373 706 |         //&'a self,
    2374 707 ~         &'a self,
    2375 708 |         //transaction_in: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
    2376 709 ~         transaction_in: Option<Rc<RefCell<Transaction<'a, Postgres>>>>,
    2377     |

Could you share the signature of PostgreSQLDatabase::begin_trans, too, please? Is the lifetime parameter of the Transaction returned from it the same as the one from &self?


You can avoid creating local_tx eagerly—which you don't need to do if the caller provided a transaction_in—by putting self.begin_trans()? in the else-branch. Or use Option/Result combinators instead of the if-else-statement (if you prefer such a thing):

        let transaction = transaction_in
            .map(|t| Ok::<_, anyhow::Error>(t))
            .unwrap_or_else(|| Ok(Rc::new(RefCell::new(self.begin_trans()?))))?;
1 Like

Here is the requested code (it's short so I included the whole thing). It doesn't specify any explicit lifetimes. Does that answer the lifetime question?

   293     /// Like jdbc's default, if you don't call begin/rollback/commit, sqlx will commit after every                            
   294     /// sql statement, but if you call begin/rollback/commit, it will let you manage                                          
   295     /// explicitly and will automatically turn autocommit on/off as needed to allow that. (???)                               
   296     fn begin_trans(&self) -> Result<Transaction<Postgres>, anyhow::Error> {                                                   
   297         let tx: Transaction<Postgres> = match self.rt.block_on(self.pool.begin()) {                                           
   298             Err(e) => return Err(anyhow!(e.to_string())),                                                                     
   299             Ok(t) => t,                                                                                                          300         };                                                                                                                    
   301         // %% see comments in fn connect() re this AND remove above method comment??                                          
   302         // connection.setAutoCommit(false);                                                                                      303         Ok(tx)                                                                                                                
   304     }                                                                                                                         

I plan to look at your suggestion thoughtfully and note here the result; thank you.

Thanks for that. When I try that now it seems to work (or the other errors (lifetime fixes pending) are preventing the compiler to get far enough along to show me the errors I will see later). I was sure I tried that before.

But I still have to add back the lifetimes to the method, or I get those lifetime errors described earlier. And adding back those lifetimes causes the original problem described, of course.

I believe this is a reduction of the problem. As the error says, the problem is probably invariance. You don't want to try to put both the local_tx_option and transaction_in to the same variable, as that will force the lifetimes to be the same.

Perhaps you can factor out the bottom of your method.

Edit: Or recursion perhaps.

1 Like

This worked! Thank you (and to all who commented)! I like the recursion idea as it is clear and the most concise. I also have extracted the end of the method (committing the local transaction) as you suggested.

But I wish I understood why my original code required all those lifetime annotations. I have read much or all of your (quinedot's) web site content about lifetimes. Maybe I should review that though, especially now about invariance.

I'd like to understand better how I could figure this out better on my own next time. In the meantime, I'm very glad that URLO has such helpful people who know more than I do!
Thanks again.

(I hope I can reach the point where this becomes easy. Or that Rust lifetimes get easier.)

Hmm, yes. Well, I actually recommend some syntax you don't have in some places:

-fn begin_trans(&self) -> Result<Transaction<Postgres>, anyhow::Error>
+fn begin_trans(&self) -> Result<Transaction<'_, Postgres>, anyhow::Error>
+//                                          ^^

I suggest this because it makes seeing where borrows are being returned easier.[1] Looking for lifetime relationships like that is usually the first thing I do when someone hands me a lifetime problem spanning methods.

But by annotations, perhaps you meant named lifetimes, especially when you had to add bounds:

    fn create_quantity_attribute<'a, 'b>(
        &'a self,
       transaction: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
        parent_id_in: i64,
    ) -> Result<i64, String>
    where
        'a: 'b;

I didn't grok your entire project/traits/etc, but I suspect that you don't actually need so many annotations, and only ended up with them due to compiler suggestions. The problem is, the compiler was trying to find some way to get the lifetimes to work out locally with your existing code by changing the lifetime relationships. But the actual fix was to change the structure of your code so that the lifetimes did not have to be related.


Let's see if we can start with the part I did grok (or rather my reduction of it), and walk through what went on and why a bit.

With this signature:

// Let's use this alias to make the type easier to talk about
type OptTxnCell<'a> = Option<Rc<RefCell<Transaction<'a, Postgres>>>>;

// No bounds here, just named lifetimes to make it easier to talk about
fn create_quantity_attribute<'this, 'in_t>(
    &'this self,
    transaction_in: OptTxnCell<'in_t>,
    // ...
)

You had a single variable transaction of type OptTxnCell<'x>. You wanted to assign to this one of:

  • transaction_in: OptTxnCell<'in_t>
  • local_tx_option: OptTxnCell<'t>

If the lifetime was covariant in OptTxnCell<'a>, this would have all worked: whatever 't is and 'in_t could have both coerced down to some smaller lifetime 'x that only needs to stick around until you stop using transaction. But the lifetime is invariant, so that can't happen. For the assignment to work, you need 't, 'in_t and 'x to all be the same lifetime.

Now, due to the invariance, there's no way for transaction_in to be anything but a OptTxnCell<'in_t> specifically. But local_tx_option gets created by, effectively, reborrowing *self (via a call to another &self method).

The 'this lifetime on &'this self is covariant. So the 't in local_tx_option: OptTxcnCell<'t> can be anything equal to or less than 'this. The compiler's diagnostic engine sees that, and -- trying to be helpful -- goes "ah, this could all work so long as 'this was longer than 'in_t! Then 't could be 'in_t (could be 'x)."

So it suggests adding 'this: 'in_t, which allows the method to compile. But that just pushes your problem elsewhere. Depending on your code structure, sometimes things like this happen in a chain and you keep pushing the problem elsewhere, until you run out of places to push and have a mess of lifetimes that still doesn't work. Or worse yet, it seems like it works (like in a library crate), but you've created something impossible to use by downstream.

That's my guess of what went on to get you so many lifetime annotations: the compiler suggested something to make a method compile locally and kept pushing your lifetime problem somewhere else.[2]


Being able to backtrack from what the compiler suggests, to why it is suggesting it, and ultimately to what other options you may have -- I think a lot of it is experience. But you do need some general foundational understanding that invariance forces lifetimes to be equal, and that's a common source of borrow checker errors.

I guess a good feel for what the annotations mean also helps:

    fn create_quantity_attribute<'a, 'b>(
        &'a self,
       transaction: Option<Rc<RefCell<Transaction<'b, Postgres>>>>,
        parent_id_in: i64,
    ) -> Result<i64, String>
    where
        'a: 'b;

First, you pretty much never want to borrow things for longer than you need to, so almost surely this is the same from a practical perspective:[3]

    fn create_quantity_attribute<'same_lt>(
        &'same_lt self,
       transaction: Option<Rc<RefCell<Transaction<'same_lt, Postgres>>>>,
        parent_id_in: i64,
    ) -> Result<i64, String>

Second, this means that "to call this method I have to borrow *self for as long as transaction is valid". Might that ever be okay? With some other method, maybe, I don't know. It wasn't okay this time. But in any case, having that understanding will at least aid in figuring out why any errors that do pop up elsewhere happened.


I'll also walk through what I did in case it's helpful.

When I first saw your OP, my reaction was "...that'll be a lot to try to pinpoint. Maybe later :slight_smile:." Before I tried, @jofas did a lot of work to minimize the example and hone in on what more information was needed. So props to them for taking the thread as far as they did.

I picked it up from your "interesting part" in this comment, and also used the signature from here. Those gave me enough context to create what I thought was a valid reduction without too much effort.

I recognized that the lifetime was invariant just due to experience. Things behind mutable handles (&mut Thing, RefCell<Thing>, ...) generally have to be invariant. (Even without that, my reduction was enough that the compiler error called out the invariance specifically.) Invariance can easily cause the type of lifetime problems you had, so concentrating on that is another instinct born from experience.

The next part was understanding that local variables have a concrete type (with a specific lifetime), and in combination with the invariance, that meant that trying to conditionally assign one of transaction_in or Some(...local_tx...) to the same variable meant that the lifetime in transaction_in and in local_tx would have to be exactly the same. Basically what I covered above.

At that point the goal was to remove the necessity for those to be the same, or to find out why they had to be (in which case you'd have a larger problem, but maybe I could at least explain why).

I skimmed the larger version of the method (after the "interesting part"), and it looked like you didn't really need the lifetimes to be equal, you just needed some transaction to work on. In which case, though it would have been unsatisfying to have:

if let Some(txn) = transaction_in {
    // A lot of code
} else {
    let local_txn = ...;
    // That same lot of code :-(
}

it still would have solved it.

I suggested refactoring instead in the hope that your actual code was close enough to the examples that refactoring was tenable (no mass of other local state created before you needed to get your hands on some transaction). Fortunately it seems that was the case. If you had come back and said "I tried that and here's a new error", I would have tried to figure out why from there.

(The recursion idea occurred shortly after posting, in classical fashion.)


Finally, let's explore why the invariance required here.

If the lifetimes of OptTxnCell<'a> were covariant, you could do this:

        let mut local_tx: Transaction<Postgres> = self.begin_trans()?;
        if let Some(other_tx) = transaction_in {
            let mut other_mut = other_tx.borrow_mut();
            std::mem::swap(&mut local_tx, &mut *other_mut);
        }

Then, when the method returns, the borrow of &self expires. But transaction_in may still exist and be used outside of the method for as long as 'in_t, which may be arbitrarily longer than the borrow of &self -- it might even be 'static. This would be wildly unsound, because you could e.g. drop the database value and cause the reference to dangle.[4]

So that's a short example of the "why" behind mutable handles requiring invariance.

(The swap also compiles with 'this: 'in_t, but now *self remains borrowed for at least as long as transaction_in is valid, preventing the unsoundness. But of course, that causes problems elsewhere.)


  1. You can force it with #![deny(elided_lifetimes_in_paths)], or wait for more targeted lints to stabilize. ↩︎

  2. It's too bad it didn't suggest using two different variables or such as an alternative, but I bet that's a difficult situation to detect. ↩︎

  3. with 'a: 'b, the only additional thing which having two lifetimes allows is borrowing *self for arbitrary larger than 'b; there are no lifetimes in the output to drive inference to do this, so the only time that would happen is if you forced it with a turbofish ↩︎

  4. You could also make a &mut _ self while there's aliasing going on, etc. etc. ↩︎

2 Likes

Thanks very much, kind sir. I have read it slowly, and I plan to give it some more thought along with a review of the materials mentioning [co-|in-]variance at your site (for future readers of this thread, I think that is Learning Rust ). Thanks for making the world better by helping people learn to use Rust.

The best thing I can think to offer in return is my web site, where I have put thoughts important to me, at https://lukecall.net . Suggestions welcome as always.

1 Like