When locks or borrows are released (i.e. dropped)

Experimenting with Rust's drop scopes caused me some headache recently. Consider the following example:

use std::cell::RefCell;

fn main() {
    let ref_cell = RefCell::new(7);
    // Code doesn't panic if the `&`s are removed:
    let _x = &*ref_cell.borrow_mut();
    let _y = &*ref_cell.borrow_mut();
}

(Playground)

The above code causes a panic:

thread 'main' panicked at 'already borrowed: BorrowMutError', src/main.rs:7:25

If I write *ref_cell.borrow_mut() instead of &*ref_cell.borrow_mut(), then the code doesn't panic.

If I understand it right, then the panic in case of using &*… happens because of what the Rust reference calls "temporary lifetime extension":

The temporary scopes for expressions in let statements are sometimes extended to the scope of the block containing the let statement.

However, the reference also warns:

Note: The exact rules for temporary lifetime extension are subject to change. This is describing the current behavior only.

Does that mean the original code above might be compile-time error in future? Or would this only happen during an edition change?

I found an old Issue #39283, whichs seems to talk about the lifetime extension as a "leftover from the old days of Rust". So I wonder what's up with this feature/behavior?

I'm a bit irritated that the reference explicitly states that the exact rules for temporary lifetime extension are subject to change; yet I don't get a warning with the default compiler settings. But perhaps I also misunderstand something.

The reason why I started to think on this issue was this thread: Reference Decisions. Directly borrowing from a smart-pointer, RefMut, etc. seems to be a nearby thing to do. However, if that behavior is indeed subject to change, then that might be a thing to warn people about?

Of course, in case of the above example, it wouldn't matter if it resulted in a compile-time error in future, because the example causes a panic anyway.

But consider this:

use std::cell::RefCell;

fn increment(x: &mut i32) {
    *x += 1;
}

fn main() {
    let ref_cell = RefCell::new(7);
    {
        let r = &mut *ref_cell.borrow_mut();
        increment(r);
    }
    println!("Value = {}", ref_cell.into_inner());
}

(Playground)

If I understand it right, this code only works because the lifetime of the temporary RefMut value (as returned by RefCell::borrow_mut) is extended to the end of the inner block (such that the mutable borrow still lives when increment(r); is executed. Couldn't such code be invalid if the rules on temporary lifetime extension were changed?

One more thing below.

I also got confused (but meanwhile understand) why the following example works:

use std::cell::RefCell;

fn increment(x: &mut i32) {
    *x += 1;
}

fn main() {
    let ref_cell = RefCell::new(7);
    increment(&mut *ref_cell.borrow_mut());
    increment(&mut *ref_cell.borrow_mut());
    println!("Value = {}", ref_cell.into_inner());
}

(Playground)

There is no let statement involved with the borrow_mut operation, so temporary lifetime extension should not happen here. My confusion was caused by the reference stating:

Each variable or temporary is associated to a drop scope. When control flow leaves a drop scope all variables associated to that scope are dropped in reverse order of declaration (for variables) or creation (for temporaries).

[…]

Given a function, or closure, there are drop scopes for:

  • The entire function
  • […]
  • Each expression
  • […]

I thought &mut *ref_cell.borrow_mut() is an expression, so I wrongly concluded that after the expression has been evaluated, the temporary value ref_cell.borrow_mut() will be dropped. But it doesn't. I didn't understand until I read the next subsection "Scopes of function parameters":

All function parameters are in the scope of the entire function body, so are dropped last when evaluating the function. […]

I wanted to give feedback that mentioning this important detail in the next subsection is perhaps a bit confusing. It's not really a huge problem (as I guess I'm supposed to read the whole chapter first before making conclusions), but I would not be surprised if other people get confused too, when they try to understand how dropping order really is. And drop order seems to be an issue that causes a lot of confusion.

That would be my understanding as well.

The rust compiler tends not to introduce breaking changes, so the remark in the reference is probably not supposed to mean that you have to worry about any breakage. If the rules ever change it's either going to be in a way so that no (reasonable) Rust code is affected negatively, or over an edition.

Yes, it’s extending to the block that the extended variable is assigned in. The let r = … is a statement of the inner block, so the temporary lifetime extension happens to the scope of the inner block.

Indeed that’s not a temporary lifetime extension things, but you misinterpreted the quote about "function parameters". That quote is referring to the parameters in a function definition e.g.

fn foo(x: i32) {
    // do something
    let y = 42;

    // y dropped here,
    // x dropped last; it was in scope for the entire function body

What’s actually happening is that the increment(&mut *ref_cell.borrow_mut()); statement as a whole is a scope for temporary variables and subexpressions don't get their own scope. The rule of thumb is always that "temporaries are in scope for the surrounding statement", at least for simple expressions including function calls, referencing and dereferencing expressions and the like. Only some control-flow expressions like match or if have some extra local scopes.

Let me try to interpret the reference so that it's in line with that rule of thumb....

....there we go: The relevant keyword here is temporary scopes not drop scopes. The former describes what scope each temporary variable belongs to. I’m not sure why exactly each expression gets its own "drop scope", perhaps these are supposed to give the framework in which to specify the temporary scopes. (I'm not sure how to read the reference e.g. w.r.t. whether "scope" and "drop scope" are always the same thing.)

Temporary scopes

The temporary scope of an expression is the scope that is used for the temporary variable that holds the result of that expression when used in a place context, unless it is promoted.

Apart from lifetime extension, the temporary scope of an expression is the smallest scope that contains the expression and is one of the following:

  • The entire function body.
  • A statement.
  • The body of a if , while or loop expression.
  • The else block of an if expression.
  • The condition expression of an if or while expression, or a match guard.
  • The expression for a match arm.
  • The second operand of a lazy boolean expression.

So with the rule of thumb in mind that the containing statement is often the scope of a temporary. Note how that listing of temporary scopes does specify "statement", but not every "expression" as a scope. (Only certain parts of if/while/match/loop get their own temporary scopes.)

By the way that link to / mention of "place context" explains why there's a temporary being created at all: the value expression ref_cell.borrow_mut() is used in a place context (the operand of a dereference expression) in *ref_cell.borrow_mut(), so a temporary is created to hold the value of ref_cell.borrow_mut(). The (innermost) surrounding statement increment(&mut *ref_cell.borrow_mut()); is the scope of that temporary, so it’s dropped after the whole statement is fully evaluated, in particular only after the call to increment.

Place Expressions and Value Expressions

[...]

The following contexts are place expression contexts:
     [...]

     [...]

3 Likes

I found an even worse example:

use std::sync::Mutex; 
 
fn main() { 
    let mtx = Mutex::new(7); 
    let mut i = *mtx.lock().unwrap(); 
    let mut j = *mtx.lock().unwrap(); 
    i += 1; 
    j += 2; 
    println!("Values = {}, {}", i, j); 
}

(Playground)

If temporary lifetime extension was changed here, then this could result in deadlock at runtime (if Mutex isn't reentrant, which it isn't on my platform).


Edit: I guess this can't actually happen because the "the usual temporary scope" would not "be too small". So ignore this (second) post please. It is not a good example.

I would agree that the remark

Note : The exact rules for temporary lifetime extension are subject to change. This is describing the current behavior only.

sounds a bit scary; IMO the reference could/should be changed in order for readers not to get the feeling that rustc would ever disobey its stability guarantees in this context.

Feel free to open an issue on GitHub - rust-lang/reference: The Rust Reference

1 Like

Okay, I have seen previous notes on requiring a new edition because drop order would be changed. I assume it's the same here then, and I can generally count on these things to be stable within the edition… I hope… (The warning just seemed so scary :fearful:)

You just wrote the same thing :grinning_face_with_smiling_eyes:.

If that is what's meant in the reference, is this clear from the wording used?

All function parameters are in the scope of the entire function body, so are dropped last when evaluating the function. Each actual function parameter is dropped after any bindings introduced in that parameter's pattern.

I guessed I misinterpreted it because this paragraph talks about "evaluating" the function, and not "executing" it or something like that. So I judged it might happen when the expression f(…) is evaluated. (But that's been my subjective impression, of course.) I'd say it could be interpreted in both ways? What do you think? Perhaps "function body" should have got me to the right track, but it's not very explicit, I think.

Ah, I should have read three subsections further :wink:.

Thanks for clarifying. But isn't that still contradicting what's been said before:

Each variable or temporary is associated to a drop scope. When control flow leaves a drop scope all variables associated to that scope are dropped in reverse order of declaration (for variables) or creation (for temporaries).

Maybe "control flow" can't really "leave" an expression, so it doesn't apply? The wording is still a bit confusing to me.

Thanks for your help in understanding the reference.

Whether these other issues are worth opening an issue, I'm not sure, but clarifying the warning might be helpful. So I'll think about opening an issue on that. Written as it is now feels like a discouragement from using temporary lifetime extension at all in productive code (which likely isn't how it's meant).

I feel like the meaning might be that there – somehow – exist such (drop) scopes for each expression, but the section does not declare what (drop) scope a temporary is actually doing going to be associated to; and that’s the role of the “temporary scopes” section? I feel like "scope" and "drop scope" could be two terms for the same thing, and the word "scope" used in the "temporary scopes" section just refers to "drop scope"s.

I’m not reading the full page in full detail right now. To me it feel like these drop scopes for each expression are somehow introduced but never used for anything, at least for most kinds of expressions. Feels a bit weird, but even if that’s what is meant, it won’t technically be incorrect, I guess?

Ah, I might’ve found it. Many of those scopes are used in this section:

Operands

Temporaries are also created to hold the result of operands to an expression while the other operands are evaluated. The temporaries are associated to the scope of the expression with that operand. Since the temporaries are moved from once the expression is evaluated, dropping them has no effect unless one of the operands to an expression breaks out of the expression, returns, or panics.

I’m more and more conviced that "scope" and "drop scope" are the same thing. It explicitly uses the word "associated" here, too.

I would say you are right. (Though it seems easy for the reader to make wrong conclusions here if not reading all the following subsections. Maybe I was just too impatient when reading it.)

That seems to make sense. Actually "temporary scopes" are also "drop scopes" (those that will cause the temporary value from the expression to be dropped). (Sorry if maybe my wording isn't 100% correct.)

Yes, I think I'd concur. The temporary scope is a drop scope but not the same drop scope as the previously introduced drop scope "for" that expression. :crazy_face: Actually it seems technically correct, but is really confusing (to me).

Yeah, maybe it would help if the subsections heading wasn't named "drop scope" if, in fact, it means "scope" (as the expression's scope isn't the scope where it's "dropped").

I just did create a new Issue #1106 for the reference.

1 Like

Note that an expression cannot be dropped. An expression is just a syntactic concept in source code. Only a variable can be dropped. (Which includes temporary variables.)

Edit: Of course every value expression creates a value which then gets assigned to either a temporary or a variable, and those are going to be dropped eventually (resulting in destructor calls if the variable has not been moved out of, and the type of its value comes with any destructors).

Yeah right… I felt like my wording was wrong, hence:

Thanks for your correction. I kinda meant the "value" of that expression… I think the reference calls it "variable" or "temporary".

You should definitely include a link to the relevant page in the reference in this issue.

@steffahn I feel not confident enough to criticize the issue with the drop scopes vs scopes vs temporary scopes vs scopes "for" an expression. So maybe you like to open an issue on that other issue? Anyway, not sure if you see it as a problem anyway, and I'm not sure either. Maybe the reference makes 100% sense, and it's just me who finds it confusing in that matter.

Done. (Wasn't sure if the URL was stable, but guess it is.)

Oh, it certainly has potential for improvements. I do feel like the reference has more problems than active contributors working on it, so with a vague idea of "this should be more clear" it might take a while until someone fixes the issue. W.r.t. the "drop scopes" topic, I do however think there’s one fairly straightforward change that should be made: Clarify that throughout the remainder of that reference page the word "scope" refers to the "drop scope"s as defined in the beginning of the page. (Or alternatively refactor the page such that the terminology is either consistently "drop scope" or consistently "scope.)

I don’t think the structure of

  • declare what scopes there are and how they’re nested
    • includes information on drop order for variables/temporaries associated to those scopes

and then

  • specify in detail what variables/temporaries are associated to which scopes

is bad; it’s just important that this very structure becomes more clear.

I agree, but if the "drop scope" "for" an "expression" is defined, it is pretty confusing if that "drop scope" isn't the scope that (when left) drops the variable or temporary.

It doesn't seem easy to give a good/simple suggestion how to fix that, though. Renaming all "drop scopes" to "scopes" might have other problems.

I might think further on it, but I'd need time to think about it before I open an issue. I understand too little of the reference yet, and I have no overview. And you're probably right that a rough note like "this should be more clear" isn't much helpful.

It’s not necessarily not helpful, it’s just not very actionable. Hence my idea to also include the concrete, actionable suggestion that the reference should at least explicitly point out that/where the term "scope" refers to "drop scope". For a minimal viable improvement that amounts to adding one sentence around the end of the "drop scope" section remarking that in the subsequent parts "scope" is used for "drop scope" terminology-wise. But of course it also allows whomever would decide to pick up the issue to go further and include further improvements / restructuring of the reference text. So I guess, I do think it’s reasonable to open an issue here. I don’t feel like spending the time writing up an issue myself though :sweat_smile:

Of course, feel free to take your time and do your own thinking first. There’s no real urgency here.

I think, reading some other parts of the reference is a reasonable way of learning more about Rust in general. And when in doubt writing small examples and testing compiler-behavior or runtime-behavior is a great approach in order to get to understand everything even better. You’ll be getting to no longer having those feelings in no time :wink:

Note that the reference is not so much a whole coherent thing and more a collection of individual pages about individual aspects of the language; so I wouldn’t know what exactly the criterion of “having overview” should entail.

1 Like

Edit/Note: There are several mistakes I made in this post. See answer.


After trying to write a text for opening another issue on GitHub, I think I finally understood what's going on (at least what happens for real, disregarding any issues concerning confusing wording or not in the reference).

I should have paid more attention to what you wrote here:

The key point is, this happens "often" but not always (which also gets more clear when following the link to "place context" as suggested by you).

Consider the following code:


use std::cell::RefCell; 
use std::ops::Deref; 
fn main() { 
    let cell: RefCell<i32> = RefCell::new(7); 
    let result: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref()); 
    println!("Result = {:?}", result); 
}

(Playground #1)

Would you expect this to

  • fail at compile-time,
  • fail at run-time,
  • or succeed?

And compare with:

use std::cell::RefCell;
//use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    //let result: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    let result: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result);
}

(Playground #2)


Now to make things really wicked, try:

use std::cell::RefCell;
use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    let result: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    let result: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result);
}

(Playground #3)


As of rustc 1.56.1, the behavior is as follows:

  1. Compile-time error:
    error[E0716]: temporary value dropped while borrowed
  2. Run-time error:
    thread 'main' panicked at 'already borrowed: BorrowMutError', src/main.rs:6:61
  3. Run-time error:
    thread 'main' panicked at 'already borrowed: BorrowMutError', src/main.rs:5:65

Now that's a bit surprising, and I would say there is a compiler bug involved in case of #3 (as uncommenting the line 6 with &* will remove the compiler-time error in the above line 5 and lead to a run-time error in the above line 5). Note that you'll get a compile-time error if you rename the second result to result2 in example #3.

But let's look at #1 first: In the expression cell.borrow_mut().deref(), the RefMut value returned by borrow_mut() indeed gets dropped after that expression has been evaluated, thus causing a compile-time error because the result of std::ops::Deref::deref has the same lifetime of its argument, and its argument gets dropped. The section "Temporary scopes" does not apply here, because apart from promotion, it is applied only for place contexts.

I would thus say that the section "Temporary scopes" only defines exceptions, but not the usual association of a drop scope to a variable or temporary.

Regarding #2, this compiles without error because the dereference operator is explicitly mentioned in the subsection on "Place Expressions and Value Expressions". So it's actually the dereference operator that makes things work special here, as you already quoted here:

Sorry for not understanding that earlier, maybe you were aware of all this before. At least I wasn't and/or misunderstood the implications of all of that. :sweat_smile: I would conclude the reference is okay, and there is no problem with it. But it's still very hard to grasp.

As for case #3, I think this is a (minor) compiler-bug triggered when the two variables have the same name? :thinking:

P.S.: Maybe it's not that "minor" because the compiler doesn't notice that a dropped value still gets used?


P.P.S.: Maybe it's not a bug but an optimization? If I uncomment the following println!, the compiler will correctly detect the line above as an error:

use std::cell::RefCell; 
use std::ops::Deref; 
fn main() { 
    let cell: RefCell<i32> = RefCell::new(7); 
    let result: (&i32, &i32) = (cell.borrow().deref(), cell.borrow().deref());
    //println!("Result = {:?}", result); 
    let result = 5;
}

(Playground)


P.P.P.S.: Regarding example #2, it was maybe a bad idea to combine & and let, as this can lead to temporary lifetime extension again. That makes it a bad example. Now I'm confused, and maybe I still misunderstand a lot of things, and my post isn't correct. I will need time to think about all of this.

The "often" qualifier was supposed to account for

  • expressions like match/if/blocks/etc
    • where the temporary wil life shorter than the statement, though I guess there often is another smaller statement within these constructs, so the rule "innermost containing statement" holds pretty often
  • temporary lifetime extension

Fail at compile-time. The &i32 references are referenceing the result of a &self -> &i32 method call on a temporary. The temporary gets dropped after the statement, but the &i32 is used later in the println.

Judging by the fact that I learned from this thread that &*EXPR qualifies for lifetime-extending the temporary variable holding the value of EXPR, this only fails at run-time.

Also fails at run-time. The first result is never used again, so unlike the example #1, there’s no problematic println usage. The second result is the same as the example #2.

Nice, looks like I was right! :slight_smile:


That’s hard to follow with the line numbers. Also line 6 in #3 is kind-of “already uncommented”? Anyways, you might be missing shadowing here.

use std::cell::RefCell;
use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    let result: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    let result: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result);
}

is the same as

use std::cell::RefCell;
use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    let result1: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    let result2: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result2); // <- look here!
}

while

use std::cell::RefCell;
use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    let result: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    // let result: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result);
}

is the same as

use std::cell::RefCell;
use std::ops::Deref;
fn main() {
    let cell: RefCell<i32> = RefCell::new(7);
    let result1: (&i32, &i32) = (cell.borrow_mut().deref(), cell.borrow_mut().deref());
    // let result2: (&i32, &i32) = (&*cell.borrow_mut(), &*cell.borrow_mut());
    println!("Result = {:?}", result1); // <- look here!
}

Oh, no, temporary scopes definitely do apply. You’ll have to desugar the methods first!!

cell.borrow_mut().deref() desugars to <cell::RefMut<'_, i32> as Deref>::deref(&RefCell::borrow_mut(&cell)). Or in shorter pseudo-syntax deref(&borrow_mut(&cell)). The &EXPR reference expressions are the only place contexts here, the &cell takes an actual place expression “cell”; only &borrow_mut(&cell) is given a value expression, “borrow_mut(&cell)” in a place expression context, so the value of borrow_mut(&cell) (of type cell::RefMut<'_, i32>) is placed into a temporary that’s associated to the drop scope of the entire let result: ... = ...; statement.


Thinking about example 2 again, reading the relevant section again,

Extending based on expressions

For a let statement with an initializer, an extending expression is an expression which is one of the following:

So the borrow expressions in &mut 0 , (&1, &mut 2) , and Some { 0: &mut 3 } are all extending expressions. The borrows in &0 + &1 and Some(&mut 0) are not: the latter is syntactically a function call expression.

The operand of any extending borrow expression has its temporary scope extended.

Examples

Here are some examples where expressions have extended temporary scopes:

// The temporary that stores the result of `temp()` lives in the same scope
// as x in these cases.
let x = &temp();
let x = &temp() as &dyn Send;
let x = (&*&temp(),);
let x = { [Some { 0: &temp(), }] };
let ref x = temp();
let ref x = *&temp();

I do get the feeling that the reference does not properly explain why the dereference expressions qualify for lifetime extension. The examples contain multiple uses of deref, i.e. (&*&temp(),) and *&temp(), but the bullet list only mentions “borrow expression” (which is &EXPR and &mut EXPR).


No bug at all, you seem to be confused by shadowing (as explained above) and the fact that example #1 does compile if you remove the println.