Rvalue Temporaries, References & Their Lifetimes

I've got a bit of a mystery on my hands, and I wonder if some kind soul out there can help explain it to me.

Take the following code for example:

fn main() {
    let foo = 42;
    let bar = &foo;
    assert!(bar == &42);
}

Rust Playground

In the above code, both of the comparison operator's operands are references. Note the rhs is a reference to a temporary (value with no binding, and statement level scope). From both my and the compiler's perspective, all is well.

Now consider:

fn main() {
    let foo = 42;
    let bar = Some(&foo);

    assert!(bar == Some(&42));
}

Rust Playground

The above code does not compile. The compiler complains that a temporary value is dropped while it is still borrowed on the assert!() line. But when I look at this code, I don't see it.

So let me walk though my understanding of what's going on, so you can help me understand where I am going wrong. Let's annotate the assert!() line with 7 main steps:

//     |--6   5--|    2--|  |--1
//     v         v       v  v 
    assert( bar == Some( & 42 ) ) ; 
//          ^       ^           ^
//       4--|    3--|        7--|

Here is a breakdown of the flow as I understand it:

  1. The value 42 (inferred to be an i32) is to be supplied in immediate mode by the compiler.
  2. The ref requires an address to be taken, so instead of supplying an immediate value, the compiler stores the value 42 in pre-allocated static memory, with statement-level scope (valid scope for 42 ends at the semicolon at the end of the line).
  3. a) Some's tuple-like struct constructor consumes the ref of the value (not the value itself).
    b) The constructor returns an initialized Some value containing a reference to the temporary 42 value. Put another way, this instance contains a borrow against the temporary 42 value.
    c) This returned Some value is not assigned to anything, so it itself is an Rvalue (temporary)
  4. bar is evaluated to be a Some value containing a ref to foo.
  5. a) PartialEq() (aka ==) is fed both the lhs (the Some value containing ref to foo) and the rhs (the temporary Some value containing ref to temporary 42 value with statement-level scope).
    b) PartialEq() returns a boolean value of true.
    c) the returned boolean value is not assigned to anything so it is an Rvalue (temporary)
  6. assert!() consumes the return temporary boolean value of true and decides not to panic.
  7. a) We've reached the end of the statement, so all Rvalues (statement level scope) are dropped in the reverse order they were created. (Note in this analysis, I arbitrarily chose to create PartialEq()'s rhs argument first. Since there is no interdependency between its lhs and rhs arguments, they could be created in either order and this flow still works).
    b) boolean true value was consumed, but if there is any internal implementation cleanup to be performed, it is dropped now.
    c) The rhs returned temporary Some() (step 3c) containing a reference to the temporary 42 is dropped. This concludes the borrow of the temporary value 42.
    d) The rhs temporary ref (step 3a) was consumed by Some's tuple-like constructor, but if there is any internal implementation cleanup to be performed, it is dropped now.
    e) The scope for the temporary 42 is closed. All outstanding borrows have been concluded so it is valid to drop/close the scope.

As you can see, in my little world, everything is as it should be, with all borrows being terminated before the referant (underlying value) leaves scope. Unfortunately, the compiler strongly disagrees with me. What am I missing?

2 Likes

It seems like you've made some assumption about the way that Rust's lifetime analysis works (I'm guessing based on how references work in C++). This statement in particular is not an accurate representation of how the borrowchecker works today, AFAIK:

Obviously this code could work with a better analysis. There's not a standard yet for Rust's memory model.

I'm no expert at compiler details, but I can at least share my observations. :slight_smile:

A temporary value does not automatically go out of scope at the end of the statement. Consider this example (playrust):

fn main() {
    let temporary = &String::from("!");
                  // ^~~ temporary value created
    
    print!("Hello world");
    println!("{}", temporary); // <-- lives until here
}

But this doesn't work (playrust):

fn main() {
    let temporary = Box::new(&String::from("!"));
                           // ----------------- ^ temporary value dropped
                           // |                   here while still borrowed
                           // temporary value created here
    
    print!("Hello world");
    println!("{}", temporary);
}

To me that looks like the compiler is smart, but not yet smart enough. It seems like there is some magic lifetime inference going on, which only works in simple cases.

If I am not mistaken these whole automatism even for the simple case is still rather new (i.e. introduced after Rust 1.0.0), so I'd expect some improvements in the future here. :slight_smile:

@withoutboats, yes, I'm coming to Rust from C++. And you're right--I shouldn't imply this level of specificity, I just mean now mustn't be an immediate value (if it ever was going to be), but rather an addressble one (in static memory or as a hidden local on the stack or some other technique--as long as the value's address can be taken).

@colin_kiegel, re: magic lifetime inference, while trying to debug this issue, I think I read somewhere that when a temporary is directly assigned, its lifetime is extended to that of the binding. But in both my Some() case and your Box::new() case, the reference isn't being assigned.

Thank you both--I hadn't considered the possibility that the lifetime rules for temporaries weren't yet "complete"...

I'll look to see if there are any discussions/RFPs going on in the Rust community around all of this. Many thanks! :slight_smile:

This line in fact doesn't compile, but for whole different reason. The following line:

assert!(Some(&foo) == Some(&42));

does compile, and the 42 is a value that lives until the end of statement (the ;) – I think this matches your mental model.

I think the problem with this weird behaviour is in PartialEq implementations. There exists:

impl<'a, 'b, A, B> PartialEq<&'b B> for &'a A
where A: PartialEq<B> + ?Sized, B: ?Sized

and

impl<T> PartialEq<Option<T>> for Option<T> where T: PartialEq<T>

Note that the implementation for Option forces both values to have exactly the same type! But this is not true in our case: bar contains a reference that should be valid till the end of main, but Some(&42) contains a reference with a shorter lifetime. Because the references have different lifetimes, the PartialEq impl for Option doesn't apply.

I don't quite understand why can't those references be temporarily shortened (just as it's possible to call fn foo<'a>(&'a u8, &'a u8) with differently-lifetimed references). I also don't understand, why this assert does compile!

assert_eq!(bar, Some(&42));
3 Likes

Option<T> should be variant over T & I don't know it wouldn't be here.

Very interesting, @krdln!

I also tried testing my own simple wrapping generic struct to see if the problem was with the struct-like tuple constructor or not. I got the same behavior calling a simple static method. I do believe I used #[derive(PartialEq)] on it, though, so I may have fallen victim to the "both operands must be the same type" issue.

Amazing that assert_eq!() works though--definitely some inconsistent behavior going on here... Thank you for these observations!

Here is the expansion of assert! vs assert_eq!

//assert
if !(bar == Some(&42)) { panic!() }

//assert_eq
match (&bar, &Some(&42)) {
    (left_val, right_val) => {
         if !(*left_val == *right_val) { panic!() }
    }
}

This match is in order to take the two arguments by reference. It seems that this causes the variance to work correctly for some reason.

2 Likes

It would be nice to get this to resolve consistently and correctly in the language...

@withoutboats, that little discovery about match could serve as a workaround, in a pinch. Thanks for posting the expansions!

If you could post a bug report on the issue tracker, this could get triaged by people who understand the internals well.

1 Like

(side note: please avoid using '&' as a shorthand for "and" when talking about Rust. It took me a moment to figure out that it wasn't some strange "T &" type notation.)

4 Likes

Good idea--done. Posted here.

My guess is this works not because of variance, but rather the temporary (Some(&42)) lifetime is extended because it's assigned/bound to an lvalue in the match arm.

I think the function/method call is doing something to limit the lifetime of the temporary. Compare:

fn ident<T>(x:T) -> T {x}
fn main() {
    let temporary = ident(&42);
    println!("{}", temporary);
}

(which fails to compile); with:

fn ident<T>(x:T) -> T {x}
fn main() {
    let temporary = &ident(42);
    println!("{}", temporary);
}

which runs perfectly happily.

It is the intervening function/method call which somehow disturbs the compiler's determination of the lifetime of the object referred to.

Finally, compare it to:

fn main() {
    let temporary = { let x = &42; x };
    println!("{}", temporary);
}

which fails with exactly the same error (but more obviously) than before.