Help: borrow lifetime seems to live too long

Yandros · July 20, 2018, 10:22am

I am trying to code a "cached" function: i have some container (called Foo in the example below) and would like to get a reference to its content if any, else compute that value and store it before returning a reference to it.
In the examples below, bad and bad_unsugared fail and I have an idea as of why.
However, bar does work (instead of returning a reference I use the value right away), but baz does not.
I keep trying to understand how I could be in violation of Rust borrowing rules on runtime but I can't find the problem.
Can you help me understand it? Thanks.

trait Foo {
  fn get<'a> (self: &'a Self) -> Option<&'a str>;
  fn insert (self: &mut Self, _: String);
}

fn bar (foo: &mut Foo) // works !
{
  if let Some(s_ref) = foo.get() {
    return println!("{}", s_ref)
  }
  foo.insert(42.to_string());
  bar(foo)
}

fn bad (foo: &mut Foo) // error
{
  if let Some(s_ref) = foo.get() {
    println!("{}", s_ref)
  } else {
    // idea: None: Option<&'borrow_lt str> here explains the error
    foo.insert(42.to_string());
    bad(foo)
  }
}

fn bad_unsugared (foo: &mut Foo) // error
{
  match foo.get() {
    Some(s_ref) => println!("{}", s_ref),
    _ => { // idea: None: Option<&'borrow_lt str> here explains the error
      foo.insert(42.to_string());
      bad_unsugared(foo)
    },
  }
}

fn baz (foo: &mut Foo) -> &str // error??? why?
{
  if let Some(s_ref) = foo.get() {
    return s_ref
  }
  // idea ??
  foo.insert(42.to_string());
  foo.get().unwrap()
}

fn main () {}

(Playground)

Errors:

   Compiling playground v0.0.1 (file:///playground)
error[E0502]: cannot borrow `*foo` as mutable because it is also borrowed as immutable
  --> src/main.rs:21:5
   |
17 |   if let Some(s_ref) = foo.get() {
   |                        --- immutable borrow occurs here
...
21 |     foo.insert(42.to_string());
   |     ^^^ mutable borrow occurs here
22 |     bad(foo)
23 |   }
   |   - immutable borrow ends here

error[E0502]: cannot borrow `*foo` as mutable because it is also borrowed as immutable
  --> src/main.rs:22:9
   |
17 |   if let Some(s_ref) = foo.get() {
   |                        --- immutable borrow occurs here
...
22 |     bad(foo)
   |         ^^^ mutable borrow occurs here
23 |   }
   |   - immutable borrow ends here

error[E0502]: cannot borrow `*foo` as mutable because it is also borrowed as immutable
  --> src/main.rs:31:7
   |
28 |   match foo.get() {
   |         --- immutable borrow occurs here
...
31 |       foo.insert(42.to_string());
   |       ^^^ mutable borrow occurs here
...
34 |   }
   |   - immutable borrow ends here

error[E0502]: cannot borrow `*foo` as mutable because it is also borrowed as immutable
  --> src/main.rs:32:21
   |
28 |   match foo.get() {
   |         --- immutable borrow occurs here
...
32 |       bad_unsugared(foo)
   |                     ^^^ mutable borrow occurs here
33 |     },
34 |   }
   |   - immutable borrow ends here

error[E0502]: cannot borrow `*foo` as mutable because it is also borrowed as immutable
  --> src/main.rs:43:3
   |
39 |   if let Some(s_ref) = foo.get() {
   |                        --- immutable borrow occurs here
...
43 |   foo.insert(42.to_string());
   |   ^^^ mutable borrow occurs here
44 |   foo.get().unwrap()
45 | }
   | - immutable borrow ends here

error: aborting due to 5 previous errors

For more information about this error, try `rustc --explain E0502`.
error: Could not compile `playground`.

To learn more, run the command again with --verbose.

vitalyd · July 20, 2018, 10:32am

These are known limitations with the current borrowck algorithm. NLL fixes some of your cases but not the early return one - that requires Polonius as well; you can read more about that aspect in Baby Steps

Yandros · July 20, 2018, 10:37am

Thank you for that (fast) answer. I was worried Rust was guarding me against some weird configuration I didn't think of.
So, while waiting for the Polonius borrow checker, do you think that writing return unsafe { &*(s_ref as *const _) } can be acceptable?

vitalyd · July 20, 2018, 10:45am

It should be sound, but whether it’s acceptable is in the eye of the beholder

Your example looks very similar in spirit to the motivation behind HashMap::entry(), so you can consider using a similar approach if none of the other safe alternatives work.

I should’ve linked you the other useful NLL blog post, which provides more color on the topic: Non-lexical lifetimes: introduction · baby steps

dodomorandi · July 20, 2018, 11:07am

What if you set the lifetime of the returned string to be bound to the lifetime of the trait (instead of being the same)? Something like this? I do not know if this could be applied to your real-world situation...

vitalyd · July 20, 2018, 11:18am

I’m surprised that works here - the 'b:'a bound doesn’t actually change anything, in theory, because 'b == 'a in this case.

dodomorandi · July 20, 2018, 11:28am

To be honest, I initially tried using NLL, then I got the error about the lifetime of the returned str. I tried this solution without expecting it to work without NLL. But using the stable it compiles fine...

What I thought is that the compiler cannot return a reference to a lifetime that, since we put a new String inside Foo, must have a shorter lifetime than Foo. Otherwise you should not be able to replace it with a new one.

But this is just how I reasoned about the fact that it compiles without NLL and I did not expect it to work...

Yandros · July 20, 2018, 11:36am

@dodomorandi Even if at first your solution seems to work, the problem is still there:

either we also have 'a: 'b in which case 'b == 'a as @vitalyd pointed out, and the problem comes back.
or we do not have 'a: 'b (unsatisifable except for 'b == 'static) and it works, but we really are no longer in the same situation

Try adding those constraints or try to implement Foo with something like

struct FooImpl(Option<String>)
impl Foo for FooImpl {
  fn get<'a, 'b> (self: &'a Self) -> Option<&'b str>
  where 'b: 'a
  {
    match self.0 {
      Some(ref s) => Some(s), // cannot satisfy &'b _ without 'a: 'b since s: &'a _
      _ => None,
  }

  fn insert(self: &mut Self, s: String) {
    self.0 = Some(s)
  }
}

vitalyd · July 20, 2018, 11:39am

The issue that Polonius fixes is early returns extending the borrow out into the caller, thus preventing subsequent mutation in the function. So the issue is really that the borrow of foo is extended, and not related to the lifetime of the returned str.

So compiler can't really tell that we're "inserting" a String since these are just trait methods with a signature - it only goes by signatures and, similarly, cannot tell that we're replacing anything (in fact, I can't tell that either without seeing a specific impl ).

I think what will end up happening is you won't be able to implement this in the real case.

EDIT: Oh, I see @Yandros just replied above with essentially the same sentiment.

vitalyd · July 20, 2018, 11:44am

@Yandros, a couple of unrelated comments/questions for you that perhaps you'll find worthwhile:

You don't need to write self: &mut Self - it can be just &mut self. Similarly for the immutable method.
Are you intentionally using trait objects in your example functions? i.e. &mut Foo and &Foo are trait objects.

Yandros · July 20, 2018, 11:47am

@vitalyd

trait objects were here to alleviate the function signatures in the examples
I force myself to write self: &Self and self: &mut Self since sadly the &self and &mut self sugars are not consistent with rust reference pattern-matching, see https://github.com/rust-lang/rust/issues/52130

dodomorandi · July 20, 2018, 11:50am

@vitalyd @Yandros you are totally right! My fault was just thinking that if it was compiling, than it is ok. How noob!

Interesting enough: with the example written by @Yandros the error given by the NLL is much, much clearer that you get from stable:

argument requires that 'a must outlive 'b

instead of a very long explanation of what is going on.

vitalyd · July 20, 2018, 11:52am

Ok, was just checking - some people new to Rust don't realize they're using trait objects (and all that entails) in cases like that.

Interesting perspective. FWIW, I never found the reference vs pattern usage of & to be confusing, but I can see how this is a YMMV scenario. I do think using the long form like you have is not canonical, and you'll likely not see it much (and conversely, someone looking at your code may find it unconventional). But ok, again, just wanted to make sure you were aware of the shorthand.

Yandros · July 20, 2018, 12:01pm

@vitalyd don't worry, my example code is poorly written (cough Foo bar baz cough) and your remarks are good (trait objects are an important subtlety indeed).
I raised the inconsistency issue when I realised that in a closure taking references (classic iteration situation) we write

|&x| { .. /* x is an owned value here, &x is the ref */ }

but when writing a method, I found myself wanting to write &self to feed a reference in the method body when the argument was &self.

I really guess it's just a matter of convenience

vitalyd · July 20, 2018, 12:05pm

It's actually perfectly written for example/illustration code - short and sweet .

Topic		Replies	Views
Returning a mutable reference extends its lifetime? help	4	1765	August 25, 2019
Necessary mutable borrow? help	2	429	December 12, 2020
Lifetime of borrowed values in struct help	18	6885	January 12, 2023
Lifetime error help	2	447	May 18, 2023
Borrowed value does not live long enough (as_byte()) help	4	940	January 12, 2023

Help: borrow lifetime seems to live too long

Related Topics