Multiple mutable methods

I think I did a bad job of commincating my point. Let me try again. While what you said is true and relavant, I don't think that it leads to a better understanding of lifetimes for those just starting to learn lifetimes. Using the word outlives, while technically wrong, gives a good enough understanding of lifetimes to begin working with them. Not enough to work with unsafe code just yet, but enough to get started. It is also much simpler to understand. So for these reasons I think it is more productive to teach in terms of outlives rather than lives at least as long as. Then when they start working with unsafe code their intuition about lifetimes will be strong enough that they should be able to transition to thinking in terms of the actual statement "lives at least as long as".

1 Like

I perceive this as an issue of phrasing. How about "has at least as long a lifetime". The problem I have with stating a non-reflexive relationship is that it may lead people new to Rust to look for sources of longer lifetimes than they need, simply because they didn't understand that, in many cases, a single shared lifetime was sufficient. If we weren't trying to avoid the negative, we could just say that "the input lifetime can't be shorter than the derived lifetime", etc.

Edit: Actually, due to drop order, "outlives" is often the actual requirement. I stand corrected.

So I thought this was representative of what I was running into... but this works fine, even though current is pointing at the same thing as HashMap o_O

I'm probably doing something weirder in my actual codebase, will hopefully get a chance to check later today - but I found it interesting that this works fine:

use std::collections::HashMap;

struct Foo<'a> {
    lookup: HashMap<u32, &'a str>,
    current: Option<&'a str>
}

impl<'a> Foo<'a> {
    fn new() -> Self {
        let mut lookup = HashMap::new();
        lookup.insert(42, "secret to life, the universe, and everything");
        
        Self{
            lookup,
            current: None
        }
    }
    
    fn calc(&mut self) -> &str {
        if self.current.is_none() {
            self.current = Some(*self.lookup.get(&42).unwrap());
        }
        self.current.unwrap()
    }
}

fn main() {
    let mut foo = Foo::new();
    
    println!("{}", foo.calc());
}
1 Like

Not always: 'a : 'b does define a partial Order between lifetimes, meaning that it is reflexive and anti-symmetrical (it constraints that 'a >= 'b).

If the constraint were 'a > 'b, then the following code would not compile:

fn foo<'a, 'b> (x: &'a String) -> &'b String
where
    'a : 'b, // needed for reborrowing
    'b : 'a, // unneeded but allowed
    // both bounds imply that `'a = 'b`
{
    &*x
}

Many things could be said to defend &self notation (main point being that it allows writing methods slightly faster), but consistency is definitely not one of those! The notation is totally inconsistent with pattern matching notation, breaking symmetry with methods such as fn method (&self, other: &Self), and preventing the following code from compiling:

/* targeted refactoring:
fn count_non_nulls<T : IsZero> (slice: &'_ [T]) -> usize
{
    slice
        .iter().cloned()
        .filter(T::is_zero)
        .count()
}
*/

fn count_non_nulls (slice: &'_ [i32]) -> usize
{
    slice
        .iter().cloned()
        .filter(|&slf: &'_ i32| -> bool {
            slf == 0
        })
        .count()
}

trait IsZero { fn is_zero (self: &'_ Self) -> bool; }

impl IsZero for i32 {
    fn is_zero (&self: &'_ Self) -> bool { // error
        self == 0
    }
}

In this case we must sacrifice method notation or pattern matching notation in the function signature for the code to compile.

1 Like

That's not what he was saying. He was originally saying that ourlives would give people the wrong idea, but he retracted that due to drop order. He was saying that 'a: 'b means 'a >= 'b. But if we are getting technical, with nll and more so with polonius, it will mean that 'a is a subset of 'b. But this is even more abstract than lives at least as long as, so I still think that teaching outlives will more be productive than either of those two.

Fwiw, for me it's not really the concept of > vs >= but the simplicity of having a single and active word as a mnemonic.

If there's an equivalent single word for "outlives" which means "lives at least as long as", then great! all the better!

1 Like

Actually, looking back at your second example in the comments, that should work if you remove lifetime annotations of Self references. I didn't think about it before posting my first answer.

1 Like

Using outlives as a shorthand for lives at least as long as is common but can indeed be ambiguous.

For instance, in @RustyYato's post

outlives is used twice with different meanings.

In general strict outliving is just a particular case of not living at least as long as:

  • in the example above, we would have:

    • if 'a strictly outlives 'b, then 'b cannot live at least as long as 'a, and we have !('b : 'a)

    • Since for all lifetimes 'x, and for all types T : 'x, &'x T "lives for 'x" (i.e., for all lifetimes 'y,
      &'x T : 'y => 'x : 'y),

      with T = Foo<'a>, 'x = 'b and 'y = 'a, we get that

      • for the borrow &'x T to be valid, we need that T : 'x , i.e., Foo<'a> : 'b and 'a : 'b

      • &'x Foo<'a> : 'y => 'x : 'y, i.e.,
        &'b Foo<'a> : 'a => 'b : 'a,

In other words, having a &'b Foo<'a> : 'a bound is a convoluted way of requiring that 'b be 'a.

&'b Foo<'a> : 'a
        |      |
        |      |
        v      v
    higher    lower
    bound     bound
      'a≥ 'b  ≥'a

In a more natural way: if a (shared) borrow (to a type living only for 'a, thus requiring that such borrow not strictly outlive 'a) needs to live at least as long as 'a, then both constraints force that borrow to live exactly 'a.

3 Likes

I used th term consistency in the context of user ecosystem, not syntex-wide. In fact I'm a huge fan of syntactic sugars like ? XD

1 Like

So this is a bit closer to what I actually have in my project:

playground

use std::collections::HashMap;

struct Foo<'a> {
    lookup: HashMap<u32, Bar<'a>>,
    current: Option<&'a Bar<'a>>
}

struct Bar<'a> {
    message: &'a str
}

impl<'a> Foo<'a> {
    fn new() -> Self {
        let mut lookup = HashMap::new();
        lookup.insert(42, Bar{message: "secret to life, the universe, and everything"});
        
        Self{
            lookup,
            current: None
        }
    }
    
    fn calc(&'a mut self) -> &str {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap());
        }
        self.current.unwrap().message
    }
}

fn main() {
    let mut foo = Foo::new();
    
    println!("{}", foo.calc());
}

And interestingly it fails to compile if I don't add the 'a to self in calc() ...

Also tried fn calc(&mut self) -> &'a str { and get the same error.

The error is confusing though... I tried writing out how I understood it here - but that just made things more confusing. So if someone doesn't mind:

  1. Run the playground without the 'a and help explain the errors (here's a permalink)

  2. In trying to understand it - I don't get how &self could ever outlive 'a ... since 'a is defined as the lifetime of Foo, musn't any anonymous reference by definition be shorter than 'a ?

Yes, now try and use foo after you call calc.

Heres one way to see how lifetime analysis is working.

First the definition of Foo

struct Foo<'a> {
    lookup: HashMap<u32, Bar<'a>>,
    current: Option<&a' Bar<'a>>
}

Here we are saying that the lifetime in current is the same as in lookup. Simple stuff, binding lifetimes together.

Next, in Foo::new

    ...
    fn new() -> Self {
        let mut lookup = HashMap::new();
        lookup.insert(42, Bar{message: "secret to life, the universe, and everything"});
        
        Self{
            lookup,
            current: None
        }
    }
    ...

Because string literals have the type &'static str, they subtype &'a str. So it is safe to use &'static str where &'a str is expected (in lookup). So now you can create an arbitrary Foo<'a>.

Finally on to calc

    fn calc(&mut self) -> &str {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap());
        }
        self.current.unwrap().message
    }

Here the important thing to note is the assignment to current. This is important because the type of current is &'a Bar<'a> (remember this, as it will be important shortly).

Also note that we have self behind a &mut, this is important because &mut T is invariant over T. What this means for lifetimes is that the &mut &'a T is only equal to &mut &'b T if and only if 'a == 'b (ignoring the lifetimes associated with the &'0 mut, labeled '0 here).

What this means for current is that, this assignment will determine the lifetime 'a.

Now, that we got that, let's desugar calc.

    fn calc<'b>(&'b mut self) -> &'b str {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap());
        }
        self.current.unwrap().message
    }

We now have some other lifetime 'b. Now let's examine the assignment. we are assigning to current, what are we assigning. If we look at the RHS we see, Some(self.lookup.get(&42).unwrap()), which is of the type Option<&'b Bar<'a>>. But we are trying to assign it to the LHS, current, which has the type Option<&'a Bar<'a>>.

Oh no! What if 'a: 'b, and 'a != 'b. This would mean that this assignment is invaid! It would produce a dangling pointer. Hence why &mut T is invariant over T. Because of this, Rust checks that 'a == 'b, but it can't enforce this in general because you could pick 'b to be any lifetime that you want.

So if we correct this signature, and enforce that &'a mut self, then 'a == 'b, and everything works out.

So

    fn calc(&'a mut self) -> &'a str {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap());
        }
        self.current.unwrap().message
    }

works. Now you may notice that I added a lifetime to &'a str. Rust does this desugaring automatically, it is called lifetime elision, and it makes working with lifetimes painless in the normal case, and it is what your lifetime annotated code desugars to :slight_smile: .

Note that this also works

    fn calc<'b>(&'b mut self) -> &'b str
    where 'b: 'a {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap());
        }
        self.current.unwrap().message
    }

I will leave this to you to figure out why.


edited because I was wrong in my previous atempt at this post, lifetimes are hard and I missed the actual problem with the code. I aslo didn't consider variance so that led me astray. But this is the correct reason for why calc didn't compile without annotation.

edit 2: formatting and wording

8 Likes

Wow... okay... starting to get it a little more! Not quite there, but closer :slight_smile:

OK... two questions:

  1. How is RHS typed as Option<&'b Bar<'a>> ? i.e. if self is &'b mut self, where is the 'a coming from... wouldn't it be Option<&'b Bar<'b>> ?

  2. Similarly, since self is defined as &'b mut self, isn't the LHS current also Option<&'b Bar<'a>> (or Option<&'b Bar<'b>>)?

I know it's almost exactly speaking to what you said here:

But I don't quite get how that's true if self is &'b mut self

Man it would be so nice if someone made like an interactive lifetime analysis tutorial thing

For example, in something like the style of the demo under the "Who's Right" section on this JS tasks post

1 Like

The reason for that is because of the signature for HashMap::<K, V>::get which is fn<'c, Q>(&'b self, &Q) -> Option<&'c V> where Q: Borrow<K>. Effectively the gives you a reference with the same lifetime as self but it doesn't change the lifetime inside V. So applying this to our example, we find that the concrete HashMap we are using is HashMap<u32, Bar<'a>> because that is the type of lookup. This means that the concrete HashMap::get function has the signature fn<'c, Q>(&'c self, &Q) -> Option<&'c Bar<'a>> where Q: Borrow<u32>. Note we have to copy and paste V because that is how generics work, so we can't change the lifetimes inside of V. This is why we get Option<&'b Bar<'a>> for the type of the RHS.

The type of current depends of Self not &'b mut Self. So no, current's type does not depend on 'b. If this was allowed, then current would change types depending on where it was called. Which doesn't match our intuition on types.


I think the confusion is coming from liferimes infecting everything. But this isn't how lifetimes work. Think of lifetimes as generic parameters along the lines of generic type parameters and this will start to make more sense.

1 Like

Okay - think I'm going to print up this page and sit with it over a cup of tea till I understand it, or at least before responding with my next bit of confusion :wink:

Definitely! In fact I even caught myself holding onto the idea of the lifetime affecting Foo as a whole instead of the individual objects it holds

1 Like

@RustyYato's explanation was great, I'll just add a comment on the following pattern, when looking at it with some "distance":

struct Foo<'values> { // I like explicitly named lifetimes
    lookup: HashMap<u32, Bar<'values>>,
    current: Option<&'values Bar<'values>>,
}

impl<'values> Foo<'values> {
    fn calc (&'values mut self) -> &'values str
    {
        if self.current.is_none() {
            self.current = Some(self.lookup.get(&42).unwrap()); // Self referential
        }
        self.current.unwrap().message
    }

This patterns creates a self-referential struct (hence the 'values lifetime requirement on the &mut self borrow, explained by @RustyYato), which in practice prevents the programmer from using that struct ever again (the main reason you may be having these borrowing issues):

  • Foo<'values> cannot live longer than the lifetime of the values it contains, 'values.

  • fn calc (&'values mut self, ...) -> ... this borrows mut-ably (i.e., in a unique / non-sharing fashion) your Foo<'values> for the whole 'values lifetime.

Since the struct cannot live longer than 'values, and since it is mut borrowed during the whole 'values lifetime, it is mut borrowed until it dies. And since a mut borrow is exclusive (the very definition of a mut borrow), after calling the .calc(...) method, it is impossible to access the struct ever again.

The solution? Do not use lifetimes / borrows for interior references.

In your case, you could just have:

struct Foo<'values> {
    lookup: HashMap<u32, Bar<'values>>,
    current: Option<u32>, // index / key instead of a reference
}

impl<'values> Foo<'values> {
    fn calc (&'_ mut self) -> Option<&'_ str> // anonymous lifetime can be shorter than `'values`
    {
        if self.current.is_none() {
            self.current = Some(42); // the line that constrained `'_` to be as big as  `'_ values` is no longer here
        }
        self.lookup
            .get(self.current.unwrap())
            .map(|bar| bar.message)
    }

     /// or more simply (no longer requiring mutation)
    fn calc (&'_ self) -> Option<&'_ str> // anonymous lifetime can be shorter than `'values`
    {
        self.lookup
            .get(self.current.unwrap_or(42))
            .map(|bar| bar.message)
    }
3 Likes

In fact, the reason I used std::cell::Cell in my example of self-referential types is because of what @Yandros said. By using Cell I can avoid the mutable reference, and thus uniquely borrowing the type and so I can still use it later, even if I can't move it after using the self-referential method.

1 Like

So - as sortof a tangent, I was thinking about this... and it seems like a really good idea even if just as a teaching tool... for example, perhaps reworking some of the above examples but with the following named lifetimes would be beneficial:

'instance: what we used for 'a, meaning the lifetime of a newly created Foo
'all: what we used for 'b, meaning every possible lifetime (e.g. forall 'b)
'instance2: what we used for 'c', meaning the lifetime of a newly created HashMap return value

Maybe not those exact names / descriptions, but I think this is a good approach to teaching lifetimes, at least sometimes.

I wonder why that is, actually, since with regular generics it's usually easier to keep them completely generic... i.e. it's more useful to think of map as like (A -> B) -> F A -> F B than something like (TYPE1 -> TYPE2) ... because generic functions really don't want to know anything about those types (putting aside traits for the moment) ... but lifetimes have inherent meaning... like they're tied to something else outside the function..?

Not sure if I'm explaining my thoughts there well, but whatever - might inspire some useful feedback :slight_smile:

Hmmmm.... I'm trying to optimize early (I know..) and avoid a HashMap lookup where it's not needed, but most likely - it's probably an unnecessary optimization overall. (e.g. I may not really need to do multiple lookups as much as I think). Reminds me a little of what @vorner wrote here:

"The difference between good and bad design is visible in the hairiness of API"

More food for thought!

I'm not quite following where 'c is coming from here... is it automatically gotten from Q ?

And if so - the reason we're saying 'c == 'b is because the locally scoped value we gave it, &42, is 'b?

Not following why this is... doesn't the mut borrow finish when calc() ends?

EDIT: okay... I think I see... by annotating a lifetime I'm explicitly telling it that it does not finish borrowing when calc() ends... but rather when the lifetime ends... that's funky!

2 Likes