A question about lifetime


pub struct IterMut<'a, T: 'a>(&'a mut[T]);

impl<'a, T> Iterator for IterMut<'a, T> {
    type Item = &'a mut T;

    fn next(&mut self) -> Option<Self::Item> {
        let slice = mem::replace(&mut self.0, &mut []);
        if slice.is_empty() { return None; }

        let (l, r) = slice.split_at_mut(1);
        self.0 = r;
        l.get_mut(0)
    }
}

In the next method, I can see the return reference must live at least as long as input reference of self. The lifetime is 'a.
Then by get_mut's signature, I can imply the &mut l reference's lifetime is also 'a. But l will be dropped at the end of the method, so why does the compiler not show an error?

2 Likes

l is &'a mut [T], i.e. it itself is a borrow, not an owned value. We can say that it is moved into get_mut and thus not dropped.

2 Likes

Incidentally, this can be (slightly) more concise if split_first_mut is used:

    fn next(&mut self) -> Option<Self::Item> {
        let slice = std::mem::replace(&mut self.0, &mut []);
        let (l, r) = slice.split_first_mut()?;
        self.0 = r;
        Some(l)
    }
pub fn get_mut<I>(&mut self, index: I) -> Option<&mut I::Output>
where
    I: SliceIndex<Self>,

Here self's type is &mut &mut [T], right?
In my opinion, move can only occur when self isn't a reference...

1 Like

Could you explain it in detail?

It's:

impl<T> [T] {
    fn get_mut<I>(&mut self, index I) -> /* ... */
}

so self is &mut [T].

Sure:

    fn next(&mut self) -> Option<Self::Item> {
        // If we went through `self`, the lifetime would be limited by
        // the anonymous lifetime of this call.  So instead, we temporarily
        // replace `self.0` to "expose" the inner lifetime.
        let slice = std::mem::replace(&mut self.0, &mut []);
        // If the slice is empty, `split_first_mut` will return `None`,
        // and the `?` will return `None` from this function.
        //
        // Otherwise, `l` will be a `&mut T` of the (previously) first
        // element, and `r` will be a `&mut [T]` to the rest of the slice.
        //
        // So this is like the `is_empty` and `get_mut` calls combined.
        let (l, r) = slice.split_first_mut()?;
        // We can put the rest of the slice back into `self` now.
        self.0 = r;
        // And finally, return the reference to the first element.
        Some(l)
    }

This might be made a little clearer, if we use not l.get_mut(0), but <[T]>::get_mut(l, 0). Here we can see that get_mut consumes l, not borrows.

fn next(&mut self) -> Option<Self::Item> {
    let slice = mem::replace(&mut self.0, &mut []);
    if slice.is_empty() { return None; }

    let (l, r) = slice.split_at_mut(1);
    self.0 = r;
    let result = l.get_mut(0);
    println!("{:?}", l); // error, can't be borrowed
    result
}

The term consumed you said means: l's lifetime is expanded by result, so it can't be aliased no longer until the last use of result.

Right?

Is consume or move a formal explain for references? Or just for explaination?
Because compiler doestn't say like that...
In my opinion, only value can be moved or consumed...

I always thought that &mut self will always wrap identifier before the dot in a &mut...
Actually, compiler will see whether the identifier's type already meet the requirement, right?

And your explain is very well. Thanks a lot.

But I'm still confused by some detail...

// If we went through `self`, the lifetime would be limited by
// the anonymous lifetime of this call.  

Is this means below?

fn next(&mut self) -> Option<Self::Item> {
    // not self.0 because comfiler say
    // "cannot move out of `self.0` which is behind a mutable reference"
    // but even wrote &self.0, compiler say
    // "cannot infer an appropriate lifetime"
    let slice = &self.0;
    let (l, r) = slice.split_first_mut()?;
    self.0 = r;
    Some(l)
}

So the reason that &self.0 doesn't work is: slice will be dropped at the end?

I don't understand why mem::replace can work?
From the signature:

pub const fn replace<T>(dest: &mut T, src: T) -> T

how can compiler determine the lifetime of slice?

1 Like

A reference is a value, too. It's just that this value is borrowing from another value, but the reference can be copied (if it's shared) or moved (if it's exclusive) like the value of any non-borrowing type.

1 Like

I don't understand this question, sorry.

First, let's rewrite the signature so it's a little easier to talk about. I've just replaced the Ts:

pub const fn replace<MyType>(dest: &mut MyType, src: MyType) -> MyType

So, we're doing

        let slice = std::mem::replace(&mut self.0, &mut []);

And

  • self.0 is a &'a mut [T]
  • So in the replace call, MyType is &'a mut [T]
    • This is where the compiler got the lifetime from
  • And &mut self.0 is a &mut &'a mut [T]
  • And thus we are calling
fn replace<&'a mut [T]>(
   dest: &mut &'a mut [T],
   src: &'a mut [T]
) -> &'a mut [T]

As you can see, we get a &'a mut [T] back out.

Here's a playground with a bunch more comments about why replace works but not using replace doesn't work.

3 Likes

Yes, that's true. It can automatically add a & or a &mut and it can also automatically dereference, but it will do neither if the receiver type already matches, e. g. if it's already a mutable reference and the method expects &mut self (of the right type). More details can be found in the reference.

2 Likes

I have another question: In your playground, you resolve the lifetime of slice by src param of replace, but in your post you get the lifetime of slice by dest. I still can't understand why the replace can extend the lifetime...I just discovered that Option::take do the similar thing...

so lifetime of self is 'borrow but lifetime of self.0 is 'a?

It’s easier to talk about types than values, and generally you can only go so far with an intuition where you’re talking about “the lifetime of {SOME VALUE}”.

For properly discussing these values, the approach is two-fold; first determine the types of the values you’re talking about, then look at the lifetime parameters in those types.

self comes from (&'borrow mut self) argument, which is a short-hand for self: &'borrow mut Self. The type Self in an impl<'a, T> Iterator for IterMut<'a, T> is a short-hand for the self-type IterMut<'a, T>, hence self: &'borrow mut IterMut<'a, T>. You see that the type of self contains two lifetime arguments, 'borrow and 'a, there’s not really a single lifetime that anyone could call the lifetime of self; however often one would still call the first/outermost lifetime 'borrow “the lifetime of self”.

Similarly, self.0 is the field of the value behind the reference self, so it’s the type of the single/first field of IterMut<'a, T>, which is &'a mut [T]. Here’s only a single lifetime, so in some ways it makes sense to say “the lifetime of self.0 is 'a”.

So I’d mostly agree with your statement cited above, even though one could be more precise




It doesn’t extend any lifetime. Maybe a good analogy is something like: If you have a reference foo: &'a mut i32 and then do let bar: i32 = *foo, the value bar can exist longer than the lifetime 'a, but all that happened here is that the value behind the reference foo was copied. Since i32 is not a type involving any lifetimes at all, you wouldn’t consider any lifetime to be extended here.

A reference &'a mut T does not mean that the value behind the reference is only alive for 'a. Instead it means that the reference itself is only alive for that long, which could be due to the memory behind that reference being short-lived; e.g. if some API passes you &'a mut T, it might as well deallocate the memory behind that &'a mut T after the lifetime 'a ends (as well as dropping whatever happens to be in that memory at the time). What’s explicitly not prohibited is that the value that’s in that memory gets replaced by any other value, so that the original value of type T could be kept for much longer than 'a.

Double mutable references, i.e. values of type &'short mut &'long mut S are potentially confusing in this regard, because the most natural way to access them – i.e. dereferencing – only gives you a &'short mut S. But if you use mem::replace, you can obtain a &'long mut S: That’s a case of dereferencing shortening the lifetime of the contained &'long mut S, not a case of mem::replace extending a lifetime. In fact, mem::replace doesn’t care at all whether the type behind a reference contains a lifetime at all. It works in general with dest: &'short mut T and src: T and returns a T. The fact that T == &'long mut S introduces a second lifetime to the story adds complexity, but nothing really changes. Replace equal types with equal types and you have that dest: &'short mut &'long mut S and src: &'long mut S as arguments to mem::replace makes it return a &'long mut S.

The type &'borrow mut IterMut<'a, T> in the example, too, is (almost) an instance of such a double mutable reference type: IterMut<'a, T> is a newtype-wrapper struct around &'a mut [T], so that &'borrow mut IterMut<'a, T> is in many ways the same as &'borrow mut &'a mut [T]. It, too, has that property that interacting “naively” with self.0 (i.e. not using something like mem::replace) can often result in either compilation failure (if you try to move that value) or dereferencing/re-borrowing might “help” you out but only give you access to a &'borrow mut [T].

6 Likes

Perhaps think of it this way: If you have a &'short mut container, you only have a 'short amount of time to modify things in the container. After that a bladed door will slam shut. Any time you modify something in the container via the &'short mut reference, you're reaching through that door.

However, if you reach in and move some piece out of the container, you don't have to worry about the door slamming shut any more. You don't have to reach through the door any more to get to the piece. But one caveat to keep in mind is that you can't just leave an empty space when your remove a piece from the container, because you're only borrowing it. The compiler won't let you do this in safe code for even the shortest amount of time.

replace and take let you get pieces out of the container by replacing them with something else immediately, so there is no empty space. And again, once the pieces are outside of the container, you don't have to worry about the door.

And all this still applies if you have a &'short mut &'long mut piece. Just because the inner door is going to stay open awhile longer doesn't mean you don't have to worry about the outer door... unless you remove the &'long mut piece from the outer container first. If you try to modify the piece without removing it first, it will behave like a &'short mut piece, because you're still reaching through that 'short outer door.

When you remove the piece, no lifetimes are extended. You're just getting access to the inner piece without having to worry about the outer lifetime for awhile, by removing the piece from behind the outer door entirely. There's no safe way to do this yourself, like the article I linked to talks about. But std::mem::replace presents a safe API to perform this unsafe operation by ensuring at a low-level that nothing can go wrong (e.g. when a destructor runs, or anything else that might happen between reading the value out and writing a replacement in its place). And take is just a special case that replaces with None.


5 Likes

Thanks, I love you guys! @quinedot @steffahn

Therw is another thing I want to know: you say it's the dereference that shorten the lifetime. Where is the dereference when i do self.0.split_first_mut()?

self.0 is &'a mut [T], but in the call, self becomes &'borrow mut [T]?

It's here:

let (l, r) = self.0.split_first_mut()?;
//               ^

Because the field access is really (*self).0.

2 Likes

Got it.

While self.0 is (*self).0 whose type is &'borrow mut [T] , so in

let slice = std::mem::replace(&mut self.0, &mut []);

, MyType is &'borrow mut [T], so the returned slice is also &'borrow mut [T]...

Oh no……

Then how does the compiler know slice is &'a mut [T]?

1 Like