Weird lifetime issue ONLY with mutable referece

help
1

I made a custom iterator for my structs and I ran into this weird issue:

109 | impl<'a> Iterator for EdgeStorageIterMut<'a> {
    |      -- lifetime `'a` defined here
110 |     type Item = &'a mut Slot;
111 |     fn next(&mut self) -> Option<Self::Item> {
    |             - let's call the lifetime of this reference `'1`
...
125 |         self.edges.get_mut(current_index)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ method was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`

The mutable iterator and the immutable iterator have an almost identical implementation. This error only happens with the mutable iterator.

I don't understand why it thinks I am returning data with lifetime of '1'.

I am trying to return an item of lifetime 'a from a vector reference that also has a lifetime of 'a...
The item itself which in this case is a usized value, should have the same lifetime as the vector. It can't live outside of it unless moved (Which also can't happen because a mutable iter takes &mut self and should exclusively lock the entire struct for the lifetime of the borrow.)

And even if this was reportedly the problem, the lifetime issue should also happen with the immutable struct, no?

What am I missing?
Any help appreciated.

Here is a short snippet from my app that replicates this compilation error:

2

This is expected.

&mut loans must always be exclusive, and the borrow checker can't verify their exclusivity in arbitrary code where the loan depends on something runtime like your current_index.

In case of & shared loans, the inability to verify they're exclusive was never an issue, which is why the same code without &mut works.

You will need unsafe to force it through, or reuse some existing construct like split_at_mut() or another iterator.

4 Likes
3

Given a self: &'short mut EdgeStorageIterMut<'a>, you can only reborrow &'short mut Slots when reborrowing through self. In order to get a &'a mut Slot, you need to remove the borrow of length 'a from self, split the borrow up into the item you want to return and everything else, put everything else back into self.

Instead of using indexing, and &mut Vec<_>, you'll probably want to use slices (&mut [_]) to do this... or just wrap an existing iterator like slice::IterMut.

I might tackle your playground later, but in the meanwhile, here's a walkthrough of creating a mutably borrowing iterator using slices.

1 Like
4

I handled some cases that may never occur with your data structure (but not every conceivable case), and I also did zero testing. Maybe it will be useful to you.

6

Okay. so after messing with it I finally realized what the issue is.
I will try to describe in simple terms in case somebody else runs into this issue:

in the argument
&mut doesn't specify a lifetime, and the compiler creates an implicit lifetime for &mut which in this case is '1.
The lifetime '1 will only exist for the duration of self borrow.

  • This will cause the &mut self to be freed from the lock as soon as the &mut self is dropped, which is at the end of the get.
  • This will cause a problem because it would free up the struct again, allowing the
    test: &mut 'a mut Vec<i32> to be accessed through the returned Option as well as through the .test (Direct field access) (As the &mut self lock will be freed at the end of the get)

The best solution would be binding the &mut self to the returned reference.
This will cause the whole struct to be locked for as long as the reference returned by the iter is not dropped. (Although that is a can of worms on its own because of the Iterator definition. I need an iterator trait that can tie the lifetime of &mut self in the argument, to the lifetime of the Output type. So either own iterator or third party. I heard StreamingIterators are a common pattern that do this)

I wrote this here just to clear up the understanding.

EDIT:

Short example how this can be solved with a different iterator.

7

No, that would borrow the iterator forever. It would render it entirely useless. &'a mut T<'a> is never the best solution.

3 Likes
8

These days LendingIterator is the better term for that pattern ("streams" have an async association).

You don't need a lending iterator here, as demonstrated.[1] It would make implementation easier, but consumption of the API more constrained.

  1. Unless there's more to your use case than you've shared. ↩︎

2 Likes
9

Oops, my bad! I didn't mean to use the same 'a that's already in the struct. I was actually thinking of a completely different 'a that would just be a parameter for the function. Totally didn't notice I'd already used 'a in the struct definition, so they ended up having the same name by accident. Talk about a coincidence... ¯_(ツ)_/¯

Anyways thanks for pointing that out.

I edited the post to better reflect the reality and in addition made a code snippet that explains exactly what I mean.

10

Yup I agree.

In my actual implementation I solved this with an unsafe code, similar to how it is handled in the std IterMut.

My only gripe with this is that you can have multiple mutable borrows from the slice / array (Which is mostly fine as they will not overlap)

Thanks for the replies.