Help to understand lifetimes

Hello
I tried to use a self-explanatory reproducer of my complex project below:

pub struct Element {
    pub flag: u32,
}

pub struct Storage<'a> {
    pub elements: Vec<Element>,
    pub elements_to_modify: Vec<&'a Element>,
}

impl<'a> Storage<'a> {
    pub fn new() -> Storage<'a> {
        Storage {
            elements: Vec::new(),
            elements_to_modify: Vec::new(),
        }
    }

    pub fn add_to_modify(&mut self, ele: &'a mut Element) {
        self.elements_to_modify.push(ele);
    }

    pub fn test(&mut self) {
        // temporary vector to iterater over self and modify self later
        let mut tmp: Vec<&'a mut Element> = Vec::new();
        for ele in self.elements.iter_mut() {
            if ele.flag == 1 {
                tmp.push(ele);
            }
        }

        for ele in tmp.iter_mut() {
            self.add_to_modify(ele);
        }
    }
}

fn main() {
    let mut storage = Storage::new();
    storage.test();
}

Rust gave me the following error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements

I am pretty new to rust language and of course I have some pain with lifetimes. I don't know what to do exactly to solve this problem, I just want to define a storage instance that lives longer than the elements and references to these elements it stores.

Any help appreciated.

Eric

I think you have answered your question yourself - you are cannot store a reference to an element that outlives the element. It is unsound since you'd eventually have a dangling pointer, and Rust simply won't allow it. In fact, even in C, using a dangling pointer is UB.
What you might be looking for is a reference-counted solution - which for Rust would be Rc. You can store the elements in an Rc, which ensures that the element is dropped only when it has no references to it stored anywhere.

1 Like

Using Rc (you can also use Arc if you work with threads), I could fix the code as follows:

+use std::rc::Rc;
+
 pub struct Element {
     pub flag: u32,
 }
 
-pub struct Storage<'a> {
-    pub elements: Vec<Element>,
-    pub elements_to_modify: Vec<&'a Element>,
+pub struct Storage {
+    pub elements: Vec<Rc<Element>>,
+    pub elements_to_modify: Vec<Rc<Element>>,
 }
 
-impl<'a> Storage<'a> {
-    pub fn new() -> Storage<'a> {
+impl Storage {
+    pub fn new() -> Storage {
         Storage {
             elements: Vec::new(),
             elements_to_modify: Vec::new(),
         }
     }
 
-    pub fn add_to_modify(&mut self, ele: &'a mut Element) {
+    pub fn add_to_modify(&mut self, ele: Rc<Element>) {
         self.elements_to_modify.push(ele);
     }
 
     pub fn test(&mut self) {
         // temporary vector to iterater over self and modify self later
-        let mut tmp: Vec<&'a mut Element> = Vec::new();
+        let mut tmp: Vec<Rc<Element>> = Vec::new();
-        for ele in self.elements.iter_mut() {
+        for ele in self.elements.iter() {
             if ele.flag == 1 {
-                tmp.push(ele);
+                tmp.push(ele.clone());
             }
         }
 
-        for ele in tmp.iter_mut() {
+        for ele in tmp {
             self.add_to_modify(ele);
         }
     }

(Playground)

Note that I also replaced one instance of .iter_mut() with .iter(), and removed the second instance of .iter_mut().

3 Likes

It is impossible for a struct to store references into another field of the same struct. Consider using indexes into the vector instead.

3 Likes

Thanks @jbe, I changed my code to use Rc, it works well, I have the feeling to not use the power of Rust by doing that (to not have an optimized code) but it's not a big deal for my first project.

Should I understand that:

let tmp: Vec<Rc<Element>> = Vec::new();
for ele in tmp.iter() {
}

and (without iter()):

let tmp: Vec<Rc<Element>> = Vec::new();
for ele in tmp {
}

are the same and can be used without any distinction?

Yes I started to do that at first, but my code became over complicated

Eric

It can be surprising that Rust (without using unsafe) has some constraints in regard to how references can work. Like @alice said, it's impossible for a struct to store references into another field of the same struct. This will require that you

  • use Rc or Arc,
  • use clones of the original data (which is sometimes worse than using Rc/Arc),
  • store indices (i.e. use your own form of "reference"),
  • use unsafe (which I would really not recommend in 99% of all cases).

Sometimes you have to do things that feel unnecessary in Rust. But they are not really unnecessary: They help to ensure that your code doesn't have certain kinds of bugs such as memory access errors. You can still have "logic errors", like storing an index which doesn't exist anymore, but using such an index results in a "clean" panic rather than some undefined behavior.

So whichever way you go, Rc or storing indices, I wouldn't worry about not using the power of Rust. You're actually doing it because Rust gives you certain safety guarantees. Some of these guarantees play out at compile-time (when you can use lifetimes), and some impose a tiny bit of runtime overhead (like Rc/Arc).

No, they are not the same! for ele in tmp.iter() will give you references while for ele in tmp will move the elements out of tmp, i.e. you cannot use tmp afterwards. It will be gone (which is okay, as you don't need it afterwards anymore).

#[derive(Debug)]
struct Person {
    name: String,
    salary: i32,
}

impl Person {
    fn increase_paycheck(&mut self) {
        self.salary *= 100;
    }
    fn show(&self) {
        println!("{self:?}");
    }
    fn consume(self) -> String {
        self.name
    }
}

fn main() {
    let mut vec = vec![
        Person {name: "Alice".to_string(), salary: 13},
        Person {name: "Bob".to_string(), salary: 12}
    ];
    
    // iterate over mutable references:
    for p in vec.iter_mut() {
        p.increase_paycheck();
    }
    // alternative syntax:
    for p in &mut vec {
        p.increase_paycheck();
    }
    
    // iterate over shared references:
    for p in vec.iter() {
        p.show();
    }
    // alternative syntax
    for p in &vec {
        p.show();
    }
    
    let mut names: Vec<String> = Vec::with_capacity(vec.len());
    
    // move out of vec:
    for p in vec {
        names.push(p.consume());
    }
    println!("Extracted names: {names:?}");
    
    //println!("We cannot print our original Vec: {vec:?}");
}

(Playground)

Output:

Person { name: "Alice", salary: 130000 }
Person { name: "Bob", salary: 120000 }
Person { name: "Alice", salary: 130000 }
Person { name: "Bob", salary: 120000 }
Extracted names: ["Alice", "Bob"]

In the example above, for p in vec will move out of vec, so you own the Person struct inside the for loop's body. That allows you to call the consume method, which requires ownership (and thus can only be called once for each Person).

In this example, for p in vec allows you to extract the names of the Persons without cloning anything. This only works if we "give up" the original list of persons.

A fourth alternative is to use a draining iterator:

-    for p in vec {
+    for p in vec.drain(..) {
         names.push(p.consume());
     }
     println!("Extracted names: {names:?}");
     
-    //println!("We cannot print our original Vec: {vec:?}");
+    println!("vec is empty now: {vec:?}");
}

(Playground)

Yet in many cases, it doesn't matter if you use for ele in tmp or for ele in tmp.iter(). That is in those cases where you are happy with either an owned value or a shared reference to it, and where you don't need the Vec afterwards. In my modification of your code, however, we need the owned value. What you could do is:

-        for ele in tmp {
+        for ele in tmp.iter() {
-            self.add_to_modify(ele);
+            self.add_to_modify(ele.clone());
        }

(Playground)

But as you see, this requires an extra clone, which is certainly unnecessary overhead now. Using .iter() without .clone() wouldn't work: Try it out on Playground here!

Actually I like that idea too. Maybe I should consider that more often in my own code. But I can also see how this might make the code appear complicated or difficult to understand. I feel like it kinda undermines the type system in a way. Hmmmm… :thinking: difficicult choice! I still think it might be better than using Rc/Arc in several scenarios. (I just realized I also used that pattern in my own code before.)

But using Rc and Arc isn't unusual either, and shouldn't be considered bad practice "per se" (in my opinion). In past, I hesitated more to use it.

P.S.: Anyway, depending on what else you do in your (full) code, storing indices and working with these might be better in your scenario. I guess I would assess the pros and cons regarding code readability and expected performance impact. (Or just use what feels better if it's difficult to judge about.)

3 Likes

I would like to add that in some cases, they behave the same, which is when tmp would be a reference. Condider this example:

fn show(vec: &Vec<i32>) {
    for n in vec { // same as vec.iter() in this case (but not generally!!)
        println!("{n}")
    }
    for n in vec.iter() {
        println!("{n}")
    }
    // interestingly `for n in &vec` doesn't work now:
    //for n in &vec {
    //    println!("{n}")
    //}
}

fn main() {
    let vec = vec![3, 7];
    show(&vec);
    show(&vec);
}

(Playground)

Output:

3
7
3
7
3
7
3
7

:exploding_head:

(These for loops are really confusing sometimes!)

In simplified terms:

  • for x in vec when you don't need vec afterwards and/or need each x to be an owned value (the latter requires that vec is an owned Vec)
  • for x in vec.iter() when you do not want to modify (or consume) vec and when it's okay that each x is a shared reference (and not an owned value)
  • for x in vec.iter_mut() when you need x to be a mutable reference that allows you to modify the elements inside the vec

I would add that when you just want to iterate over a collection by reference, for x in &vec and for x in &mut vec also work, using IntoIterator for the respective reference types. All standard collections provide such implementations. .iter() and .iter_mut() are convenience methods, mostly provided for chaining (for x in vec.iter().cloned().enumerate(), etc.)

1 Like

I mentioned that as the alternative syntax above:

But you cannot always write &vec instead of vec.iter() as shown by the other example:

fn show(vec: &Vec<i32>) {
    // interestingly `for n in &vec` doesn't work now:
    for n in &vec {
        println!("{n}")
    }
}

fn main() {
    let vec = vec![3, 7];
    show(&vec);
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0277]: `&&Vec<i32>` is not an iterator
 --> src/main.rs:3:14
  |
3 |     for n in &vec {
  |              ^^^^ `&&Vec<i32>` is not an iterator
  |
  = help: the trait `Iterator` is not implemented for `&&Vec<i32>`
  = note: required because of the requirements on the impl of `IntoIterator` for `&&Vec<i32>`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `playground` due to previous error

So I currently prefer to explicitly write .iter() where I want to iterate over shared references, but I'm not always consistent in my own code either. There's just so many options on how to iterate :sweat_smile:.

P.S.: @erict Sorry if those extended examples became a bit complex/confusing. Feel free to ask if anything is unclear.

P.P.S.: If you are always aware of whether your vec is an owned Vec or a reference (which you ideally should be), then you might to work with & and &mut instead of .iter() and .iter_mut(). Insofar, .iter() and .iter_mut() aren't strictly needed except for convenience for chaining (as @trentj said). But I still find them handy to quickly show a reader of the source code what happens, as otherwise you need to be very careful in checking whether you have a Vec or a reference to a Vec, etc.

Another option, that can be useful sometimes, is to use a crate creating self-referencing types with macros that can write the unsafe code for you in a safe way... In particular e.g. the ouroboros crate.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.