Pass Vec<V> as param to a function, but compiler throws borrowed value does not live long enough error

1

I searched this forum, and found threads that talk about this issue. However, they are either String vs &str or borrowed a value inside a struct like this. That does not look like my case. My problem is I pass in a Vec as a parameter to a function. And it seems that because of calling iter() so the lifetime of borrowed V may be invalid after this function execution/ when the next iter() operation (I could be wrong because I may totally misunderstand it).

Below is the code I use. How to fix it? Thanks.

#[derive(Clone, Debug)]
struct V {
    data: f64,
}

impl V {
    fn subtract(self, other: V) -> V {
        V {
            data: self.data - other.data,
        }
    }
}

fn f(data: Vec<V>) -> impl Iterator<Item = f64> {
    let max_value_opt: Option<&V> = data.iter().max_by(|x, y| x.data.total_cmp(&y.data));
    let exps = data
        .iter()
        .map(|v| {
            max_value_opt.iter().map(|max_value: &&V| {
                let cloned: V = (**max_value).clone();
                v.clone().subtract(cloned)
            })
        })
        .flatten();
    let total: f64 = exps.clone().into_iter().map(|v| v.data).sum::<f64>();
    exps.clone().into_iter().map(|value| value.data / total)
}
2

yeah the problem is that the output iterator should own everything it needs to work while yours currently relies on borrowing several things that are temporary for the function.
i suggest you rework your function to read the vector a first time to get your max and divisor as owned values and then do a final pass where you use them to do data.into_iter().map(|item|{item.subtract(max).data /divisor})

you don't even have to worry about the value of max and divisor in case of the empty Vec and use placeholders since there will be no elements for the closure to be called on

1 Like
3

first of all, your function f() takes the argument data: Vec<V> by value, or in other words, data is moved into the function f().

the consequence of this is data will be dropped when the function returns, which means it's impossible to return anything that needs to borrow data, including some opaque impl Iterator type.

so, the first change you need to make is to change the parameter type to a reference. although &Vec<V> would work, it is more flexible to use &[V]:

-fn f(data: Vec<V>) -> impl Iterator<Item = f64> {
+fn f(data: &[V]) -> impl Iterator<Item = f64> {

then, there's the closure capture problem. in short, if your closure captures a local variable (max_value_opt and total, in this case) by reference, the (opaque) return type cannot contain this closure. at minimal, the closure must capture them by value:

    let exps = data
        .iter()
-       .map(|v| {
+       .map(move |v| {
 //...
    let total: f64 = exps.clone().into_iter().map(|v| v.data).sum::<f64>();
-   exps.clone().into_iter().map(|value| value.data / total)
+   exps.clone().into_iter().map(move |value| value.data / total)

finally, the nested closure has a similar problem, but you cannot move the outer capture into the inner scope, so you need to make a clone:

 //...
-           max_value_opt.iter().map(|max_value: &&V| {
-               let cloned: V = (**max_value).clone();
+           max_value_opt.clone().into_iter().map(|max_value: &V| {
+               let cloned: V = (*max_value).clone();
                v.clone().subtract(cloned)
            })
 //...

this is the minimal change that needed to make the code compile.

however the same algorithm can be rewritten in a much simpler way, your code is way to convoluted.

for example, you can change the definition of type V, either derive Copy, or use references in the subtract() method, which can reduce a ton of .clone() boilerplates, something like this:

impl V {
	fn subtract(&self, other: &V) -> V {
		V {
			data: self.data - other.data,
		}
	}
}

fn f(data: &[V]) -> impl Iterator<Item = f64> {
	data
		.iter()
		.max_by(|x, y| x.data.total_cmp(&y.data))
		.into_iter()
		.flat_map(|max| {
			let exps = data.iter().map(|v| v.subtract(max));
			let total: f64 = exps.clone().map(|v| v.data).sum();
			exps.map(move |v| v.data / total)
		})
}
4 Likes
4

I learned a new lesson, where

  1. iter() vs into_iter() that the former only borrow the values while the latter transfer the ownership.
  2. Vec and &[T]. I falsely thought Vec is something like String where T comes from outside so it should live long enough, but in fact the argument data: Vec is taken by value, once out of scope, it is dropped.

I refactored a new version, which is over complicated, though it get compiled. Apparently the solution with max_by(), then transferring ownership by into_iter(), as well as the rest, is more concise and cleaner. Many thanks!

    let max_value_opt = data
        .iter()
        .max_by(|x, y| x.data.total_cmp(&y.data))
        .map(|v: &V| (*v).clone());
    let exps = data
        .into_iter()
        .zip(max_value_opt)
        .map(move |(v, max_value): (V, V)| v.clone().subtract(max_value));
    let total: f64 = exps
        .clone()
        .into_iter()
        .map(|v: V| v.data)
        .sum::<f64>();
    exps.clone()
        .zip(std::iter::repeat(total))
        .map(|(v, total): (V, f64)| v.data / total)
5

Vec<T> is analogous to String and &[T] is analogous to &str.

3 Likes
6

Now, just to be correct .into_iter() transfers ownership on the value it is called on, which can be a reference as well, so the following would work:

let my_vec = vec![1,2,3,4];

let my_borrowing_iter_1 = my_vec.iter();
let my_borrowing_iter_2 = (&my_vec).into_iter();

In fact for Vec<T> these are identical, if you take a look at the std library implementation of Intoiterator for references to a Vec (&Vec<T,A>) it just forwards the call to .iter().

This fact is useful to know when trying to implement generic functions which want to turn something into a borrowing iterator, it is convention that all types that provide a .iter() to also provide a implementation of IntoIterator for &Self, so one can write the following trait bound:

fn i_need_something_i_can_turn_to_iter<'a, T>(my_val: &'a T)
where
    &'a T: IntoIterator
{
    //            \/ - my_val is a reference here.
    let my_iter = my_val.into_iter();
    // do something
}

which one can't directly do for the .iter() method since that is not provided by a trait but instead implemented on the type directly.

Similarly to how there is a correspondence between foo.iter() and (&foo).into_iter() there is also a correspondence between foo.iter_mut() and (&mut foo).into_iter().

You may have seen this before when using a vec in a for loop, clippy might suggest you to write

for value in &collection {}
for value in &mut collection {}

over

for value in collection.iter() {}
for value in collection.iter_mut() {}

(if you have the corresponding lints enabled)

The for loop is just syntactic sugar for calling .into_iter()on whatever you pass after the in keyword and running .next() on that iterator for each iteration of the loop.

Per the reference the following for loop

'label: for PATTERN in iter_expr {}

desugars to

{
    let result = match IntoIterator::into_iter(iter_expr) {
        mut iter => 'label: loop {
            let mut next;
            match Iterator::next(&mut iter) {
                Option::Some(val) => next = val,
                Option::None => break,
            };
            let PATTERN = next;
            let () = { /* loop body */ };
        },
    };
    result
}.

Hope that provided some additional context :slight_smile:

2 Likes