Cloning iterator costs

help
1

I'm writing a small set of technical-analysis functions and utilising the impl Iterator feature to avoid having to allocate intermediate vectors if I need to chain these.

However for the macd() function, to avoid use after move errors I have to clone the iterators. What is the cost of this given the function as a whole is lazy? Will it copy the underlying elements or is this more for clarity rather than anything else.

Any other hints to make this performant always appreciated.

fn sma<'a, I>(mut i: I, period: usize) -> impl Iterator<Item = f32> + Clone + 'a
where
    I: Iterator<Item = f32> + Clone + 'a,
{
    use std::collections::VecDeque;
    let mut v: VecDeque<f32> = i.by_ref().take(period - 1).collect();
    let initial = v.iter().sum::<f32>() / period as f32;
    v.push_front(0.0);
    i.scan(initial, move |state, it| {
        let pop = v.pop_front().unwrap();
        *state = ((*state) * period as f32 + it - pop) / period as f32;
        v.push_back(it);
        Some(*state)
    })
}

fn macd<'a, I>(mut i: I, slow: usize, fast: usize, signal: usize) -> impl Iterator<Item = (f32, f32, f32)> + 'a
where
    I: Iterator<Item = f32> + Clone + 'a,
{
    let macd = {
        let slow_sma = sma(i.clone(), slow);
        let fast_sma = sma(i, fast);
        slow_sma.zip(fast_sma).map(|(slow, fast)| slow - fast)
    };
    let signal_sma = sma(macd.clone(), signal);
    let hist = macd.clone().zip(signal_sma.clone()).map(|(macd, signal)| macd - signal);
    izip!(macd, signal_sma, hist)
}
2

Cloning an iterator is generally just copying its internal iteration state, which should be cheap. However, an arbitrary iterator can do arbitrary things in its clone impl.

3

If it's done the old way with a struct Macd and impl Iterator for Macd I can immediately see what a clone() would do - it would copy the fields.

But in the new Rust 1.26+ way, where we just return an interface, what inferences can I make? Is the call to sma.clone() copying the the VecDeque (might be expensive) or just the state of scan()?

4

Yeah, it's easier to see the state that an explicitly coded iterator captures.

i.scan(initial, move |state, it| {
        let pop = v.pop_front().unwrap();
        *state = ((*state) * period as f32 + it - pop) / period as f32;
        v.push_back(it);
        Some(*state)
    })

You've moved the VecDeque into the closure, which means the synthesized iterator will own the VecDeque. That's also the only possible thing here because you wouldn't be able to return the iterator to the caller since it would be hanging to a reference to a local VecDeque that's going to be destroyed. To that end, the 'a lifetime is unneeded and doesn't do anything.

So yes, in your particular example, you're going to end up cloning the VecDeque each time, I believe.

5

Yes, I had to, for the initialisation step. I played around with alternatives using skip() and take() but it looked worse. At least a constant factor of O(period) per clone() is acceptable. And it might be memcpy anyway.

Thank you