Are iterators like transducers?

In JS if I wanted to optimize a sequence of map, filter, etc. over an array - I’d need a trick like “transducers” for performance reasons, so that it isn’t creating new temporary array in between each call.

Is this necessary in Rust, or can I assume that something like iter().map().filter().map() is already optimized?

It’s already optimized. Under the hood an array iterator is just a pointer to some memory. After the iterator gets one value, it simply moves the pointer forward to the next value.

It also stores an end pointer so it can quickly test if it has reached the end of the array. This is very efficient.

Chaining iterators is often no less efficient. The first iterator takes one value from memory, the next iterator operates on the value from the first iterator and then the next operates on the value from that iterator, etc until all iterators have run.

Then it goes round again. The first iterator takes another value from memory… etc, etc. Exactly like a for loop.

4 Likes

Rust iterators operates value one-by-one. Therefore when you do something like .map(...), it pulls in the value, transform it, and spit out the transformed value, rather than pulling in an entire array and spits out an entire array (like javascript’s Array::map). So, yep, kind of like transducer (except it can also drop value, as filter, take does, and count value, and also collect values into a Vec or something).

2 Likes

You can use https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.inspect to see more about what’s happening here. For example, try running:

fn main() {
    let v = (1..100)
        .inspect(|x| { dbg!(x); })
        .map(|x| x * 3)
        .inspect(|y| { dbg!(y); })
        .take(10)
        .collect::<Vec<_>>();
    dbg!(v);
}

https://play.rust-lang.org/?gist=93274cba3b838032bf312fbd7748d80e

6 Likes