The primary feature enabled by HKTs is to create streaming iterators (and streaming streams in async world), i.e. an iterator that produces values that may contain a reference into the iterator itself.
With the current Iterator
trait, the following code must be valid no matter what the item type is:
let a = iter.next();
let b = iter.next();
println!("{:?}", a);
However imagine what would happen if a
held a reference into iter
. Then, since .next()
mutates the iterator, the second call to next
might invalidate the reference stored in a
.
One feature you might imagine using this for would be an alternative to std::io::Lines
. The Lines
iterator currently allocates a new String
every time you call .next()
, but there's no reason you have to do that (and in fact you can avoid it with read_line
instead). Such an alternate Lines
iterator would have item type &str
, with a String
buffer stored inside the iterator, whose allocation is reused on every call to .next()
.
To see how this ties into HKTs, the technicality you need to understand is that &str
is not a type because it's missing the lifetime. If you want a specific type, you need to say &'a str
for some specific lifetime 'a
, and given two different lifetimes 'a
and 'b
, there are two distinct types &'a str
and &'b str
. If we now take the example
let a = iter.next();
println!("{:?}", a);
let b = iter.next();
println!("{:?}", b);
then a
and b
have different lifetimes, and thus, despite both being a &str
, have different types. With an ordinary iterator, every call to .next()
must return the same type, but with HKTs for the item type, you are allowed to vary the generic lifetime parameter from call to call.