Is using LendingIterator in for .. in loops possible?

Now that GATs have landed in 1.65 - the typical example is a LendingIterator.

I was wondering if there was anyway these could be used in a typical for .. in type loop.

All my searching indicates that "for.. in" type loops only accept Iterator Traits - which was not what I was expecting coming from a C++ background where the equivalent type of for code works as long as you match the begin/end names.
eg I was expecting that as long as there was a "next() method that returned an Option" for.. in loops would work.

I know I can do a

while let Some(val) = iter.next() {
}

But prefer the syntax sugar of for..in

I suspect that this may take an edition change to make happen, to avoid breaking existing code. I would be surprised if this wasn't on the language devs radar, though.

C++ is static language but C++'s metaprogramming language is almost 100% dynamic. Typechecking happens during instantiation phase.

Rust is static language with static metaprogramming language. Typechecking happens during definition time.

That would be entirely different language. Not even remotely similar.

It may be better or worse, IDK. But that wouldn't be Rust.

2 Likes

OK I am just looking at the docs - they seem to indicate that for..in loops just desugar to this code as in the example:

let values = vec![1, 2, 3, 4, 5];
for x in values {
    println!("{x}");
}

// Desugars to:
let values = vec![1, 2, 3, 4, 5];
{
    let result = match IntoIterator::into_iter(values) {
        mut iter => loop {
            let next;
            match iter.next() {
                Some(val) => next = val,
                None => break,
            };
            let x = next;
            let () = { println!("{x}"); };
        },
    };
    result
}

My testing seems to be fine if I used the expanded code manually (obviously ugly), but the compiler complains about the Iterator interface if I use for .. in.- even though the Iterator interface is not used in the expanded code.

Perhaps the docs are old / wrong / misleading?

Edit: It seems my issue is with IntoIterator::into_iter - I was previously replacing that with my iterator. Will investigate if I can override this somehow.

Have you actually used the std::iter::IntoIterator? Or have you used your own iterator type there.

If you used std::iter::IntoIterator then I find it very strange that your code with just LendingIterator typechecked.

It's used when you use std::iter::IntoIterator.

1 Like

No - I edited above to say that - You must have been replying while I was editing. It does seem you cannot override the return type in the IntoIterator trait to be the type you want.

That's the whole point of static typing. It makes it possible to do polymorphic generics (in languages like Java or Swift) but severely limits flexibility.

C++, actually, attempted to add something like this long ago. But these were kicked out of C++0x and then C++20 got severely limited version.

They are close to Python's typehints than to proper static typing, though.

2 Likes

This desugaring is meant to be educational, but the compiler doesn't actually copy-paste code like this. It does require the actual Iterator trait specifically, not things that happen to have same syntax.

1 Like

If I implement IntoIterator and IntoLendingIterator, and for supports both, which do I get? If it's deterministically one or the other, it would have to be IntoIterator, modulo an edition maybe. But even then, sometimes you'd want one and sometimes the other. Deciding based on the loop body seems pretty fraught. Perhaps if it there was a way to choose explicitly.

Actually, if they're both accepted and which is chosen is implicit, implementing either trait (when the other is already implemented) probably becomes a breaking change.

Any normal iterator is, quite obviously, a LendingIterator. Thus the idea is to always use LendingIterator and also provide blanket implementation of LendingIterator for normal Iterator.

That's not possible because you can turn Iterator into LendingIterator but the opposite transition is not possible.

No, it wouldn't. With blanket implementation you wouldn't be allowed to implement both for any type. You either implement Iterator and then LandingIterator is implicitly implemented or you implement LandingIterator and then you can not implement Iterator.

1 Like

Derp, blanket implementation of course. Too far out of cache.

If for loop uses LendingIterator instead of Iterator, wouldn't this code be broken?

fn main() {
    let mut v = vec![1, 2, 3];
    let mut mut_refs = vec![];
    for mut_ref in &mut v {
        mut_refs.push(mut_ref);
    }
    for mut_ref in mut_refs {
        *mut_ref += 1;
    }
    println!("{:?}", v);
}

mut_refs can exist only because they're borrowed from v, according to the Iterator interface, and not from the (hypothetical) temporary LendingIterator.

2 Likes

Nope. This code compiles and runs as expected if I add blanket impls for LendingIterator and IntoLendingIterator for I: IntoIterator and I: Iterator.

fn main() {
    let mut v = vec![1, 2, 3, 4];

    let mut mut_refs: Vec<&mut i32> = vec![];

    // pretend the for-loops desugared to something like this
    let mut iter = IntoLendingIterator::into_lending_iter(&mut v);
    while let Some(mut_ref) = LendingIterator::next(&mut iter) {
        mut_refs.push(mut_ref);
    }

    for mut_ref in mut_refs {
        *mut_ref += 1;
    }
    println!("{:?}", v);
}

(playground)

Small explanation. It's precisely where GATs do their magic.

Each implementation of LendingIterator can pick any lifetime it wants (as long as it's not too short).

But particular implementation can provide larger lifetime (like Iterator and automatically derived LendingIterator do).

If you write generic code which accepts LendingIterator then tricks like what you have showed would be forbidden. But if you know the concrete type (and thus know that this particular iterator is not, actually, lending) then everything should work.

You are using something similar with most trivial for loops:

    let sum = 0;
    for i in 0..10 {
        sum += i
    }

This only works because i is not some random E from some random iterator but very concrete i32 which supports Add. If you would want to process generic type you would say that it satisfies Add explicitly.

3 Likes

I wondered about the middle ground for compatibility -- generic code that accepts IntoIterator, so we don't know the concrete type. But it still seems to be smart enough to see that this blanket impl of LendingIterator doesn't actually borrow from the iterator.

fn collect_vec<I: IntoIterator>(iterable: I) -> Vec<I::Item> {
    let mut v = vec![];
    //for item in iterable {
    let mut iter = IntoLendingIterator::into_lending_iter(iterable);
    while let Some(item) = LendingIterator::next(&mut iter) {
        v.push(item);
    }
    v
}

(playground)

1 Like

Looking at the error messages, it sounds like this is more because of a limitation in the borrow checker which infer the lifetime in Item<'_> to be 'static when it should actually be anything, rather than a design issue.

fn main() {
    let mut v = vec![1_i32, 2, 3, 4];

    increment_in_place(&mut v);

    println!("{:?}", v);
}

fn increment_in_place<'a, I>(items: I)
where
    I: for<'b> IntoLendingIterator<Item<'b> = &'a mut i32> + 'a,
{
    let mut mut_refs: Vec<&mut i32> = vec![];

    // pretend the for-loops desugared to something like this
    let mut iter = IntoLendingIterator::into_lending_iter(items);
    while let Some(mut_ref) = LendingIterator::next(&mut iter) {
        mut_refs.push(mut_ref);
    }

    for mut_ref in mut_refs {
        *mut_ref += 1;
    }
}

(playground)

   Compiling playground v0.0.1 (/playground)
error[E0597]: `v` does not live long enough
  --> src/main.rs:47:24
   |
47 |     increment_in_place(&mut v);
   |     -------------------^^^^^^-
   |     |                  |
   |     |                  borrowed value does not live long enough
   |     argument requires that `v` is borrowed for `'static`
...
50 | }
   | - `v` dropped here while still borrowed
   |
note: due to current limitations in the borrow checker, this implies a `'static` lifetime
  --> src/main.rs:54:36
   |
54 |     I: for<'b> IntoLendingIterator<Item<'b> = &'a mut i32> + 'a,
   |                                    ^^^^^^^^^^^^^^^^^^^^^^
1 Like

That limitation is actually discussed right in the blog post which discusses stabilization of GATs.

And, of course, it was also discussed in another blog post before.

You can not use LendingIterator like that, ironically enough, which is funny if you recall that LendingIterator is, more-or-less, the first example people discuss when they talk about GATs.

I would think that discussing other usecases which are actually useful in today's Rust would be more appropriate, but I wasn't the one who decided that LendingIterator should be a prime example of GATs.

My assumption is that Rust developers have an idea how to fix that, otherwise they would have postponed stabilisation of GATs, but I'm not yet sure what it is.

Introduce “limited HRTBs”, maybe?