Fully owned Iterator causing lifetime problems

Hello, I have a very baffling (to me) situation that I'm hoping somebody can shed some light on. Here is the Rust Playground that demonstrates the problem.

Here is the test code there:

use itertools::iproduct;

type Point = (isize, isize);

trait PointExt {
    fn neighbor_points(&self) -> impl Iterator<Item = Point>;
}

impl PointExt for Point {
    fn neighbor_points(&self) -> impl Iterator<Item = Point> {
        let point = self.clone();

        iproduct!(-1isize..=1, -1isize..=1).filter_map(move |(dy, dx)| {
            let point = (point.0 + dx, point.1 + dy);

            (dx != 0 || dy != 0).then_some(point)
        })
    }
}

fn weird() -> impl Iterator<Item = Point> {
    let point = (0, 0);

    point.neighbor_points()
}

fn main() {
    for p in weird() {
        println!("{p:?}");
    }
}

Here is the unexpected compiler error that this generates:

error[E0597]: `point` does not live long enough
  --> src/main.rs:24:5
   |
22 |     let point = (0, 0);
   |         ----- binding `point` declared here
23 |
24 |     point.neighbor_points()
   |     ^^^^^------------------
   |     |
   |     borrowed value does not live long enough
   |     argument requires that `point` is borrowed for `'static`
25 | }
   | - `point` dropped here while still borrowed

For more information about this error, try `rustc --explain E0597`.

I understand the error but I am very confused as to why it applies. The compiler seems to think that the Iterator returned from neighbor_points is borrowing from the Point object (via the &self argument) but, looking at the code, the returned Iterator should be fully self-owned since the only use of self is to clone it into the first point variable, which is then moved into the closure passed to filter_map. There are no other references to self in the entire function and, specifically, not in the closure.

If there is a genuine borrow going on here, I have no idea where it could be happening. Otherwise the compiler is just unable to determine that the Iterator does not borrow from the Point (maybe based solely on the function signature?), but I do not know how to express this fact to the compiler, having tried a few different lifetime specifications (e.g. returning impl Iterator<Item = Point> + 'static for all the functions), but nothing seems to work.

Interestingly, consuming the Point instead by passing self instead of the &self reference in neighbor_points does fix the problem, but I do not see why this should be necessary.

Any insight on what is going on here would be much appreciated!

The compiler doesn't look inside a function to type check calls to that function, it only looks at the function signature.

1 Like

Make the neighbor_points function take self rather than &self. The point is a small copyable type, so a reference is not needed anyway.

2 Likes

This is an unfortunate aspect of RPITIT which currently has no satisfying workaround. By RPITIT I mean using this (RPIT) in a trait (…IT):

fn neighbor_points(&self) -> impl Iterator<Item = Point>
//                        ^^^^^^^^^^^^^^^^

The unfortunate aspect is that all input parameters -- including all lifetimes -- are captured by the opaque return type. That means they effectively act as if they do contain a borrow of self in this case.

Generally speaking, it's the function signature that defines the API contract of the function, not the function body. RPIT/RPITIT pokes some holes in this, but what is captured is not one of them. So it doesn't matter that you aren't actually capturing borrows in the body.

For your particular case, if it's acceptable, you could remove this problem by taking self by value. (Large types could implement PointExt for &Large... but will probably have the same borrow problem then.)

Eventually, the preferred workaround would probably be to have an associated type instead.[1] But this can only work for your example on stable without changes if you return Box<impl Iterator<Item = Point>> or otherwise type-erase the unnameable closure that is part of your iterator.

Another approach would be to use the associated type and define a custom type that "manually" performs the iteration, replacing the map closure, and thus making a nameable iterator type available.


Interestingly, RPIT outside of traits does not capture generic lifetime parameters (but still captures all the rest). But the plan is to make it capture lifetimes too, in the next edition. This is the tracking issue for that.


  1. Or a GAT, when there are other generic parameters you do want to capture. ↩︎

5 Likes

Thank you very much for the insightful response!

It is an interesting and useful fact that only function signatures are used for the API contract. Given how good Rust's type inference can be, I guess I assumed that the body of a function was used to determine elided lifetimes.

It's also interesting that using an associated type fixes the issue. This is the solution I went with because I use nightly anyway since my project is not that serious. It sounds like the idea is that the associated type no longer requires the use of a RPITIT, so that the Iterator is no longer assumed to capture the &self reference.

Rust allows eliding lifetimes in some cases, but the presence or absence of lifetimes[1] in the return type is never inferred; it's part of the API contract. (The same is true of type parameters too.)

For example this

struct Ref<'a>(&'a str);
fn example(r: &String) -> Ref {
    Ref(&**r)
}

Is the same as these

fn example(r: &String) -> Ref<'_> { /* ... */ }

fn example<'r>(r: &'r String) -> Ref<'r> { /* ... */ }

So if you know the unelided form, you know what's captured.

RPIT throws a bit of a wrench into this because (a) there's no inline unelided form (b) the unelided form isn't stable (c) there's a lot of nuances, especially around lifetimes.[2]

But let me introduce the unelided form anyway. It takes the form of a type-alias impl Trait, or TAIT:

fn example<T: Display>(t: &T) -> impl Display { /* ... */ }

// pre-edition 2024 RPIT outside of traits
type ExampleOut<T> = impl Display;
fn example<T: Display>(t: &T) -> ExampleOut<T> { /* ... */ }

// edition 2024 or RPITIT
type ExampleOut<'a, T: 'a> = impl Display;
fn example<T: Display>(t: &T) -> ExampleOut<'a, T> { /* ... */ }

(The notional TAITs of RPITs are unnameable.)

Without going too far into the nuances, you can see that it's still the case that "lifetimes captured or not" is part of the API. It's just unfortunately invisible, and also non-obvious if you don't know these unstable details.

An associated type works here because a non-generic associated type must be the same type for the entire implementation, which means it can't vary by lifetime in the return of the method.[3]

More generally, the unelided form of an RPITIT is a... sigh, TAITIT I guess... but really, it's a form of associated type or GAT. So in your OP it's like

impl PointExt for Point {
    type Points<'a> where Self: 'a = impl Iterator<Item = Point>;
    fn neighbor_points(&self) -> Points<'_> {

And to utilize the "unelided form" to not capture the lifetime, you just drop that parameter from the TAITIT:

impl PointExt for Point {
    type Points = impl Iterator<Item = Point>;
    fn neighbor_points(&self) -> Points {

  1. "is a lifetime captured or not" ↩︎

  2. The last part makes it hard to even make concise, completely truthful statements about capturing APIs. ↩︎

  3. Types that vary by lifetime, even if they only vary by lifetime, are still distinct types. ↩︎

1 Like

More good information, and now I get why the associate type (without the lifetime parameter) works.

I was also not not aware that you could even do TAIT prior to seeing your solution! For anyone that may read this thread later, here is some good information about this and ITs in general.