Box<Iterator<&u8>> does not fulfill required lifetime


#1

Hey guys,

I’m trying to figure out why I cannot put an empty iterator into a box.

Basically, I’m not sure why this doesn’t pass lifetime checker

use std::iter::empty;
fn get_iter (_: &u8) -> Box<Iterator<Item=&u8>> { Box::new (empty ()) }

however, if I use Empty struct instead of the Iterator trait, it appears to work just fine

use std::iter::{ empty, Empty };
fn get_iter (_: &u8) -> Box<Empty<&u8>> { Box::new (empty ()) }

Box::new doesn’t seem to require any specific lifetimes or whatnot


#2

Iterator is a trait, so boxing creates a trait object. You must explicitly state the lifetime of the trait object itself using this + syntax:

fn get_iter<'a>(_: &'a u8) -> Box<Iterator<Item=&'a u8> + 'a> { Box::new(empty()) }

Empty is a struct, so returning Empty requires no trait object and the lifetimes can be elided.


#3

Ah, now I see - that’s because the trait contains a ref with a lifetime, and it should outlive this reference itself.
thank you


#4

Technically speaking, I think the explicit lifetime here serves to denote that the object must not outlive 'a. The default assumed lifetime for trait objects is 'static, which outlives everything.

(come to think of it, it’s a bit surprising actually; it’s like the opposite of T:' a.)


#5

this is interesting - if the trait object has 'static lifetime, why cannot it work out just the same way as a simple structure with 'static lifetime would do (e.g. Empty struct which passes lifetime checker)


#6

Suppose you have an Iterator<&'a T> whose lifetime is 'static; and suppose that we are currently outside the lifetime 'a.

Certainly, the 'static lifetime allows this; and that’s also precisely the problem. Because ultimately the borrow checker must somehow forbid it, as you could obtain a &'a T reference by calling next().


#7

Well, in this case I would assume that borrow checker could easily see that Iterator (even if it would be 'static) shouldn’t live that long, because it has <Item = 'a>. Doesn’t it do exactly that with structs?


#8

I can never remember this, but I think trait objects are invariant over that lifetime, so it not only can’t live longer than 'a, it also can’t live shorter than it.


#9

Yes, seems like this is the case.
However, I’d like to understand why is it required by the lifetime checker, since I cannot see any downsides of having the trait object living longer than its guts (apart from that borrow checker should check the guts to restrict the trait object to actually live longer) :slight_smile:


#10

I’ll admit it does seem like it should be able to figure this out.

One could try arguing the “explicit is better” or the “local reasoning” approach, but honestly I feel this is one of those cases better left implicit; it conveys no useful information to the programmer, at least so far as I can tell.

Maybe when there are multiple traits like &(Foo+Bar) it might be useful?


#11

Perhaps lifetime elision rules can be extended to automatically propagate the input ref’s lifetime to the trait object itself, rather than use the 'static default.


#12

To be honest I’m not even sure what rules the compiler is using to restrict these lifetimes. Here’s an example of something allowed by the compiler:

fn main() {
    use_box(Box::new(|x: &u8| { println!("{}", x); }));
}

// Note: the type could also be written as `Box<FnMut(&u8)>`;
// I am just purposefully being explicit.
fn use_box(mut closure: Box<for<'a> FnMut(&'a u8)>) {
    // Invoked with two lifetimes that have no overlap
    { let x = 3; closure(&x); }
    { let x = 3; closure(&x); }
}

This compiles, so clearly the compiler does not mind that the FnMut outlives 'a. However, Iterator::Item is an associated type while FnMut’s argument is a generic, so maybe the compiler only restricts associated types?

fn main() {
    use_box(Box::new(|x| { println!("{}", x); x }));
}

fn use_box(mut closure: Box<for<'a> FnMut(&'a u8) -> &'a u8>) {
    { let x = 3; closure(&x); }
    { let x = 3; closure(&x); }
}

The compiler still accepts this, even though the closure outlives FnMut::Output at each callsite.

Hm. Maybe this is just some special convenience hack for HRTB? Let’s monomorphize the lifetime. (and in fact, it turns out the associated type has no impact here either, so let’s remove the function output as well)

fn main() {
    // Totes okay.
    let x = 3;
    let f = Box::new(|x| { println!("{}", x); });
    use_box(&x, f);

    // Error: x does not live long enough.
    let f = Box::new(|x| { println!("{}", x); });
    let x = 3;
    use_box(&x, f);
    
    // Error: x does not live long enough.
    // so apparently this has nothing to do with Trait objects! asdgafjgskfg
    let f = |x| { println!("{}", x); };
    let x = 3;
    f(&x);
    
    // But this works.
    let f = Box::new(|x: &u8| { println!("{}", x); });
    let x = 3;
    use_box(&x, f);
}

fn use_box<'a>(x: &'a u8, mut closure: Box<FnMut(&'a u8)>) {
    closure(x);
}

Notice how the error in the "f outlives x" case above ends up having nothing to do with trait objects.

…Bah.

In the end, this investigation has taught me almost nothing about how these “+ 'a” demands from the compiler come about, and all I’ve really learned is that there are some really dark, ugly and confusing corners in Rust when one looks hard enough. (that, or the compiler is totally right in all of these cases, in which case I say lifetimes have certainly earned their reputation of being difficult to reason about)


#13

Well, it appears to be a flaw in the lifetime checker:

    // `x` does not live long enough
    let f = |y| { println!("{}", y); };
    let x = 5;
    f(&x);

However, if we explicitly say to the compiler that the argument is a reference, lifetime checker knows that at the moment of x drop closure does not already keep the reference, and it compiles just fine:

    let f = |&y| { println!("{}", y); };
    let x = 5;
    f(&x);

However, I agree with you that it appears not to be related to the trait object lifetime elision in a box =)


#14

Isn’t “&y” in a closure argument a pattern? So in fact you’re destructuring the reference you pass but since it’s an i32 (ie Copy) it all works out. In the first case you’re passing a reference, and it doesn’t like it.

Edit: the Copy aspect doesn’t matter for this example - you’re just moving the x into closure. Since it’s Copy caller could continue to use it, but that’s a separate thing.


#15

haha, caught me :slight_smile:
anyway, the idea is still in place:

    let f = |y: &u8| { println!("{}", y); };
    let x = 5;
    f(&x);

#16

Sorry, what’s the idea? :slight_smile: The above works no different than if you were calling a separate function with same signature.


#17

In case of explicit typing of the closure argument borrow checker can see that x is just borrowed for the closure invocation:

let f = |y: &u8| { println!("{}", y); };
let x = 5;
f(&x);

However, in another case, even though it’s still &u8, borrow checker thinks that the value is still borrowed after the closure call:

let f = |y| { println!("{}", y); };
let x = 5u8;
f(&x);

What makes me think that it might be a flaw in the borrow checker implementation.


#18

It does seem strange. No idea why they differ - maybe the inference picks a lifetime identical to x rather than being anonymous and letting borrow checker narrow/coerce it to just the call, but just guessing here.

Unless someone chimes in with an explanation here, maybe file a github issue and see what the core devs say.


#19

Yeah, good idea, here it is: https://github.com/rust-lang/rust/issues/39943