I can't seem to resolve the following borrow checker situation:
use std::cell::RefCell;
trait Component {}
struct MyComponent {}
impl Component for MyComponent {}
struct Holder<'a> {
number: Box<dyn Iterator<Item = &'a mut dyn Component> + 'a>,
}
fn main() {
let test = create_cmps();
let mut test_borrow = test.borrow_mut();
let combined = test_borrow.iter_mut().map(|cmp| cmp.as_mut());
let _ = Holder {
number: Box::new(combined),
};
}
fn create_cmps() -> RefCell<Vec<Box<dyn Component>>> {
RefCell::new(vec![
Box::new(MyComponent {}),
Box::new(MyComponent {}),
Box::new(MyComponent {}),
])
}
My use case is converting a list of a (runtime borrow checked) trait objects to an iterator over mutable borrows (lazy iteration for performance + stripping the Box wrapper to avoid coupling consumers of the iterator with implementation details).
The error appears when I attach map(|cmp cmp.as_mut()) - this is the "stripping the Box" part.
Key parts of the error are:
test_borrow does not live long enough
test_borrow dropped here while still borrowed
returning this value (cmp.as_mut()) requires that test_borrow is borrowed for 'static
I have to note that in a simpler example I had the following Holder definition:
struct Holder<'a> {
// the "requires that ... is borrowed for 'static" issue was resolved by adding the `+ 'a` bound on the Iterator
number: Box<dyn Iterator<Item = &'a mut i32>>, // added "+ 'a" here to fix it
}
However when I extended the example to be a dyn Iterator over dyn Component items the "requires that ... is borrowed for 'static" error appeared again, but this time I don't have a + 'a workaround (and it probably wouldn't make sense since lifetime elision for dynamic trait objects should work if I have &'a mut before dyn Component).
Any help from the experts here is going to be dearly appreciated as I failed to figure this out on my own (hence my first post here )
You have two dyn types here, dyn Iterator and dyn Component. The dyn Component type also has its lifetime, which the elision rules end up setting equal to 'a:
This then ends up being over-constraining. (Generally when you have “dropped while still borrowed”, in a situation where you aren’t in fact trying to use a borrow of something that was dropped, it means you have an over-constrained lifetime that is equal to another lifetime that it shouldn’t be equal to.) The solution is to introduce a separate named lifetime for the inner trait object:
Advice for the future (it wouldn’t have helped much here): what you have quoted is missing key parts of the error. All those “this” and “here” are supposed to be pointing at specific parts of the source code. I’m guessing you copied this error text out of your IDE — never do that, because no IDE is actually good at displaying Rust errors yet. When you have a confusing error that you are trying to understand, your very first step should be to run cargo check (or cargo test or whatever build-related operation you need) and look at the error output in your terminal, and if you are going to share it, copy the text from your terminal. These complete messages will be much more helpful than the notes viewed without their intended context.
What you suggested works! Thank you very much
The rule for determining the elided reference lifetime also seems to be described here in a simplified manner - Basic guidelines - Learning Rust
I'll also make sure to copy-paste the full compiler output next time!
However I still struggle to understand how this works.
What is &'itered mut (dyn Component + 'component) exactly?
What's the issue of the two lifetimes being equal - in reality everything lives until the end of the scope, so is this a borrow checker imperfection or is there an actual problem with the lifetimes?
Regarding 1.
- Up until now I thought that lifetimes are a concept unique to references - AFAIK the dyn keyword converts the &mut reference to a fat pointer - but it's still a single reference, so that means a single lifetime - not two.
- The dyn Component part is referred by "Learning Rust" as a "type constructor with a lifetime parameter", so maybe in reality we are talking about something like &'itered mut ConcreteDynType<'component>, but even then I don't follow - it's not like there are any references within the component itself, so what does the 'component lifetime mean...
I did read something along the lines of "dyn erases the type which also erases the lifetimes", but I'm not sure how to understand that - logically it makes sense that to validate a reference you need two lifetimes - the lifetime of the reference itself and the lifetime of the data it points to... however how come we never mentioned "non-reference lifetimes" until we got to the dyn Trait part... why don't I have to specify &'a usize + 'b, but I have to specify &'a dyn Trait + 'b?
What is &'itered mut (dyn Component + ''component) exactly?
A mutable reference.
That exclusively borrows something for 'itered.
That points to items implementing the Component trait.
That are valid for at least 'component.
The 'itered lifetime says how long the components are borrowed. The 'component lifetime says how long the components themselves (under this understanding of their type) can continue to exist for.
What's the issue of the two lifetimes being equal - in reality everything lives until the end of the scope, so is this a borrow checker imperfection or is there an actual problem with the lifetimes?
Mutable/exclusive reference lifetimes don't just say that a value exists, they say that the value is exclusively borrowed for that long. So, mutable/exclusive reference lifetimes don't just need to be long enough, they must be not too long to allow you to do other things with the borrowed value than the mutation.
Whenever you write &'a mut SomeTypeWhichMentionsALifetime<'a> — where SomeTypeWhichMentionsALifetime is standing in for any kind of type that contains a lifetime parameter somewhere — you end up expressing “this thing is exclusively borrowed by this reference for the rest of its existence”, which is generally not what you want — in most cases, you want a mutable/exclusive borrow to be released so you can use the thing that is mutated again, even if only to be Dropped.
Up until now I thought that lifetimes are a concept unique to references …
Lifetimes that appear in types other than reference types are about those other data types containing references. For example, your type Holder<'a> contains an iterator that contains a reference to the data in your RefCell, which is why it must have a lifetime parameter of its own. That doesn’t happen to be relevant to Component, though.
it's not like there are any references within the component itself, so what does the 'component lifetime mean...
The type dyn Component + 'a means that if the component contains any references (it does not have to), then those references will be valid for at least 'a.
I did read something along the lines of "dyn erases the type which also erases the lifetimes", but I'm not sure how to understand that - logically it makes sense that to validate a reference you need two lifetimes - the lifetime of the reference itself and the lifetime of the data it points to... however how come we never mentioned "non-reference lifetimes" until we got to the dyn Trait part... why don't I have to specify &'a usize + 'b , but I have to specify &'a dyn Trait + 'b ?
There is no “the lifetime of the data” (and + 'b is part of the dyn syntax only). But some types (not including usize) have lifetime parameters, and those lifetime parameters are what the compiler validates. For example, if 'long is a lifetime longer than 'short, you cannot have a &'long &'short i32, you cannot have a &'long std::slice::Iter<'short, i32>, and you cannot have a &'long dyn Component + 'short.
The lifetime in a dyn type is a summary of the lifetimes in the concrete type it is hiding.
When you coerce a reference such as &'a mut MyComponent into a trait object reference &'a dyn MyComponent + 'b, the compiler sees that MyComponent has no lifetime parameters and so there is nothing that it has to check against 'b.
In fact, if you never want to work with components that contain references, a simpler definition of Holder will do:
This says that if the component type contains any lifetimes (contains any references) then they must be &'static references. This is very common for situations where borrowing is impractical anyway — so common that Box<dyn Component> is shorthand for Box<dyn Component + 'static>.
Rust lifetimes -- those '_ things -- are almost always about the duration of borrows, and not about the liveness scope of values. Despite the unfortunate overlap in terminology, and despite the approach of most learning materials about the borrow checker.
A variable can't be borrowed when it goes out of scope, so there is a relation between the two, but lifetimes aren't equal to liveness scopes; liveness scopes aren't assigned lifetimes. Instead, going out of scope is consider a use of a variable which conflicts with being borrowed -- as is being move, or having a &mut _ taken.
Rust lifetimes are also part of the type system, which comes up when we start talking about things like variance. And it comes up with dyn Trait + '_. In that case, some original type got erased. That erased type might or might not have had lifetimes in it (which might or might not have been references). The Rust type system uses lifetimes to make sure the erased type cannot be used when it's invalid -- for example, that some type erased reference can't be used after it dangles.
The origin of a lifetime is almost always a reference, and the lifetime is approximately the duration of the borrow of whatever you're creating the reference to. But you can parameterize nominal types with lifetimes when they have a field that contains a reference -- or a non-reference with a lifetime parameter. Like your Holder<..>.
They can also just be part of some type system mechanisms without any reference existing, but let's ignore that for this topic.
You can only coerce from SomeConcreteType to dyn Trait + 'a if SomeConcreteType: 'a holds. That means that for any lifetime 'x which appears in SomeConcreteType, 'x: 'a must also hold.
So, let's say I had a tuple = (&'a str, &'b str) and I coerced &mut tuple to a &mut dyn Trait + 'c. The requirements are 'a: 'c and 'b: 'c -- i.e. 'c is some intersection of 'a and 'b. In practical terms: I can only use the type-erased tuple while both of its borrows are still valid. Which is exactly what we want!
Ultimately, the dyn Trait + '_ lifetime is the type-system mechanism to make sure the type that got erased can only be used while it is still valid, even though it got erased.
When you have a dyn Component + 'comp, it is possible that the type erased component does contain a reference, or something else with a non-'static lifetime. It doesn't have to, but it is possible.
dyn Trait + '_ has some special capabilities here, but let me try to answer this for the general case first.
Let's say you have a s: Something<'long>. You can't use s "outside" of 'long; we could say, 's is only valid for 'long.
So when we create a &mut Something<'original_borrow>, due to the invariance discussed above, 'original_borrow has to be the exact lifetime in the type of the borrowed value -- that is, the duration that the borrowed value is valid for.
Next let's say we've annotated things so that we force the outer lifetime on the &mut to be equal to the inner lifetime on Something<'_>:
The outer lifetime represents the duration of an exclusive borrow. Now we've created an exclusive borrow of Something<'a> that lasts for 'a -- the entire duration the borrowed value is valid for. We have effectively borrowed the original value forever. When something is borrowed, you can't move it or run a non-trivial destructor on it. When something is exclusively borrowed, you can't use it at all. (And if you could, it would be unsound.)
So that's the general problem with forcing nested lifetimes to be equal.
Because &'a mut dyn Trait desugars to &'a (mut dyn Trait + 'a), this can actually be a somewhat important capability. In fact, it can be used to fix your OP without changing anything outside of main! It didn't kick in for a quite subtle reason.
This is going to get into the weeds. I'm not sure if that will be helpful or hurtful to helping you understand more, so feel free to skip or skim or ignore it.
Let's walk through your OP and look at the types.
// (1) This is a `RefCell<Vec<Box<dyn Component + 'static>>>`
let test = create_cmps();
// (2) This is a `RefMut<'t, Vec<_>>`.
let mut test_borrow = test.borrow_mut();
// (3) This is a
// ```
// slice::IterMut<'m, Box<dyn Component + 'static>>
// ```
// It's iterator implementation has
// ```
// Item = &'m mut Box<dyn Component + 'static>
// ```
let iter = test_borrow.iter_mut();
(I split up the iterator line to better highlight some important parts below.)
In every type so far, the dyn Component + '_ lifetime has been in an invariant position and too nested to unsize the lifetime to something shorter. Then you have this line:
// (4) This is a `Map<IterMut<'m, _>, F>` where `F` is the closure.
//
// The closure signature is:
// ```
// FnMut(&'m mut Box<dyn Component + 'static>) -> Ret
// ```
//
// Where `Ret` will also be the `Item` type of the `Map<_, F>` iterator.
let combined = test_borrow.iter_mut().map(|cmp|
// (5) This call returns a `&mut (dyn Component + 'static)`
cmp.as_mut()
);
Within the function body, we create a &'m mut (dyn Component + 'static). That could be coerced to a &'m mut (dyn Component + 'm) using the special dyn lifetime capability. But for reasons we'll come back to, the compiler decided that there was no reason to try such a coercion at location (5).
Finally you try to box up and type-erase combined when putting it into a Holder.
// (6)
// The field is a:
// ```
// Box<dyn Iterator<Item = &'a mut (dyn Component + 'a)> + 'a>
// ```
//
// So the compiler expects `combined` to be an iterator with
// ```
// Item = &'a mut (dyn Component + 'a)
// ```
//
// It finds one with
// ```
// Item = &'m mut (dyn Component + 'static)
// ```
//
// These can only line up if `'m` is `'static`.
let _ = Holder {
number: Box::new(combined),
};
This is why your original error message said
returning this value requires that `test` is borrowed for `'static`
Now, back to the closure at points (4) and (5). The return types of closures are inferred. Why didn't inference see that you needed to return a &m mut (dyn Component + 'm)? That would have avoided this error.
let mut test_borrow = test.borrow_mut();
+ // We add an explicit coercion from `&'m mut dyn (_ + 'static)`
+ // to get a `&'m mut dyn (_ + 'm)` vvvvv
+ let combined = test_borrow.iter_mut().map(|cmp| cmp.as_mut() as _);
- let combined = test_borrow.iter_mut().map(|cmp| cmp.as_mut());
let _ = Holder {
let mut test_borrow = test.borrow_mut();
- let combined = test_borrow.iter_mut().map(|cmp| cmp.as_mut());
+ let combined = test_borrow.iter_mut().map(|cmp| { cmp.as_mut() });
let _ = Holder {
In both cases, the presence of a coercion site allows how combined is used to influence the return type of the closure. The ability to use the special dyn lifetime coercion means the return type can be &'m mut (dyn _ + 'm), and 'm is no longer required to be 'static for the lifetimes to match up with the Holder field when you type-erase at point (6).
.