I bumped upon this strange behavior and I am not quite sure which is the right thing. I try to cast a vector of futures to a vector of shorter-lived futures and it works. But when I try to cast a SegQueue to a SegQueue of shorter lived futures it fails. I don't quite understand why the difference in behavior and which one is the "correct" one.
use std::pin::Pin;
use std::future::Future;
use crossbeam::queue::SegQueue;
// This works
struct Futures<'a> (Vec<Pin<Box<dyn Future<Output = ()> + 'a>>>);
// This does not
// struct Futures<'a> (SegQueue<Pin<Box<dyn Future<Output = ()> + 'a>>>);
impl<'a> Futures<'a> {
fn covariance<'b>(self) -> Futures<'b>
where
'a: 'b,
{
// Here I get mismatched types
self
}
}
This is because SegQueue<T> is invariant in T, which in turn is because it’s a type with interior mutability (i.e. it can be modified in &self methods), which is why &SegQueue<T> must be invariant for the same reason why &mut T is invariant [1].
The approach Rust libraries take to ensure that a type like &SegQueue<T> is invariant whilst &S is covariant in S is to make the interiorly-mutable type sucht as SegQueue<T> itself invariant, despite the fact that converting something like SegQueue<Pin<Box<dyn Future<Output = ()> + 'a>>> into SegQueue<Pin<Box<dyn Future<Output = ()> + 'b>>> for an owned datastructure like this is arguably technically not problematic; still even in your case, &Futures<'a> must definitely not be covariant in 'a anymore, so the same logic applies why Futures<'a> shall be invariant, too.
The way this works in practice is that the interior mutability primitive, UnsafeCell<T>, itself is already not covariant in T, and many other such types, including SegQueue (in this case through usage of AtomicPtr) inherit this property.
that is: if Foo<'a> can be converted into Foo<'b> but not vice versa, then converting &mut Foo<'a> into &mut Foo<'b> would mean you can assign the Foo<'b> into the resulting &mut Foo<'b>, and then read a Foo<'a> out of the original &mut Foo<'a>, in effect converting the wrong way; hence converting &mut Foo<'a> into &mut Foo<'b> can’t be permitted. ↩︎
First of all thank you very much for the response it is very helpful.
So let me put forth my particular case. I am trying to make an asynchronous sychronization primitive:
I have a structure Futures<'long> where I store all the futures to be synchronized.
I have some impl Future<..> + 'short (where 'long:'short) that should wait for the synchronization primitive.
A corresponding guard of lifetime 'guard is created within an impl Future + 'short when it is popped from the queue.
I can make sure that impl Future + 'short objects are popped from the queue before the end of 'short. (because 'short: 'guard and pop happens at the beginning of 'guard).
It is probably (next to?) impossible to convince the borrow checker that the guards will ensure the final point so I was planning on implementing an exceptionally unsafe function that rhymes with
impl<'long> Futures<'long> {
// SAFETY: Make sure anything inserted in Futures is fully evaluated by the end of
// 'short.
unsafe fn covariance<'short>(&self) -> &Futures<'short>
where
'long: 'short,
{
// Do my worst
}
}
I know this is a vague question but, in your opinion, is it possible to get such an approach right? Would you suggest a more idiomatic way?
If you are using unsafe to circumvent borrowck and the lifetime system, then you are almost certainly missing some subtle detail and the result is going to be surprise-UB. If you show us some actual, minimal, self-contained example code that almost compiles, we might be able to help much more easily with the solution in this specific case.
The description is indeed a bit vague. I read your bullet list a few times and still am not sure what exactly you have in mind, in particular with regards to the "guard" mentioned in the 3rd point.
Whether or not an API is sound is also always a question of details, that can only be answered for a very concrete API.
Regarding the general question, one can say that it is allowed in unsafe rust to modify lifetimes of a type via transmute (i. e. there's no rule that says that this is instand undefined behavior) and also lifetimes don't carry any inherent "semantic" meaning beyond enduring a sound API, ... so yeah in principle, if you do somehow ensure that all added futures are removed from the queue before their respective lifetimes become invalid, then that's something you can do in principle, using unsafe code, so this can work.
On the other hand, it's always easy to get things wrong: misunderstanding some lifetimes, forgetting about corner cases; in case of a synchronized shared mutable queue like this perhaps even edge cases in ordering of execution between multiple threads; so there's a lot to consider... also, panic safety, and for Futures, the possibility that those can be canceled, etc.