I'm trying to poll a future that I'm storing in a struct, which I can wrestle into the type Pin<&mut Box<(dyn Future + 'static)>>. When trying to call poll on this, I'm getting an error that Future isn't implemented for Box<(dyn Future + 'static)> because (dyn Future + 'static) doesn't implement Unpin.
All of that matches up with the docs, which is fine, but what I'm trying to understand is why a Future wrapped in a box doesn't automatically implement Unpin. Isn't the purpose of wrapping futures as Pin<Box<Future>> is that Box makes anything it wraps Unpin? Or do I need Pin<&mut dyn Future> instead of Pin<&mut Box<dyn Future>>? If so, is there a good way of getting rid of the extra layer of reference inside of the Pin?
Pin doesn't make whatever it wrapps Unpin. Pin works in tandem with the the Unpin marker in a way that any type which is !Unpin must be pinned and can't move until it's dropped.
The error message gets confusing since what you're trying to do is something that can't be done when the type is !Unpin. This suggests that you're not actually pinning the future as expected.
Now, onto your problem. It would be great if you could add some more context to this, but I have a feeling that you don't want to store a reference to a Box as in Pin<&mut Box<(dyn Future + 'static)>>, you should probably have something looking like Pin<Box< dyn Future + 'static>>. There is a shortcut for that using Box::pin when you box your future.
I could be wrong since I don't have more context so see if you can recreate an example in the playground if you're still having trouble and add that so you'll get a proper answer if this doesn't help.
I'm honestly not sure why I didn't include a playground link, so here it is (hopefully it's not too contrived, I didn't want to display my actual mess of an implementation): Rust Playground
The background is that I'm trying to implement a Stream that can repeatedly call a paginated API until there are no items left. My current strategy has been to store the future in an Option<Box<dyn Future>>: if there is a future, poll it, and if there isn't, create a new one for the next page in the API.
Using the pin_project crate, I can get the field as Pin<&mut Option<Box<dyn Future>>>, and then I can manage to remove the Option and get it down to a Pin<&mut Box<dyn Future>>. Like I've discovered, though, I can't call pin on that since the dyn Future doesn't implement Unpin.
This might be a case where my initial concept of how to implement Stream is just off.
Basically an Box<SomeFutureType> allows you to mem::swap the future inside, which would move it. A Pin<Box<SomeFutureType>> does not allow that operation. This is why only the former requires the Unpin trait.
You should give some more details on what you're trying to do. Do you want to call some async function repeatedly and use the return value as items in the stream?
Now there is some problems still since you're returning an usize but the signatures indicates you want to return an Option but I'm not 100% sure how you want to decide wether to return None or Some(usize). But I hope it answers your initial question.
Ok, I think removing pin_project and restructuring the type worked, here's the fleshed out version in playground with your changes: Rust Playground (fixed the link)
I'm gonna mess around with this some more, but that may have been it. I think when I saw pin_project being used, it was on a generic struct and the field was Option<F>, so the Future was probably colocated with the struct (making pin_project more useful). This set up probably makes more sense now.
I think the link is to a version that still uses pin_project, but good to hear you're on the right track.
Just to add a little extra explanation. When you have a Pin<&mut Box<..>> you're pinning a pointer to the heap allocated data. I haven't used pin_project before but AFAIK it's for stack pinning a field of a struct (struct.field is a projection) without having to use unsafe (which you'll normally have to do when you pin data to the stack).
When you pin data to the stack you create a Pin<&mut data>, but since you're using a Box you avoid this complexity. However, since Box is a pointer already you shouldn't have a Pin<&mut Box<..>>.
Links updated, and ya, that's all starting to click a lot more. I was butting my head into some Pin/Unpin stuff earlier, and a video from Jon Gjengset gave me an idea at 3am to try using pin_project. Now I think I've got a much better idea of the use of that library.
Thats good, the videos from Jon is really good but Pinning can be a bit hard to get a good mental model of. Not too long ago I wrote about Pinning (and a lot of other stuff) to make this easier for myself (and others) to understand. It might be worth checking out if you find the time.
@cfsamson and @alice Thanks for the help, after messing around with the code (with and without pin_project), I've got a much better understanding of futures and pins.
Also, I remember bookmarking that page you wrote on pinning and planned to read it later. I think if I had read that excerpt on the naming scheme of !Unpin being a safety feature, I probably would've realized I should go to bed to tackle this in the morning.