Specifying the type of a `FuturesUnordered`

I've mis-structured some futures-related code in such a way that I can create a FuturesUnordered from a Vec of futures, but I cannot pass individual futures of that type to the FutureUnordered's push method. I need a way to specify the type of the FutureUnordered more precisely or come at this a different way.

To create an individual future, I have a utility function with the following signature.

fn get_cmd_future(
    name: &str,
    subprocess: &CmdSpec,
) -> Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error> {

Side note: the futures I'm using are child processes provided by tokio::Process, and mapping the output to a new type with some additional stuff my program needs. The Tokio call that creates the "process child future" can fail, which is why the whole thing is in wrapped in a Result.

To create a Vec of these futures, I have another utility function, whose signature is:

fn into_futures(
    cmds: &Cmds,
) -> Vec<Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error>> {

Creating the FuturesUnordered looks like this. Notice the "opaque type" on the FuturesUnordered. I couldn't figure out how to specify a type here, and the opaque type what allows it allowed it to compile:

    let mut all_futures: FuturesUnordered<_> =
        cmd_futures.into_iter().filter_map(Result::ok).collect();

On the above, the filter_map removes the future creations that were Err. It's a bit of a mess. Probably should do that in the into_futures function.

The opaque type is a band-aid that allows it to compile and work, but, trying to use the "future creation" utility later with the FuturesUnordered instance does not work. This is how I tried:

let reinvocation = get_cmd_future(&child.name, cmd_spec).unwrap();
all_futures.push(reinvocation);

And this is the compiler error that results:

error[E0308]: mismatched types
   --> src/boss.rs:133:50
    |
54  | ) -> Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error> {
    |             ---------------------------------------------------------- the found opaque type
...
88  | ) -> Vec<Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error>> {
    |                 ---------------------------------------------------------- the expected opaque type
...
133 |                                 all_futures.push(reinvocation);
    |                                                  ^^^^^^^^^^^^ expected opaque type, found a different opaque type
    |
    = note: expected opaque type `impl core::future::future::Future` (opaque type at <src/boss.rs:88:17>)
               found opaque type `impl core::future::future::Future` (opaque type at <src/boss.rs:54:13>)
    = note: distinct uses of `impl Trait` result in different opaque types

I tried various ways around this. impl Trait can't be used in a let binding, so that wasn't the solution. Specifying the Future and its Output doesn't work either because the size is not known. I thought the unknown size resulted from that type (CompletedCmd) contained a String, but converting that to a borrowed &str didn't help. Maybe the other fields (Instant and ExitStatus are unknown sized too.

Is there a way to restructure this so I can create the FuturesUnordered from a Vec of futures and push individual futures onto it later?

Thanks,
Chuck

The issue is that every impl Future is actually its own unique type under the hood, and you can't put two different future types into the FuturesUnordered. To get around this, you have two options.

  1. Box the futures

Call Box::pin(fut) on every future before adding it, and specifying the type like this:

use futures::future::BoxFuture;

// the underscore is the return type
let mut all_futures: FuturesUnordered<BoxFuture<_>> = ...;
  1. Use Either.

The either type is a sort of light-weight boxed future, that allows two specific actual types. It's cheaper than boxing, but boxing allows any number of future types.

let mut all_futures: FuturesUnordered<_> = cmd_futures.into_iter()
    .filter_map(Result::ok)
    .map(|fut| Either::Left(fut))
    .collect();

let reinvocation = get_cmd_future(&child.name, cmd_spec).unwrap();
all_futures.push(Either::Right(reinvocation));

Nice, Either did the job here. Thanks for the tip. Either seems to come in handy for other situations where there's slight variability in types (I guess "slight" really means "two"...). I think I've used it in other situations, but it certainly did not occur to me for this one.

I did not try the Box method. That did cross my mind. However, I thought the purpose of impl Trait syntax was to avoid having to Box those return values. But my understanding is hazy. In any, I would have not known the stuff about Pinning. I may try to make it work that way too.

On the uniqueness of every `impl Future: every place where a trait implementation is returned, that forms a distinct type. They may look exactly the same, but are not equivalent?

Given that fact, is it true there would be no way to write the variable declaration of the FuturesUnordered without the <_> syntax that allows the type to be "unknown" or "opaque"?

Thanks,
Chuck

In principle you can put Either inside of Either to get three or four, but yes.

Correct.

When it's a place-holder for an impl Future, then yes, you can't put in the actual type. That said, in my box example, the place-holder is for the return type, which you can definitely put in.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.