Type mismatch when refactoring code into a function

cmusser · August 20, 2020, 11:16pm

I have a match block that is repeated in more or less identical form in a few places. I'd like to refactor that into a function, but ran into type mismatch issues. The match block uses a future creation function that returns an impl Trait and I think this is making the types harder to get right.

The future creation function signature is:

fn get_cmd_future(
    name: &str,
    cmd: &mut CmdSpec,
) -> Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error> {
}

The match block that I'd like to refactor is:

match get_cmd_future(cmd_to_start, &mut cmd) {
    Ok(spawned_child) => {
        println!("starting {}", cmd_to_start);
        all_futures.push(Either::Right(spawned_child));
        cmds.insert(cmd_to_start.to_string(), cmd);
    }
    Err(e) => println!("spawn failed: {:?}", e),
}

Background: all_futures is a FutureUnordered<_> and cmds is a HashMap<String, CmdSpec>. CmdSpec is a run-of-the-mill struct with a few fields.

The closest I could get to a successful compilation was the following:

fn start_process<T: Future<Output = Result<CompletedCmd, std::io::Error>>>(
    cmd_to_start: &str,
    cmd: CmdSpec,
    cmds: &mut Cmds,
    all_futures: &mut FuturesUnordered<Either<T, T>>,
) {
    match get_cmd_future(cmd_to_start, &mut cmd) {
        Ok(spawned_child) => {
            println!("starting {}", cmd_to_start);
            all_futures.push(Either::Right(spawned_child));
            cmds.insert(cmd_to_start.to_string(), cmd);
        }
        Err(e) => println!("spawn failed: {:?}", e),
    }
}

This produces the following error:

error[E0308]: mismatched types
   --> src/boss.rs:121:44
    |
67  | ) -> Result<impl Future<Output = Result<CompletedCmd, std::io::Error>>, std::io::Error> {
    |             ---------------------------------------------------------- the found opaque type
...
112 | fn start_process<T: Future<Output = Result<CompletedCmd, std::io::Error>>>(
    |                  - this type parameter
...
121 |             all_futures.push(Either::Right(spawned_child));
    |                                            ^^^^^^^^^^^^^ expected type parameter `T`, found opaque type
    |
    = note: expected type parameter `T`
                  found opaque type `impl core::future::future::Future`
    = help: type parameters must be constrained to match other types

If I look at the type for spawned_child, it's not quite what I expect: impl core::future::future::Future. I would have expected it to be more specific, as the return signature of `get_cmd_future() would indicate.

But, I don't really understand the error message and I am still not clear what on opaque types actually are. I think the error says "spawned_child needs to be a more specific type", but I couldn't find a way to "coerce" it or otherwise make it match. Is there a way to fiddle with the type specifications here to get this to work or is a different approach needed?

Thanks,
Chuck

2e71828 · August 21, 2020, 8:31am

An opaque type represents a value of some concrete type known at compile time, but that exposes a limited interface. This allows library authors to change their implementation without breaking user code. If we replace the impl Future<...> in your example with a hypothetical struct CmdFuture, the problem becomes more apparent:

struct CmdFuture { /* ... */ }
impl Future for CmdFuture {
    type Output = Result<CompletedCmd, std::io::Error>;
    /* ... */
}

fn get_cmd_future(
    name: &str,
    cmd: &mut CmdSpec,
) -> Result<CmdFuture, std::io::Error> {
    /* ... */
}

fn start_process<T: Future<Output = Result<CompletedCmd, std::io::Error>>>(
    cmd_to_start: &str,
    cmd: CmdSpec,
    cmds: &mut Cmds,
    all_futures: &mut FuturesUnordered<Either<T, T>>,
) {
    match get_cmd_future(cmd_to_start, &mut cmd) {
        Ok(spawned_child) => {
            println!("starting {}", cmd_to_start);
            all_futures.push(Either::Right(spawned_child));
            cmds.insert(cmd_to_start.to_string(), cmd);
        }
        Err(e) => println!("spawn failed: {:?}", e),
    }
}

Here, you’re trying to store a value of Right(CmdFuture) into an Either<T,T>. Because T isn’t guaranteed to be CmdFuture, the compiler complains. The fix in our hypothetical case is simple: all_futures should be &mut FuturesUnordered<Either<T, CmdFuture>>. Unfortunately, there’s no way to name a particular impl Trait type like this.

There are a few solutions to this:

If you are ok with using nightly-only features, you can explicitly name the return type of get_cmd_future with #![feature(type_alias_impl_trait)]
You can box the futures and use dynamic dispatch
You can return Option<impl Future<...>> from your function and let the caller be responsible for storing the result.

cmusser · August 22, 2020, 1:06am

OK, thanks for the insights there.

I did consider Boxing the futures, but I was trying to avoid that because I thought the needed changes would ripple through and give me even more compilation headaches. It doesn't seem as efficient (heap allocations) too, but this isn't really a performance-critical program.

I didn't understand how wrapping the impl in an Option would do it. Is that because it just makes it a distinct concrete type?

What I did try successfully was your first suggestion: switch to nightly and use the syntax described in RFC 2515. I got a little lost reading about existential types, before I encountered that. With a little rework, I got the factored-out function to compile and run correctly. I don't understand it totally, but I think I'll think about it some more and try to summarize what I learned here in a bit.

2e71828 · August 22, 2020, 7:25am

Sorry; I should’ve explained that one better. impl Trait as part of a return value behaves differently from impl Trait in the argument list: it’s a statement about what you’re going to provide instead of what you’ll accept. So, if your function signature looks like this:

fn start_process(
    cmd_to_start: &str,
    cmd: CmdSpec,
    cmds: &mut Cmds,
) -> Option<impl Future< Output = Result<CompletedCmd, std::io::Error>> {
    /*... */
}

You can pass through the output of get_cmd_future without any trouble here. You’ll obviously have to deal with the impl Future at the callsite one level up, but that might be able to use type inference:

let mut all_futures: FuturesUnordered<Either<_,_>> = ...;
if let Some(fut) = start_process(cmd_to_start, cmd, cmds) {
    all_futures.push(Either::Right(fut))
};

Yandros · August 22, 2020, 3:08pm

Minimal repro:

fn foo () -> impl Foo
{
    /* ... */
}

fn bar (out: &'_ mut impl Foo)
{
    *out = foo();
}

errors with:

error[E0308]: mismatched types
  --> src/lib.rs:11:12
   |
4  | fn foo () -> impl Foo
   |              -------- the found opaque type
...
9  | fn bar (out: &'_ mut impl Foo)
   |                      -------- this type parameter
10 | {
11 |     *out = foo();
   |            ^^^^^ expected type parameter `impl Foo`, found opaque type
   |
   = note: expected type parameter `impl Foo` (type parameter `impl Foo`)
                 found opaque type `impl Foo` (opaque type at <src/lib.rs:4:14>)
   = help: type parameters must be constrained to match other types
   = note: for more information, visit https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters

Basically the idea is that, on the one hand, we have:

-> impl Trait, called ITRP (impl Trait in return position),
which is just a way to hide, API-wise, the specifics / the name of the type returned by the function. But such type is still a concrete type compiler-wise, as @2e71828 explained.

In practice, such type is callee-chosen: the actual body / implementation of the function will let the compiler determine which concrete type is hidden.
arg: impl Trait, called ITAP (impl Trait in argument position),
which is sugar for <Arg : Trait> ... arg: Arg, that is,
a caller-chosen type parameter.

In the minimal repro above, foo() is thus of some concrete type (implementation-specific), whereas *out is of a type chosen by the caller, with the only constraint of being a type that impls Foo. So a caller could choose a type different from that of foo(), hence the error.

Your example is exactly the same case, but with Future being the trait involved, and the type of out being a bit more convoluted. But it's still some form of container where you want to push those specific futures.

The solutions, in practice, are thus mentioned by @2e71828.

I'll just add another approach, which can be interesting, at least for the educational aspect:

when you think about this, for some given concrete implementation of get_cmd_future, you do have an associated specific return type. So, in a way, meta-programmign-wise, we could say that the function you are trying to write is generic over the get_cmd_future in scope.

And since the issue you have is that the type, by virtue of being unnameable (imagining that type X = impl ... is not available), leads to the function needing some form of genericity to be able to cover that unnameable type, we could solve everything by making your whole function generic over get_cmd_future: this lets us constraint that the caller-chosen type parameter T must match the caller-chosen get_cmd_future implementation.

fn start_process<T, GetCmdFuture> (
    get_cmd_future: GetCmdFuture,
    ...
    all_futures: &mut FuturesUnordered<Either<T, T>>, // <------------+
)                                                                  // |
where                                                              // | Same `T`, they must match.
    T : Future<Output = io::Result<CompletedCmd>>,                 // |
    GetCmdFuture : FnOnce(&str, &mut CmdSpec) -> io::Result<T>, // <--+
{
    ...
}

So now, if a caller goes and provides our concrete get_cmd_future implementation:

fn get_cmd_future (
    name: &str,
    cmd: &mut CmdSpec,
) -> io::Result<impl Future<Output = io::Result<CompletedCmd>>>
{
    ...
}

type inference will take care of picking T equal to that unnameable type:

let ref mut all_futures = FuturesUnordered::new();
start_process(get_cmd_future, /* ... */ all_futures)

Playground

Ideally, if defining fn functions also defined an equally named type that implemented Fn... with the same signature as the function, then we would be able to write:

type T = <get_cmd_future as FnOnce<(&str, &mut CmdSpec)>>::Output;

fn start_process </* no longer generic */> (
    cmd_to_start: &str,
    cmd: CmdSpec,
    cmds: &mut Cmds,
    all_futures: &mut FuturesUnordered<Either<T, T>>,
)
{ ... }

But since that is not possible, in practice the aforementioned type T = impl ... is the go-to solution (but nightly-only!):

+ type T = impl Future<Output = io::Result<CompletedCmd>>;
  fn get_cmd_future(
      name: &str,
      cmd: &mut CmdSpec,
- ) -> io::Result<impl Future<Output = io::Result<CompletedCmd>>>
+ ) -> io::Result<T>
  { ... }

- fn start_process <T : Future<Output = io::Result<CompletedCmd>> (
+ fn start_process /* no longer generic */ (
      cmd_to_start: &str,
      cmd: CmdSpec,
      cmds: &mut Cmds,
      all_futures: &mut FuturesUnordered<Either<T, T>>,
  )
  { ... }

system · November 20, 2020, 3:08pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Confusing opaque type with nom help	9	638	January 21, 2024
Impl Trait problems with pattern matching help	3	1316	January 12, 2023
Expected opaque type, found a different opaque type	11	13057	August 19, 2019
Matching trait returns help	10	2681	January 12, 2023
Generic error, don't understand error message help	5	1541	December 13, 2019

Type mismatch when refactoring code into a function

Related topics