Error: distinct uses of `impl Trait` result in different opaque types

Hi,

I am new to Rust but I tried playing around with some code samples and see the limits. I tried doing a match on some string that would return me one of two async functions in order to later call it, see the following code:

fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    let func = match input.trim() {
        "1" => a,
        _ => b,
    };
    func();
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}

(Playground)

Despite how silly this code is, what would be the correct way forward? Is there a fix I can use to keep this idea, or should I just write something else completely?

I don't understand why I am getting distinct uses of 'impl Trait' result in different opaque types in this case.

Well that’s just a note and not the “main” error message and it is certainly slightly confusing since you don’t write impl anywhere. The main error is `match` arms have incompatible types and this means that a and b have different types. Now what are the types of a or b?

A way to “desugar” the async function a

async fn a() -> u32 {
    1
}

is like this:

fn a() -> impl Future<Output=u32> {
    async {
        1
    }
}

there we got our impl.

What this means is that a() returns some type that is not further specified in the signature (but is going to be some concrete, in this case anonymous, type) that implements the trait Future. This type is generated by the async block. Now the return type of a is different from b, even though b looks really similar

fn b() -> impl Future<Output=u32> {
    async {
        b
    }
}

See, the reason they look similar is only that we don’t explicitly write the actual return type, (and neither could we, since it’s an anonymous type) so b() as well returns some type of future, but this future is clearly a different type of future. Why? Well, it behaves differently, it produces a 2 instead of a 1 when evaluated. (And even if it did behave the same, we must not rely on this because it must be possible to change the implementation of these “opaque” types without this change being problematic.)


Now on to the question how we could try to fit these two types back together into a single one. There’s two routes to go here—trait objects or enums.

Let’s consider trait objects first. The way to fit all kinds of impl Future into the same type is to convert them into dyn Future (roughtly speaking). I’ll just present some code that also uses Box and Pin. The Box because that’s a simple way to handle trait objects (and I couldn’t find a nice way to avoid it) and the Pin because you need it for Future for technical reasons. Code follows:

use std::future::Future;
use std::pin::Pin;

fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    let func: fn() -> Pin<Box<dyn Future<Output=u32>>> = match input.trim() {
        "1" => || Box::pin(a()),
        _ => || Box::pin(b()),
    };
    func();
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}

One thing to note is that the call to func() doesn’t do anything yet. I would suggest using tokio to allow for an async main function, and call func() with the necessary await to actually get a value out:

use std::future::Future;
use std::pin::Pin;

#[tokio::main]
async fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    let func: fn() -> Pin<Box<dyn Future<Output=u32>>> = match input.trim() {
        "1" => || Box::pin(a()),
        _ => || Box::pin(b()),
    };
    let result: u32 = func().await;
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}

The enum version uses, well, an enum. See, an enum is the way to combine multiple types. Want an int of float? Do this!

enum IntOrFloat {
    Int(i64),
    Float(f64),
}

Want either this opaque future type of that opaque future type that you both can’t name and you don’t actually care about the name of the enum either and would prefer not to have to implement a new one either and the best thing would be if like the dyn Future construct version the enum would also be a type that is a future itself?

Perhaps a bit too specific this question but the answer is, you can have it! It’s called Either (there’s multiple versions of this type throughout different crates, I’m referring to futures::future::Either which has different behavior around Pin than either::Either if I understand this correctly).

Without further ado, the code example, hope it helps:

use futures::future::Either::*;

#[tokio::main]
async fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    let func = match input.trim() {
        "1" => || Left(a()),
        _ => || Right(b()),
    };
    let result: u32 = func().await;
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}
1 Like

Thank you for your answer! It shines a lot of light on what I should focus learning about :slight_smile:

Ah true, in my original experiment I was using tokio already, but for the sake of simplifying my example I forgot to include it back.

Now on to the question how we could try to fit these two types back together into a single one.

Just looking at your Box + Pin example, it looks like this solution would scale to more than 2 branches inside the match statement, did I understand correctly? I also see that you are evaluating the async functions inside the match branches, although it might not execute any code until we await it as part of some tokio task?

Regarding the enum example, I have trouble right now seeing how it would work with multiple async functions?

actually, I’m not executing them, note the ||. I am wrapping them into new functions that add the conversion to Pin<Box<..>> or Either<...>. In the case of functions with more than zero parameters, you’d also need to pass the parameters, like |x,y,z| Left(f(x,y,z)). In the case of async functions without parameters, it could actually be very reasonable to actually those functions directly in the match and just keep the futures around.

Yes, and the Either one scales, too. You can use 3 branches with something like for example Left(Left(a)), Left(Right(b)) and Right(c). I’m not 100% sure what the most efficient way further is, either only doing Left(Left(...(Left(Right( chains of different length, of going full “binary encoding”. (I have a feeling the binary version might be less efficiently layouted in Rust, but that would have to be tested.)

In case you have no idea what I mean:

Left Left Left Left Left Left Left
Left Left Left Left Left Left Right
Left Left Left Left Left Right
Left Left Left Left Right
Left Left Left Right
Left Left Right
Left Right
Right

vs

Left Left Left
Left Left Right
Left Right Left
Left Right Right
Right Left Left
Right Left Right
Right Right Left
Right Right Right
1 Like

I would have expected Rust to fold the discriminants of nested enums together via niche optimization, but it looks like it isn‘t happening: (Playground)

size  type
----  -------------------------------------------------------
   1  u8
   8  usize
  16  Either<usize, usize>
  32  Either<Either<Either<usize, usize>, usize>, usize>
  24  Either<Either<usize, usize>, Either<usize, usize>>

Edit: this is a known bug.

1 Like

It doesn't fold the discriminants together because it has to be able to produce a reference to the Either inside the Either, which ultimately means each level needs its own discriminant.

1 Like

Only applies to the balanced versions like Either<Either<usize,usize>,Either<usize,usize>>.


Let’s lay out a Either<Either<Either<usize,usize>,usize>,usize> (proceeding bottom up):

  • Either<usize,usize> needs 1 byte tag + 4 byte payload. With alignments that’s
    tag (1 byte, with niche 2..=255) | 3 bytes padding (uninitialized) | 4 byte data
    tag value is 0 for Left and 1 for Right.
    value examples (list of decimal bytes, * is uninitialized):
      Left<42>  ≙ 0 * * * 0 0 0 42
      Right<42> ≙ 1 * * * 0 0 0 42
    
  • Either<Either<usize,usize>,usize> needs to contain a Either<usize,usize>, so 8 bytes at least. For its tag it can use the niche of Either<usize,usize> for the Right case and just directly contain the Either<usize,usize> for the Left case (similar to Option)
    tag (1 byte, with niche 3..=255) | 3 bytes padding (uninitialized) | 4 byte data
    tag value is `0..=1` for Left and 2 for Right
    value examples (list of decimal bytes, * is uninitialized):
      Left<Left<42>>  ≙ 0 * * * 0 0 0 42
      Left<Right<42>> ≙ 1 * * * 0 0 0 42
      Right<42>       ≙ 2 * * * 0 0 0 42
    
  • Either<Either<Either<usize,usize>,usize>,usize> needs to contain a Either<Either<usize,usize>,usize, so 8 bytes at least. For its tag it can use the niche of Either<Either<usize,usize>,usize for the Right case and just directly contain the Either<Either<usize,usize>,usize for the Left case (similar to Option)
    tag (1 byte, with niche 4..=255) | 3 bytes padding (uninitialized) | 4 byte data
    tag value is `0..=2` for Left and 3 for Right
    value examples (list of decimal bytes, * is uninitialized):
      Left<Left<Left<42>>>  ≙ 0 * * * 0 0 0 42
      Left<Left<Right<42>>> ≙ 1 * * * 0 0 0 42
      Left<Right<42>>       ≙ 2 * * * 0 0 0 42
      Right<42>             ≙ 3 * * * 0 0 0 42
    
2 Likes

Ah indeed!

Hmm to be honest I really don't like the shape this method is taking, even following your examples and explanations... It looks kind of odd, at least to me :slight_smile:

But don't get me wrong: everything you went through is really interesting, but something tells me I shouldn't stay stubborn trying to do what I tried to do.


My conclusion right now is that I tried to do something silly (and my opinion is that the compiler is just making my life hard in order to make me understand that this is not a viable way of doing what I want to do, ahah)

The original idea was to have some dynamic dispatch between different async handlers. Do you know of a more idiomatic way of writing this? So far I've just been trying code substitutions until all errors are fixed, but I feel I could approach the problem differently?

If you want dynamic dispatch, you can just Box::pin them. Note that instead of writing out the pin box type, you might prefer the BoxFuture type alias in the futures crate.

1 Like

I'll read about those, thanks!

You can use late-initialized stack variables and references to them [playground]:

fn main() {
    let input: String = /* ... */;
    let mut place_a;
    let mut place_b;
    let future: &mut dyn Future<Output=u32> = match input.trim() {
        "1" => { place_a = a(); &mut place_a }
        _ => { place_b = b(); &mut place_b }
    };
    drop(future); // do something with it
}

The problem is pinning, as you can't just use futures::pin_mut! to pin the values to the stack, as that relies on being able to create new bindings and shadow names to be sound. I believe it would be sound to just Pin::new_unchecked the reference, as both places the future may be are potentially uninitialized and thus unable to have a regular &mut reference to them created, but I'm not going to say that for certain.

(Also, IIUC this reserves stack space for both futures. I think it should be possible for this to overlap the stack space for both places in the future, but I'm pretty sure that optimization is not done currently. Use Either instead of this trick, it's easier for the compiler to work with and probably easier for you as well.)

1 Like

@marechal-p I totally forgot I wanted to mention this one, too. I even had example code like

use futures::future::{FutureExt, BoxFuture};

fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    let func: fn() -> BoxFuture<'static, u32> = match input.trim() {
        "1" => || a().boxed(),
        _ => || b().boxed(),
    };
    func();
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}

Also you could introduce your own tag type and construct a future as an async block:


fn main() {
    let mut input = String::new();
    println!("input unsigned integer:");
    std::io::stdin()
        .read_line(&mut input)
        .expect("readline error");
    enum Tag { Case1, Case2 };
    use Tag::*;
    let tag = match input.trim() {
        "1" => Case1,
        _ => Case2,
    };
    let func = || async {
        match tag {
            Case1 => a().await,
            Case2 => b().await,
        }
    };
    func();
}

async fn a() -> u32 {
    1
}

async fn b() -> u32 {
    2
}
2 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.