Closure as argument. Parameterization with `impl Trait`

Hello. What is wrong with this code?

pub struct VectorFormer<ContainerEnd>
where
    ContainerEnd: Fn(),
{
    on_end: ContainerEnd,
}

impl<ContainerEnd> VectorFormer<ContainerEnd>
where
    ContainerEnd: Fn(),
{
    fn new(on_end: ContainerEnd) -> Self {
        Self { on_end }
    }
}

//

pub struct CommandFormer {}
impl CommandFormer {
    pub fn problem<ContainerEnd>() -> VectorFormer<ContainerEnd>
    where
        ContainerEnd: Fn(),
    {
        let on_end = || {
            println!("on_end");
        };
        VectorFormer::<ContainerEnd>::new(on_end)
    }
}

Output:

   |
37 |       pub fn problem<ContainerEnd>() -> VectorFormer<ContainerEnd>
   |                      ------------ this type parameter
...
41 |           let on_end = || {
   |  ______________________-
42 | |             println!("on_end");
43 | |         };
   | |_________- the found closure
44 |           VectorFormer::<ContainerEnd>::new(on_end)
   |                                             ^^^^^^ expected type parameter `ContainerEnd`, found closure
   |
   = note: expected type parameter `ContainerEnd`
                     found closure `[closure@src/main.rs:41:22: 43:10]`
   = help: every closure has a distinct type and so could not always match the caller-chosen type of parameter `ContainerEnd`

Playground

A generic type parameter is chosen by the caller. However, on_end is defined in the function, by the function, so its type is also determined by the function. It doesn't make sense to expect it to be adapted to anything a potential caller might choose.

4 Likes

First, consider this variation, where I've replaced all your Fn() bounds with Display bounds and your closure with 0. problem fails because the signature said that you could support any type ContainerEnd: Display, but then I tried to build a Vec of some specific type. Your code has the same problem, just with a different bound.

If I want to be able to store multiple types in the same Vec this way, I need to type erase the multiple underlying types into a single type like Box<dyn Trait>. This restriction is not closure-specific.


The other thing that the error is talking about is that each closure has a unique, distinct type. So this won't compile for example:

fn f(b: bool) -> impl Fn() {
    let hey = "foo".to_string();
    if b {
        move || println!("{}", hey)
    } else {
        move || println!("{}", hey)
    }
}

As the two closures have different types (even though they have the same captures and body). For this particular example I could do:

fn f(b: bool) -> impl Fn() {
    let hey = "foo".to_string();
    let closure = move || println!("{}", hey);
    if b {
        closure
    } else {
        closure
    }
}

But if you can't do this -- for example, if the closure types in question are input parameters or wrap other closures that are inputs -- then again, you're going to need something like Box<dyn Fn()> to erase the types.

3 Likes

@H2CO3 @quinedot thank you for the explanations. Especially for the example with Display. In the case of the example with Display I know I can use type i32 as returned type and problem solved. But in case of closures, I don't know how to declare what is returned.


impl CommandFormer {
    pub fn problem() -> VectorFormer<_> {
        let on_end = || {
            println!("on_end");
        };
        VectorFormer::<_>::new(on_end)
    }
}

The compiler does not deduce the type. What should be instead of _?

Playground

You can use the impl Trait syntax.

pub fn problem() -> VectorFormer<impl Fn()> {
    let on_end = || {
        println!("on_end");
    };
    VectorFormer::<_>::new(on_end)
}

The compiler never infers types in function signatures.

4 Likes

Thanks, that works!

By the way equivalent does not work:

pub fn problem< F : Fn() >() -> VectorFormer< Fn > {
    let on_end = || {
        println!("on_end");
    };
    VectorFormer::new(on_end)
}

The impl Trait syntax is not equivalent to generics when used as a return value. The fact that it means something different when used as an argument or return value was actually one of the major arguments against introducing the impl Trait syntax, or limiting it to return values only.

6 Likes

It's not the equivalent. A generic argument/parameter type is an input type, just like a function argument is an input to the function. If you try to return a closure using a type that is a generic type, that is like calling a function like sin() and insisting that it always returns e.g. 0. That doesn't make sense – you can't constrain what value a function returns for a specific argument, nobody other than the function itself decides that.

Here, since your function itself defines the closure to be returned, it unambiguously determines its type for itself, and there is no way in the world you can force it to be a different type. Therefore, using a generic type parameter, an input type, doesn't make sense and isn't allowed.

The return-position impl Trait syntax is nothing like generics. It specifies an output at the type level. It says that "this value has a type that promises to implement Trait, but you aren't allowed to know or prescribe what this concrete type is".

3 Likes

Very clear. Thank you, for the explanation @H2CO3. Now I understand this:

In argument position, impl Trait is very similar in semantics to a generic type parameter.
However, there are significant differences between the two in return position.
With impl Trait, unlike with a generic type parameter, the function chooses the return type, and the caller cannot choose the return type.

The function:

fn foo<T: Trait>() -> T {

allows the caller to determine the return type, T, and the function returns that type.

The function:

fn foo() -> impl Trait {

doesn't allow the caller to determine the return type.
Instead, the function chooses the return type, but only promises that it will implement Trait.

1 Like

Yes. Incindentally, this is why I think impl Trait in argument position was a mistake. It is completely different from impl Trait in return position, and that in itself is confusing. I usually recommend avoiding it altogether, and only using this syntax in return position.

5 Likes

Is that different in argument position? In the case of argument position caller neither can't define the type of such argument. Or can? I am still confused.

The caller can indeed choose it when its an argument:

use std::fmt::Debug;

fn print_me(value: impl Debug) {
    println!("{:?}", value);
}

fn main() {
    print_me("foo_bar"); // here it's a &str
    print_me(2384); // here it's an i32
}
1 Like

fn f(_: impl Trait) {} is similar to fn f<T: Trait>(_: T) {}, where it is generic and monomorphizes, in contrast with being concrete and inferred. However,

  • It's opaque (no name), even to the function writer (e.g. they cannot call T::trait_method())
  • Callers can't use turbofish notation: f::<SomeT>()

The latter makes it a breaking change to go from <> to argument position impl Trait.

1 Like

In argument position impl Trait, the caller determines the concrete type. (If neither the caller nor the callee could, then who can?)

1 Like

Related question to clarify the space of possibilities.

Another way of seeing these things, which is mentioned in the impl Trait RFC itself, and which I've found of way better clarity, is to consider any MyTrait and some MyTrait as the syntax used for impl MyTrait, which makes it explicit whether it is universal / generic / caller-chosen or of it is existential / callee-chosen:

fn f<T : Display>(arg: T)

is —modulo turbofish—, equivalent to:

fn f(arg: impl Display)

which, with the syntax I've mentioned, would be written as:

//! Pseudo-code!
fn f(arg: any Display)
  • In all these cases, the caller picks the specific Displayable type, since the function can handle anyone of them:

    f(42_i32); // OK, `i32 : Display`
    f(true); // OK, `bool : Display`
    

Then, similarly, but in return position this time, consider:

fn g<T : Default>() -> T {
    <_>::default()
}
  • With it, the caller picks the return type:

    let default_i32: i32 = g();
    let default_bool: bool = g();
    
    enum NotDefault { A, B }
    let error: NotDefault = g(); // Error, caller can only pick a type that implements `Default`
    

would be:

//! Pseudo-code!
fn g() -> any Default {
    <_>::default()
}

Which is different than:

fn g() -> some Default {
    42_i32 // the type `i32` is indeed `Default`
}

or

fn g() -> some Display {
    42_i32 // ditto
}

both of which are, in practice / actual Rust, written as -> impl …:

fn g() -> impl Display {
    42_i32
}

and which feature the semantics of some single type —about which we only know that it is Displayable—, which is chosen by the callee: g() always returns the same type, no matter how the caller calls it.

With an extra nuance: for the sake of future-proofing the APIs and avoid breakage, when a callee defines a function as returning some Display, they never promise / tell which one it is concretely.

So, in the example:

// fn g() -> some Display {
   fn g() -> impl Display {
       42_i32
   }

a caller cannot rely on the "some Display type" being i32, even if it technically is that type. That is, Rust will hide / encapsulate the return type in an opaque layer that only let's the Display aspect of the concrete return type slip through:

let n: i32 = g(); // Error, expected ("unknown" caller-chosen) `some Display`, got `i32`.
4 Likes

Amazing! Thank you :smiley:

//! Pseudo-code!
fn g() -> any Default {
    <_>::default()

Type depending on value? Kind of second-order typed lambda calculus?

That particular function is nothing new as far as I can tell – it would be equivalent with fn g<T: Default>() -> T.

2 Likes

@Yandros @H2CO3 @quinedot @alice I am writing an article based on the discussion. Is anyone interested giving me feedback on that? Please :slight_smile: