Option<Fn> fails to compile

help
1

I'm trying to make a callback as a parameter, and make it Option-al. I'm probably abusing the syntax or misunderstanding the type of do_skip_node, but is it possible to make such a function with a default fallback?

struct Node {}

struct MyParser {
	skip_nodes: bool
}

fn do_skip_node(n: Node) {}

impl MyParser {
    // this compiles
	fn works<NC>(&mut self, node_cb: NC) where NC: Fn(Node) {}

    // this does not compile
	fn fails<NC>(&mut self, node_cb: Option<NC>)
	where NC: Fn(Node)
	{
		let ncb = if let Some(n) = node_cb {
			self.skip_nodes = false;
			n
		} else {
			self.skip_nodes = true;
			do_skip_node  // or |n: Node| {}
		};
	}
}

Here's the error message. Do I understand it correctly that the upper part of the if returns a type, not the actual fn instance?

11 |       fn fails<NC>(&mut self, node_cb: Option<NC>)
   |                -- this type parameter
...
14 |           let ncb = if let Some(n) = node_cb {
   |  ___________________-
15 | |             self.skip_nodes = false;
16 | |             n
   | |             - expected because of this
17 | |         } else {
18 | |             self.skip_nodes = true;
19 | |             do_skip_node  // or |n: Node| {}
   | |             ^^^^^^^^^^^^ expected type parameter `NC`, found fn item
20 | |         };
   | |_________- `if` and `else` have incompatible types
   |
   = note: expected type `NC`
           found fn item `fn(Node) {do_skip_node}`
2

The tldr of the situation is that each function is its own type. Fn(Node) represents a trait, not a concrete type.

2 Likes
3

It returns a type, but it may not be a function, it may be, you know, callback (as you wanted).

If you would use Option<fn(Node)> instead of Option<FN> then your code would work, of course, but then you wouldn't be able to pass around a closure, of course.

Types of different closures are different because they may grad arbitrary amount of captured data.

If you want to work with different closures would be to move them to heap and use Box<dyn Fn(Node)>

1 Like
4

No. Functions with the same arguments denote the same types. Closures are different, for obvious reasons.

2 Likes
5

No. Expressions don't yield types, they always yield values. (Expressions have a type, which is the type of the value they yield.)

Your problem is that you are misunderstanding what generics are. A generic function means that the caller gets to choose the type. However, in this case, you can't let the caller choose the type if the else arm is hit: the type of fn item do_skip_node is a concrete type, not the same as the (arbitrary) type parameter the user might have substituted for NC.

If the callback you passed in was always coercible to a function pointer, then you could return fn(Node) instead. (You will need some sort of indirection/dynamism, because you allow returning two, potentially different functions.)

However, if the callbacks you pass are expected to be arbitrary callables, and you want to keep the NC: Fn(Node) signature, then you'll have to return e.g. a Box<dyn Fn(Node)>, which can accommodate both fn items and closures.

5 Likes
6

No, the items fn foo() {} and fn bar() {} have distinct types. (They both coerce to fn(), though.)

13 Likes
7

That's technically correct, but I wonder whether this difference is ever possible to observe in practice. I believe the coercions will be automatically applied in a search for a common type if you try to return different types from different branches. It does show up in Debug output, of course.

8

So the answer to forcing it to error is to suppress coercions. A coercion from a function ZST type to fn() requires covariance, so in theory we can suppress it by forcing a type parameter to be invariant, e.g.

pub struct Invariant<T>(T, PhantomData<fn(T) -> T>);

fn a() {}
fn b() {}

pub fn oops() -> impl Sized {
    if true {
        Invariant(a, PhantomData)
    } else {
        Invariant(b, PhantomData)
    }
}

This is tricky to hit even when trying to, though; returning Invariant<fn()> will back-propagate and coerce the values while they're still variant. I'm reasonably confident in saying accidentally hitting this unintentionally is unreasonably unlikely so long as it's impossible to name these types.

4 Likes
9

Invariance doesn't seem to be necessary, preventing the coercion appears to be as simple as wrapping with any type, e.g. this errors:

fn foo() {}
fn bar() {}

fn main() {
    let _x = if true { Some(foo) } else { Some(bar) };
}
8 Likes
10

A brief follow-up: I couldn't get it to work with Box (Option<Box<dyn Fn ...>>). Another try was to do it this way:

	pub fn process<N, W, R>(&mut self,
		node_cb: Option<N>, way_cb: Option<W>, relation_cb: Option<R>) -> Result<(), Box<dyn Error>>
		where N: FnMut(Node) -> OkOrBox + Copy,
		W: FnMut(Way) -> OkOrBox + Copy,
		R: FnMut(Relation) -> OkOrBox + Copy {
		loop {
			let res = self._next()?;
			match res {
				Some(OsmObj::Node(n)) if !self.skip_nodes => node_cb.unwrap()(n)?,
				Some(OsmObj::Way(w)) if !self.skip_ways => way_cb.unwrap()(w)?,
				Some(OsmObj::Relation(r)) if !self.skip_relations => relation_cb.unwrap()(r)?,
				None => return Ok(()),
				_ => {}
			}
		}

This way I couldn't get all the necessary traits of functions either.

So, the final solution was to make a function for each type of object (it will process the entire file every time, but that's acceptable. If I need all 3, I can resort to a large match block, which I tried to avoid with this callback passing).

	pub fn map_nodes<F>(&mut self, mut cb: F) -> OkOrBox
	where F: FnMut(Node) -> OkOrBox {
		self.skip_ways = true;
		self.skip_relations = true;
		for res in self.into_iter() {
			if let OsmObj::Node(n) = res? { cb(n)? }
		}
		Ok(())
	}
11

Specifically how? I don't think Box should affect anything here.

What does it mean to "get the traits of functions"?

12

Here are three options that compile based on your code above

Playground

#![allow(dead_code)]

struct Node;

fn do_skip_node(_: Node) {}

struct MyParser {
    skip_nodes: bool,
}

impl MyParser {
    fn dyn_ref<NC>(&mut self, node_cb: Option<NC>)
    where
        NC: Fn(Node),
    {
        let ncb: &dyn Fn(Node) = if let Some(n) = node_cb.as_ref() {
            self.skip_nodes = false;
            n
        } else {
            self.skip_nodes = true;
            &do_skip_node
        };

        ncb(Node)
    }

    fn boxed<NC>(&mut self, node_cb: Option<NC>)
    where
        NC: Fn(Node),
    {
        let ncb: Box<dyn Fn(Node)> = if let Some(n) = node_cb {
            self.skip_nodes = false;
            Box::new(n)
        } else {
            self.skip_nodes = true;
            Box::new(do_skip_node)
        };

        ncb(Node)
    }

    fn enumeration<NC>(&mut self, node_cb: Option<NC>)
    where
        NC: Fn(Node),
    {
        enum Callback<A, B> {
            Provided(A),
            Fallback(B),
        }

        impl<A, B> Callback<A, B>
        where
            A: Fn(Node),
            B: Fn(Node),
        {
            fn call(&self, node: Node) {
                use Callback::*;

                match self {
                    Provided(p) => p(node),
                    Fallback(f) => f(node),
                }
            }
        }

        let ncb = if let Some(n) = node_cb {
            self.skip_nodes = false;
            Callback::Provided(n)
        } else {
            self.skip_nodes = true;
            Callback::Fallback(do_skip_node)
        };

        ncb.call(Node)
    }
}
3 Likes
13

I haven't read the whole thread, but I think something like the following might also solve the problem, and no dyn is needed. Not sure if @semicoleon's approach using &dyn is faster.

struct Node;

struct MyParser {
	skip_nodes: bool
}

impl MyParser {
	fn works<NC>(&mut self, node_cb: NC) where NC: Fn(Node) {}

	fn fails<NC>(&mut self, node_cb: Option<NC>)
	where NC: Fn(Node)
	{
	    if node_cb.is_some() {
			self.skip_nodes = false;
        } else {	        
			self.skip_nodes = true;
	    }
		let ncb = |x| if let Some(n) = node_cb.as_ref() {
			n(x)
		};
		ncb(Node);
		ncb(Node);
	}
}

(Playground)

1 Like
14

I meant "get the traits of functions together", in other words the compiler always pointed at some trait missing.

I tried another time the approach. It seems to work, just as the solutions by @semicoleon and @jbe. But as soon as I try to pass None instead of a callback, the compiler never gets happy.

Here's the source code that works:

    // inside the struct
	pub fn map_all<F1, F2, F3>(&mut self, mut node_cb: Option<F1>, mut way_cb: Option<F2>, mut rel_cb: Option<F3>) -> OkOrBox
		where
			F1: FnMut(Node) -> OkOrBox,
			F2: FnMut(Way) -> OkOrBox,
			F3: FnMut(Relation) -> OkOrBox
	{
		self.skip_nodes = node_cb.is_none();
		self.skip_ways = way_cb.is_none();
		self.skip_relations = rel_cb.is_none();

		for item in self.into_iter() {
			match item? {
				OsmObj::Node(n) => if let Some(ref mut ncb) = node_cb { ncb(n)?; },
				OsmObj::Way(w) => if let Some(ref mut wcb) = way_cb { wcb(w)?; },
				OsmObj::Relation(r) =>  if let Some(ref mut rcb) = rel_cb { rcb(r)?; },
			}
		}
		Ok(())
	}

...

// usage:

	let mut nodes_count:i32 = 0;
	let mut ways_count:i32 = 0;
	let mut rels_count:i32 = 0;

	let ncb = |_n: Node| -> OkOrBox { nodes_count +=1; Ok(()) };
	let wcb = |_w: Way| -> OkOrBox{ ways_count +=1; Ok(()) };
	let rcb = |_r: Relation| -> OkOrBox { rels_count +=1; Ok(()) };

	let mut my_rdr = OsmXmlReader::from_path(&osm_xml_path)?;
	my_rdr.map_all(Some(ncb), Some(wcb), Some(rcb))?;

This works fine. But if you try passing a None, the compiler starts giving more and more requirements: first, "I need type annotations", then when I make them:

my_rdr.map_all(Some(ncb), Some(wcb), None as Option<dyn FnMut(Relation) -> OkOrBox>)?;

it tells the size can't be known. Then I make the parameters Option<Box<dyn FnMut...>> and it wants the generics to be 'static. I put static, it wants ?Sized, but then doesn't allow ?Sized in the return type.

The version with applied changes that still doesn't work is in this commit.

Of course, I can make a stub function that does nothing, and this whole issue goes away (and tell via skip_X flags that I want to skip those variants of OsmObj).

I tried to run Option<FnMut...> in a background thread (replace into_iter with in_background), and it gave the same kind of errors. As I see, passing generic callbacks with loose types in threads is too hard (or impossible because of mutability and data safety).

15

You shouldn't try to annotate with dyn FnMut directly, because it's unsized. The Some(_)s you are passing aren't containing the trait objects directly, either – they are boxed.

Accordingly, if you annotate with &mut dyn FnMut or even more simply with fn(_) -> _, it compiles as expected.

4 Likes
16

Oh, this makes sense, thanks!

Did you know it from somewhere or figured it out? What should I read to understand functions types better?

17

I previously suggested an approach involving an uninhabited closure type, and a helper function which can make the call both shorter and arguably more efficient (since the Option<…> becomes zero-sized).

Applied to this case, and updated to work with edition 2021, it looks something like

#[inline]
pub fn nop<T>() -> Option<impl FnMut(T) -> Result<(), Box<dyn Error>>> {
    enum Void {}
    None::<Void>.map(|void| move |_: T| {
        let _ = &void; // edition 2021 closure capture workaround
        match void {}
    })
}

fn main() -> Result<(), Box<dyn Error>> {
    let ncb = |_n: Node| -> Result<(), Box<dyn Error>> { Ok(()) };
    let wcb = |_w: Way| -> Result<(), Box<dyn Error>> { Ok(()) };
    let rcb = |_r: Relation| -> Result<(), Box<dyn Error>> { Ok(()) };

    map_all(Some(ncb), None::<fn(_) -> _>, Some(rcb))?;
    map_all(Some(ncb), nop(), Some(rcb))?; // shorter to call, and the type is smaller :-)

    Ok(())
}

Rust Playground

Depending on the use-case, i.e. on how these closures are used, in case that None is merely handled by doing nothing whilst the Some case has the closure called, you might also be able to avoid the need for Option entirely, instead just expecting the user to pass some no-op closure (possibly aided by a similar convenience function if that’s considered nicer than having to write the closure manually).

If the choice between Some vs. None does change behavior in different ways, this would not work of course.

1 Like
18

This can be inferred from the fact that you are passing the Option by-value, so whatever it contains by-value must also be sized. (I.e., this is not specific to function types.)

1 Like
19

Oh, I thought of keeping stub functions, but this is even better! Thanks a lot!

20

I meant the None::<fn(_) -> _> syntax for Option<fn...>. There are 2 new things I saw here:

  • fn... inside turbofish. When it comes with collect, the full type must be written (collect::<Vec<item type>>())
  • the underscore... trick? I saw underscore used like "guess it" in lifetimes context. But not here.