Passing a list of fn pointers or closures

help
#1

Hi,

I have a function that handles commands on the command line. It gets in its input a (ordered) vector of function pointers, that are handlers for the messages. Then for each message, it tries to let each handler handle the message, until one succeeds. The function something like:

pub type Handler = fn(&str, &Vec<&str>) -> Option<i32>;

pub fn run_cli(handlers: Vec<Handler>) -> i32 {
    let mut count: i32 = 0;
    let stdin = io::stdin();

    loop {
        print!("[{}]: ", count);
        io::stdout().flush().unwrap();

        let cmd = stdin.lock().lines().next().unwrap().unwrap();
        let mut cmd_iter = cmd.split_ascii_whitespace();
        if let Some(cmd) = cmd_iter.next() {
            match cmd {
                "exit" => {
                    println!("Done!");
                    return 0;
                }
                _ => {
                    let args: Vec<&str> = cmd_iter.collect();
                    for handler in &handlers {
                        if let Some(_) = handler(cmd, &args) {
                            break;
                        }
                    }
                }
            }
        }

        count = count + 1;
    }
}

This works fine if I have handlers that don't need any additional input. But now I have to write a handler that needs to get access to some external state. So I thought I could define it using curryng and closures. But a closure is not a function pointer.

Is there a way to either pass a list closures, or convert a closure to a fn pointer, or is there another idiomatic way to do this in Rust?

#2 
pub type Handler =
    Box<dyn FnMut(&str, &Vec<&str>) -> Option<i32>>
;

would be the usual solution. This way the captured state, if any, gets heap-allocated, which introduces the necessary indirection that makes merging closures of different captured states (or capture-less closures, such as function items) be merged as a common type.

2 Likes
#3

Every closure (or combination of closures) will have a distinct type. So you can’t put them in a traditional collection without dyn. If the handlers all implement the same trait, though, you can also implement that trait for combinator types.

pub trait Handler<In,Out> {
    fn handle(&mut self, _:&In)->Option<Out>;
}

impl<I,O,F:FnMut(&I)->Option<O>> Handler<I,O> for F {
    fn handle(&mut self, i:&I)->Option<O> { (self)(i) }
}

impl<In,Out,A,B,> Handler<In,Out> for (A,B,)
where A:Handler<In,Out>,
      B:Handler<In,Out>,
{
    fn handle(&mut self, i:&In)->Option<Out> {
        let (
             ref mut a,
             ref mut b,
        ) = self;
        None
            .or_else(|| a.handle(i))
            .or_else(|| b.handle(i))
    }
}

These can be nested to an arbitrary depth and generally treated like a lisp-style cons list. This can avoid the heap allocations necessary for dynamic dispatch, but is much less ergonomic: In particular, you have to use bounded recursion to build these values instead of a loop.

(The style of this code is a little odd because it’s the template for a macro for arbitrary-length tuples.)

#4

It's also worth noting that Box<dyn FnMut> does not cause a heap allocation when it points to a fn or a captureless closure. So this adds allocation overhead only for the items that actually need it.

7 Likes
#5

How does it not allocate? Is this just a special case?

#6

It's because whenever you make a closure, it gets its own unique type, and this type has size zero when the closure doesn't capture any data. Allocations of size zero don't allocate.

The same applies to when you mention a function by name. That gives you a zero-sized type known as a function item.

5 Likes
#7

Hence the if any :wink:, although that way of phrasing that was maybe a bit too subtle :smile:

To be very exact, when Boxing a function item (or a stateless closure), given their lack of runtime information (it is all encoded at the type level, similar struct _1; impl HasValue for _1 { const VALUE: usize = 1; } struct _2; impl HasValue for _2 { const VALUE: usize = 2; }, whereby _1 and _2 are zero-sized types each, that yet contain their own information encoded within their types. That's also the reason you can't put in the same collection both _1 and _2, since they are of different types), there is nothing / no information to put in the heap (in practice, an allocator has no API for zero-sized allocations), so a dummy pointer is forged to occupy the "heap pointer" slot, since it will never be dereferenced.

Then, when performing the dyn Methods pointer-fattening coercion, a second pointer is attached to the first one (in both the zero-sized and "normal" cases), which contains, mainly, an fn call_mut (&mut self, &str, &Vec<&str>) -> Option<i32> (virtual method for the call_mut part of the FnMut API), and an fn drop_in_place (&mut self), to run the drop glue when the boxed item is deallocated (even if fn items and stateless closures don't have such, a stateful closure may be capturing something that does, such as a String, so it needs to be deallocated when the Box is).

So, the only performance difference between an fn… pointer and a Box<dyn FnMut…> are:

  • an extra level of pointer indirection to go from an &mut (dyn FnMut...) to its fn... virtual method;

  • a virtual call to a no-op drop_in_place function.

Both of these are of negligible cost.

And in exchange, we get the option to capture stateful closures too (although in this case an actual heap allocation will be happening).

In C parlance, the basic fn pointer would match a function pointer with no void * state, whereas an &mut (dyn FnMut…) would match a void * data, ret_ty (*)(void * data, …) pair.

#8

So I almost got this to work. My only issue is how do I get the function calls to happen. In the line:

                        if let Some(_) = handler(cmd, &args) {

I am getting an error:

error[E0596]: cannot borrow `*handler` as mutable, as it is behind a `&` reference
  --> src\cli_handler.rs:48:42
   |
47 |                     for handler in &handlers {
   |                                    --------- this iterator yields `&` references
48 |                         if let Some(_) = handler(cmd, &args) {
   |                                          ^^^^^^^ `handler` is a `&` reference, so the data it refers to cannot be borrowed as mutable

I would of thought that calling *handler(cmd, &args) would do the trick, but it does not. How do I unbox the fn pointer?

#9

An FnMut requires mutable access to call it. Either borrow it mutably with for handler in &mut handlers or replace FnMut with Fn.

1 Like
#10

Oh yes, I forgot to write about that: given your use case (handle in &handlers), I could have suggested you use a Box<dyn Fn(…) -> _>, but given that you are not requiring to poll the iterator concurrently (you are doing it sequentially), then FnMut() is a less restrictive bound to use, provided you use &mut handlers as your iterable :wink: