How to clone a boxed closure

[xcaptain]

I want to implement a feature that can evaluate an expression from a string, I come across some problems may relate to borrow checker, please help me.

use std::collections::HashMap;
use rhai::{Engine, FnRegister};

fn add1(a: i64, b: i64) -> i64 {
    return a+b;
}

fn add2(a: i64, b: i64) -> i64 {
    return a+b+1;
}

pub struct P {
    pub fm: HashMap<&'static str, Box<dyn Fn(i64, i64) -> i64>>,
}

impl P {
    pub fn new() -> Self {
        let mut h: HashMap<&'static str, Box<dyn Fn(i64, i64) -> i64>> = HashMap::new();
        h.insert("add1", Box::new(add1));
        h.insert("add2", Box::new(add2));
        return P {
            fm: h,
        };
    }

    pub fn test(&self, expr: &str) -> i64 {
        let mut engine = Engine::new();
        for (key, func) in self.fm {
            engine.register_fn(key, func.clone());
        }
        let res = engine.eval::<i64>(expr);
        match res {
            Ok(result) => {
                return result;
            },
            Err(msg) => {
                println!("failed eval: msg={}", msg);
                return 0;
            }
        }
    }
}

fn main() {
    let expr = "add1(2, 3)";
    let p = P::new();
    p.test(expr);
}

https://github.com/rust-lang/rust/issues/44490 this issue says that rust will copy or clone a closure when need, why in my code the clone method doesn't work? How to clone a boxed closure or any other way to fix my code?

[leudz]

I'm not sure it can be fixed with boxed Fn but it should work with function pointer.
You just have to replace your dyn Fn with fn and since they all have the same size you can remove the Box. They are Copy too so no need for clone.
This will only work if you don't capture the environment however.

As a side note, return is usually reserved for early returns, it's more idiomatic to just have the value without the return keyword.

[xcaptain]

I'm using Box dyn because of https://github.com/rust-lang/rust/issues/28796#issuecomment-481502369 , Box makes a closure have a compile time size and dyn makes the compiler recgonize add1 and add2 as the same type of closure.

[xcaptain]

Thank you, I followed your advice and replaced Fn with fn, now it compiles.

use std::collections::HashMap;
use rhai::{Engine, FnRegister};

fn add1(a: i64, b: i64) -> i64 {
    return a+b;
}

fn add2(a: i64, b: i64) -> i64 {
    return a+b+1;
}

pub struct P {
    pub fm: HashMap<&'static str, fn(i64, i64) -> i64>,
}

impl P {
    pub fn new() -> Self {
        let mut h: HashMap<&'static str, fn(i64, i64) -> i64> = HashMap::new();
        h.insert("add1", add1);
        h.insert("add2", add2);
        return P {
            fm: h,
        };
    }

    pub fn test(&self, expr: &str) -> i64 {
        let mut engine = Engine::new();
        for (key, func) in self.fm.iter() {
            engine.register_fn(key, func.clone());
        }
        let res = engine.eval::<i64>(expr);
        match res {
            Ok(result) => {
                return result;
            },
            Err(msg) => {
                println!("failed eval: msg={}", msg);
                return 0;
            }
        }
    }
}

fn main() {
    let expr = "add1(2, 3)";
    let p = P::new();
    let v = p.test(expr);
    println!("result={}", v);
}

I have never used Function Pointer before, it looks better than closure.

[leudz]

There are a few things I want to clarify.

Your bound issue (the compiler error) was because your closure is not considered Clone anymore once you put it into its Box.
The issue is not that your closure is Clone or not, it to keep that information until the call to clone.
We would like something like Box<dyn Fn(i64, i64) -> i64 + Clone> but it's not possible. If you really want to you can but it's a lot easier with function pointer.

@xcaptain

Box makes a closure have a compile time size

Closures have a known size, you can do:

let add = |x: i64, y: i64| x + y; 

But you can't name the type, each closure has it's own, something like [closure@src/main.rs:5:19: 5:41].
To store it however we need a name, we can use function pointers or trait objects.
If we go with trait objects, we loose the Sized bound and need some kind of pointer.

@xcaptain

it looks better than closure.

Fn-like trait is not equivalent to closures and function pointers to functions.
You can make a Fn-like trait object from a function and a function pointer from a closure (that doesn't capture anything, this is the restriction).

[xcaptain]

Thanks for the detail guide, I wrote a piece of code to test your words.

type TestFn = fn(usize, usize) -> usize;
type Testfn = fn(usize, usize) -> usize;

fn main() {
    let add1: Testfn = |a: usize, b: usize| -> usize {
        a + b
    };
    let add2 = Box::new(add1).clone();

    let v2 = add2(1, 2);
    println!("v2={}", v2);
}

When using add1: TestFn, I got a compile error, says

But when I changed to add1: Testfn, it compiles, and if I remove this type declaration for add1, it compiles. I think when using a closure or function as a parameter, rust will first treat as a function pointer

Also this example shows boxed closure can't be cloned, but boxed function pointer can.

[leudz]

It will first treat it as the real type: [closure@test4.rs:10:24: 12:6]. This may not seem like a type but it is.

The next two errors are because a trait object may not be sized and can't be on the stack as a result.

The last error is the same as before, there is nothing in the type indicating that what is inside the box implements Clone.

This is a working example with trait object based on my previous link.

[Yandros]

By the way, if you choose to use function pointers rather than a trait object dyn FnClone encapsulating both the callable interface (Fn(...) -> _) and the cloneable interface (clone_box), you no longer need to use Box:

impl P {
    pub
    fn new () -> Self
    {
        let mut fm = HashMap::<&'static str, fn(i64, i64) -> i64>::new();
        fm.insert("add1", add1);
        fm.insert("add2", add2);
        Self { fm }
    }

[kornel]

You can also replace Box<dyn Fn()> with Arc<dyn Fn()>, which makes it clone-able. Technically it doesn't clone the closure itself, but can make clones of shared references to the same closure.

[Yandros]

@kornel

You can also replace Box<dyn Fn()> with Arc<dyn Fn()> , which makes it clone-able. Technically it doesn't clone the closure itself, but can make clones of shared references to the same closure.

This is definitely the best solution in practice, since a dyn Fn is something that can be called / held concurrently, and that requires heap-allocation to be held in an owned fashion; hence negating the only "drawbacks" of Arc / Rc (no DerefMut and requires heap allocation).

On the other hand, if you are not using Fn(...) -> _ + Send + Sync + 'static, but just Fn(...) -> _ + 'static (as seems to be your case), you can go and use Rc instead of Arc, since the latter would provide no extra functionality:

use ::std::rc::Rc;

let add1: Rc<dyn Fn(i32, i32) -> i32> = Rc::new(|a, b| a + b);

dbg!(add1.clone()(1, 2));

[xcaptain]

Thanks, Rc is a good way to share ownership but it doesn't work if this closure is passing to another function as a parameter.

use std::rc::Rc;

fn main() {
    let add1: Rc<dyn Fn(i32, i32) -> i32> = Rc::new(|a, b| a + b);

    let res = call_f(add1, 1, 2);
    dbg!(res);
}

fn call_f(f: Fn(i32, i32) -> i32, a: i32, b: i32) -> i32 {
    f.clone()(a, b)
}

This code doesn't compile, because f doesn't have a size at compile time.

In this case a better way is to use fn rather than closure

fn main() {
    let add1: fn(i32, i32) -> i32 = |a, b| a + b;

    let res = call_f(add1, 1, 2);
    dbg!(res);
}

fn call_f(f: fn(i32, i32) -> i32, a: i32, b: i32) -> i32 {
    f(a, b)
}

This code compiles.

[xcaptain]

Digging deeper I have a much generic question, in meta-programming field, we often need to generate a function from a factory method and then pass to a template method, like the code below:

fn main() {
    let p1 = P1::default();
    let f1 = generate_f(p1);
    let res1 = call_f(f1.clone(), 2, 2);
    println!("res1={}", res1);
}

fn call_f(f: Fn(usize, usize)->bool, a: usize, b: usize) -> bool {
    f(a, b)
}

fn generate_f(t: impl T + 'static) -> Box<dyn Fn(usize, usize) -> bool> {
    let cb = move |a: usize, b: usize| -> bool {
        t.add1(a, b) == 5
    };
    return Box::new(cb);
}

pub trait T {
    fn add1(&self, a: usize, b: usize) -> usize;
}

#[derive(Default)]
pub struct P1 {}

impl T for P1 {
    fn add1(&self, a: usize, b: usize) -> usize {
        a + b + 1
    }
}

In this code snippet, the method generate_f is a factory function that will generate a lot of functions, based on the incoming t. And then the generated function is passed to a template function call_f. but this call_f function will only receive a closure but the generated function is a boxed closure, we know a boxed closure can't be cloned, so this code doesn't compile.

A good way is modify the call_f makes it accept a boxed closure, but usually we will just define it accepting a closure, so what should I do in this occasion

[leudz]

You can't take dyn Fn(i32, i32) -> i32 as argument directly because it may not be sized.
This works:

use std::rc::Rc;

fn main() {
    let add1: Rc<dyn Fn(i32, i32) -> i32> = Rc::new(|a, b| a + b);
    let add2 = Rc::clone(&add1);

    let res = call_f(&*add2, 1, 2);
    dbg!(res);
}

fn call_f(f: impl Fn(i32, i32) -> i32, a: i32, b: i32) -> i32 {
    f(a, b)
}

[Yandros]

@xcaptain

fn call_f(f: Fn(usize, usize)->bool, a: usize, b: usize) -> bool {

fn generate_f(t: impl T + 'static) -> Box<dyn Fn(usize, usize) -> bool> {

In the same way that you needed : impl T, you "need" : impl Fn(...) -> _ : Fn is yet another trait, it's just that the Fn trait has sugar support from Rust (the () operator).
But all we are doing above we could do it with your trait T.

[xcaptain]

Thanks, a Rc wrapped closure is actually better than a Boxed closure, because Rc support clone while box not, and a rc closure can be derefered while boxed closure not, is that right?

[leudz]

You can clone a Box, you just have to help the compiler understand what you are doing (link in a previous reply).
You can also deref a Box but you don't have to since Box<dyn Fn> implements Fn itself.

[xcaptain]

fn generate_f(t: impl T + 'static) -> Rc<dyn Fn(usize, usize) -> bool> {
    let cb = move |a: usize, b: usize| -> bool {
        t.add1(a, b) == 5
    };
    return Rc::new(cb);
}

This code solves my problem, the returned closure can be derefed and can be cloned, so it can be passed to call_f.

[Yandros]

Rc / Arc do not require the environment captured by the closure to be cloneable, whereas Box does.

Your callbacks captured no environment, which were thus trivially Cloneable, but because of the type erasure of dyn ... (trait objects), any functionality not described by one of the traits within the trait object is lost.

• You'd need something like Box<dyn Clone + Fn(...) -> _>, but when thinking about it this cannot work for technical reasons (Clone::clone(&self) -> Self is not object-safe (it requires static knowledge of the size of Self, also lost by trait object erasure).

  So we need something like Clone but with an object-safe interface; that's what @leudz FnClone trait was for, and with that trait Box<dyn FnClone> can indeed be Cloned (which is, by the way, only marginally cheaper than Rc<dyn Fn(...) -> _> cloning when no environment is captured, and in that case raw function pointers did the job).

  For an example of a closure capturing environment, take:

  fn add_n (n: i32) -> impl Fn(i32, i32) + Clone + 'static
  {
      move // this closure captures `n: i32`; a function pointer no longer suffices
      |x: i32, y: i32| -> i32 {
          x + y + n
      }
  }
  

---

Cloneable callable: recap

That's why, if you know you won't be capturing environments, fn(...) -> _ function pointers suffice

• fn(...) -> _

but for everything else, if cloning the closure is required (by the way, this does not come as often as a requirement as you may think, since a borrow of a closure is also a closure), you should use:

• Rc<dyn Fn(...) -> _ + 'static> in a single-threaded context,

• Arc<dyn Send + Sync + Fn(...) -> _ + 'static> elsewhere.

[xcaptain]

Now I have a better understanding about this piece of code. By default a boxed closure can not be cloned, so you created a new trait FnClone, it's also a closure, then implement clone trait for Box<dyn FnClone> so the boxed closure can be cloned, it's a very clever, I can never come up with this solution.

[Yandros]

Sadly it requires to be done again and again for each trait object that you wish to be Cloneable, so it is quite unergonomic.

Luckily, there is a crate that helps automating this: ::objekt:

trait FnClone : Fn(i32, i32) -> i32 + ::objekt::Clone {}

::objekt::clone_trait_object!(FnClone);