Problem using Arc with a closure parameter

help
#1

Hi there,

I've been troubling over a struct that provides a new function with a param that declares a function to do something. Here's my declarations:

pub struct MyStruct {
    pub my_fn: Arc<
        Box<dyn Fn() -> u32>
    >,
}

impl MyStruct {
    pub fn new<F>(my_fn: Arc<Box<F>>) -> MyStruct
    where
        F: Fn() -> MyStruct,
    {
        let _ = my_fn(); // Pretend this does something useful
        MyStruct {
            my_fn,
        }
    }
}

The compiler complains:

   |
10 |     pub fn new<F>(my_fn: Arc<Box<F>>) -> MyStruct
   |                - this type parameter
...
16 |             my_fn,
   |             ^^^^^ expected trait object `dyn Fn`, found type parameter `F`
   |
   = note: expected struct `Arc<Box<(dyn Fn() -> u32 + 'static)>>`
              found struct `Arc<Box<F>>`
   = help: type parameters must be constrained to match other types
   = note: for more information, visit https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters

If I remove Arc from the above, then all is well. I suspect that the problem isn't anything to do with the problem being reported...

Here's the Rust Playground version: an example.

Any pointers? Thanks.

#2

Arc can hold trait objects (dyn) like box. So you can get rid of the box layer.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=767669942ce0be84c4da721c92bce131

Might need some more to turn a generic into a dyn.

2 Likes
#3

To add some explaination, you are trying to coerce an Arc<Box<TypeThatImplsTrait>> into an Arc<Box<dyn Trait>>, which would be a type of unsizing coercion. They work through one layer of pointers but not two, which is why removing either Arc or Box fixes it.

Here's an example using simpler types and bounds.

1 Like
#4

Converting Box<impl Trait> into Box<dyn Trait> is like converting i32 into i64 - both add up extra runtime information and widen the size of the type. You can't convert Box<i32> into Box<i64> or Arc<i32> into Arc<i64> without allocation as they're different even in size.

3 Likes
#5

Thanks. Ok, so I can omit the Box when using Arc... I'm sure there are good reasons, but first... a simplified version is requiring my function to be static:

pub struct MyStruct {
    pub my_fn: Arc<dyn Fn() -> u32>,
}

impl MyStruct {
    pub fn new<F>(my_fn: Arc<F>) -> MyStruct
    where
        F: Fn() -> u32,
    {
        MyStruct {
            my_fn,
        }
    }
}

...now saying that F needs to live longer... My original thinking around the box is that the closure lives on the heap, so it should be fine until its box goes out of scope... hence having an Arc on a Box in the first place... Thanks for further help.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f06760d1e89e0ad16c27a1805513cbc5

#6

Perhaps I should also explain my goal: I've got a struct that can be shared between threads and I want to share a closure that can be invoked multiple times. That closure has no mutable state.

#7

The 'static binding says that the closure will either have no references, or only 'static references. If you have some non static data that needs to be shared by independent closures you can likely use an Arc instead of a reference for that data.

So you probably just need to to change

F: Fn() -> u32,

to

F: Fn() -> u32 + 'static,

It's very unlikely you need and you should definitely try to avoid a non 'static closure for this type of design.

If you want to dive deeper you can read

2 Likes
#8

Thanks again for the amazing answers. I really appreciate them.

So, with the static lifetime declared on F, if I now introduce a spawn method, then the size isn't known at compile time... Here's the code:

use std::sync::Arc;

pub struct MyStruct {
    pub my_fn: Arc<dyn Fn() -> u32>,
}

impl MyStruct {
    pub fn new<F>(my_fn: Arc<F>) -> MyStruct
    where
        F: Fn() -> u32 + 'static,
    {
        let _ = my_fn();
        MyStruct {
            my_fn,
        }
    }
    
    pub fn spawn(&mut self) -> MyStruct {
        MyStruct::new(self.my_fn)
    }
}

fn main() {
    let a = MyStruct::new(Arc::new(|| 0));
    let _ = a.spawn();
}

Again, my goal is to share a function that produces new MyStructs. Am I just doing this all wrongly?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0545da8e04f41d50b3a4a1208ba5baf5

#9

Two solutions (could do both) depends on if there's a reason to have the Arc as part of the external API.

I kind of guessed what you wanted the spawn to do.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c65ea8dcfed3a5a93261d8963a4e8292

1 Like
#10

Yeah, I want to share the memory that contains the function passed in, so Arc needs to be exposed.

What you've done in essence, is to remove the where clause that I had and replaced the closure declaration inline:

    pub fn new(my_fn: Arc<dyn Fn() -> u32>) -> MyStruct {
        let _ = my_fn();
        MyStruct {
            my_fn,
        }
    }

...which works... but why? I thought I could use the where clause to make my function declaration clearer... but there's something going on here that I don't understand.

Am I still able to use the where clause? Rust Playground

#11

dyn Fn() is not Sized, but generic type parameters have implicit : Sized trait requirement. If not, it can't take a argument with those types since variables including the function parameters requires statically known size. You can opt-out this requirement with : ?Sized pseudo-requirement.

https://doc.rust-lang.org/stable/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait

#12

Thanks. I'm beginning to get this! However, something is missing still:

use std::sync::Arc;

pub struct MyStruct {
    pub my_fn: Arc<dyn Fn() -> u32>,
}

impl MyStruct {
    pub fn new<F>(my_fn: Arc<F>) -> MyStruct 
    where
        F : Fn() -> u32 + ?Sized,
    {
        let _ = my_fn();
        MyStruct {
            my_fn,
        }
    }
    
    pub fn spawn(&mut self) -> MyStruct {
        MyStruct::new(self.my_fn.clone())
    }
}

fn main() {
    let mut a = MyStruct::new(Arc::new(|| 0));
    let _ = a.spawn();
}

error[E0277]: the size for values of type `F` cannot be known at compilation time
  --> src/main.rs:14:13
   |
8  |     pub fn new<F>(my_fn: Arc<F>) -> MyStruct 
   |                - this type parameter needs to be `Sized`
...
14 |             my_fn,
   |             ^^^^^ doesn't have a size known at compile-time
   |
   = note: required for the cast to the object type `dyn Fn() -> u32`

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c1917830e7f512fb42f5549d7500065a

#13

I feel that I want to express something like this:

    pub fn new<F>(my_fn: Arc<dyn F>) -> MyStruct

...but this is disallowed.

#14

What is also disallowed, and remains a mystery to me is why the following works:

    pub fn new<F>(
        my_fn: F,
    ) -> MyStruct
    where
        F: FnOnce() -> u32

...but this doesn't:

    pub fn new(
        my_fn: FnOnce() -> u32,
    ) -> MyStruct

I get that a pointer to the function is required in the second scenario, but how comes it works in the first?

#15

The where clause is there to constrain the types of generics to have certain traits. Not to clarify non generic types.

/// A trait (which is not a type)
Trait Example {}

/// A generic function, so a new copy of that function will be made for every F that it's a called 
/// for because Arc<A> and Arc<B> are different types.
fn test<F>(t: Arc<F>) where F: Example {} 

/// A "trait object" which is a single type, that is unsized so is nearly always behind a pointer (fat pointer to be percise) of some sort (Arc, Box, &, &mut)
dyn Example

/// Also a single type.
Arc<dyn Example>

/// This will normally return an  Arc<A> but can return an Arc<dyn Example>, assuming A: Example if that's
/// the expected type inferred when it's called,
/// like how 1 can be a u8,i8,u16,i16... or how collect() can return a Vec, HashMap or even Result<Vec<_>,_>.
Arc::new(A)

Since you want one type to hold any Fn() -> u32, it should be stored in a form of trait object. And since multiple independent things to access it it then an Arc<dyn Fn() -> u32> is probably most appropriate trait object. If the caller needs to keep access to the closure then the Arc needs to be exposed at the API so the caller can keep a clone of it.

In that case the caller needs to create the Arc<dyn Fn() -> 32> which is a single type so there's nothing to be generic over. They cannot pass in Arc since that's a different type and the only way to convert that to Arc<dyn Fn() -> u32> is make a new Arc to hold it an either clone or take the closure out of the first Arc (which is only possible with a single owner, at that point why use an Arc). A confusing part is probably that Arc::new can return either type depending on what's expected where it's called.

To be pendantic
You could have a generic function that built and returned the Arc<dyn Fn() -> u32>, but it's probably not the best API.

Also are you sure the caller will need a copy of the closure, I suspect you're confusing how data is typically shared. Between two different closures sharing some value, both would have their own clone of an Arc for just that value, but they don't themselves need to be stored in an Arc.

2 Likes
#16

FnOnce is not a pointer it is a trait. So any closure, function or even custom struct that implements that trait (the appropriate closures and functions will automatically). Will work in

    pub fn new<F>(
        my_fn: F,
    ) -> MyStruct
    where
        F: FnOnce() -> u32

As for

 pub fn new(
        my_fn: FnOnce() -> u32,
    ) -> MyStruct

It doesn't compile because "FnOnce() -> u32" is not a type. A function pointer would be fn() -> u32 (not capitalized), which pretty much is a pointer but is very limited and mostly used for C FFI. A closure can hold data + code and each closure has a unique type.

1 Like
#17

Fantastic answers! Thanks! To answer the above, I want a function that produces a certain structure. That function will get called again and again, but it will always produce a new instances of my required structure. It is a factory function and in real-world usage, it will produce a tuple of Sender and Receiver from either unbounded or bounded channels.

#18

Thanks. Yes, I understand the difference. However, it seems odd to me (being relatively new to Rust - obviously!), that both of those function declarations sort of read the same to me. My first impression here is that I'm expanding out F in a where clause because it is tidier to do so. But both declarations are indeed different, and I'm still not sure why. I'm sure I'm missing something important here, but hopefully you understand where I'm tripping up.

#19

Note that, together with the hard error, you're expected to see the warning like the following:

warning: trait objects without an explicit `dyn` are deprecated
 --> src/lib.rs:1:11
  |
1 | fn new(_: FnOnce() -> u32) {}
  |           ^^^^^^^^^^^^^^^ help: use `dyn`: `dyn FnOnce() -> u32`
  |
  = note: `#[warn(bare_trait_objects)]` on by default

This can be seen as a hint on the next step, which will probably lead to this chapter of the Rust book, describing the problem in more detail.

#20

The where clause only constrains generics (so requiring particular traits on an unknown type). It's generally cleaner than putting the constraints on the declaration of the generic type

so

pub fn new<F: FnOnce() -> u32>(my_fn: F) -> MyStruct

is the same as

pub fn new<F>(my_fn: F) -> MyStruct
where 
  F: FnOnce() -> u32

Which is easier to read with lot's of generic types. So it's not a way to give type a nicer name, that would be "type"

type Callback = Arc<dyn Fn() -> i32>;

pub fn new<F>(my_fn: Callback) -> MyStruct {}
1 Like