Dyn FnOnce without allocating

[mattfbacon]

How can I go from impl FnOnce to dyn FnOnce in a way that allows me to call the function without moving it to the heap? I am trying to write code with generic callbacks that need to be FnOnce for API ergonomics, but I don’t want a copy of the implementation for every callback, so I want to use dyn, but I don’t want to have to make an unnecessary heap allocation every time this function is called just to be able to call the dyn FnOnce.

One solution I thought about is impl FnMut for Option<impl FnOnce> (through a wrapper struct) which would take the function out and panic on None but 1 I can’t implement it in stable Rust and 2 it can’t be unsized due to the Option.

I am mostly annoyed because I don’t intuitively recognise any reason why this shouldn’t be possible but there is some strange combination of language limitations that is not alowying this to work.

[mattfbacon]

My desires for a solution here, in order of importance:

1. No allocation
2. No runtime tracking / no possible panics, e.g. with Option as described. A good API would use self to prevent this.
3. Clean API

[nerditation]

each closure implementing FnOnce has different size, if you don't want to box it, then you must use some wrapper with a compile time known maximum size, one example of such solution is no_alloc:

GitHub - rjsberry/no_alloc: Static memory allocators for applications without...

or, depending on your use case, if the callback doesn't need private data, you can just use a regular function pointer instead of FnOnce.

EDIT:

never mind, my brain is lagging severly like my internet today. I was thinking FnMut all the way, but you actually need FnOnce. in that case, I don't think no_alloc can work, unfortunately.

both FnOnce and Box in the standard library has magic power that other libraries don't have. I don't know any solution for unboxed type-erasure of FnOnce.

[RustyYato]

You’re on the right track, you could wrap the function in a closure like so.

fn generic<F: FnOnce()>(f: F) {
    let mut g = Some(f);
    let f: &mut dyn FnMut() = &mut || (g.take().unwrap())();
    // option 2: You could also wrap `&mut dyn FnMut` in a wrapper struct which consumes `self` when calling `&mut dyn FnMut()`
    special(f)
}

// fn special(f: impl FnOnce()) { // option 3, this way you enforce that &mut dyn FnMut() is only called once.
fn special(f: &mut dyn FnMut()) {
    f();
}

If you ignore the comments, then you will just need to ensure that you don’t call &mut dyn FnMut() multiple times. But that shouldn’t be too hard. Otherwise you can choose one of the options to help ensure that it’s only called once.

This is a fairly common pattern to convert a impl FnOnce() to dyn … to reduce the monomorphization costs.

[akrauze]

Can you please share some snippet of your ideal API usage? I find your question very open-ended. Having some concrete example would help with providing better suggestions.

[quinedot]

This is an aside, since it doesn't address your goals, but in case it's useful.

@mattfbacon

One solution I thought about is impl FnMut for Option<impl FnOnce> (through a wrapper struct) which would take the function out and panic on None but 1 I can’t implement it in stable Rust

You can't implement it at all due to the orphan rules, and std can't implement it without a SemVer breaking change because the FnMut trait is fundamental.

But you can do this.

[mattfbacon]

I don’t understand your point. The closure is already on the stack, why would I need to care about its size?

[mattfbacon]

That’s why I said “through a wrapper struct”, but thanks for the playground demo which helpfully expands on RustyYato’s suggestion with a more generic solutio.

[mattfbacon]

Thanks, I think this is it. I will look into the wrapper struct. I think in that case I can reach all three of my goals by replacing Option with MaybeUninit.

[mattfbacon]

I got this, tested with miri:

use std::mem::MaybeUninit;

struct CallOnce<'a, A, R>(&'a mut (dyn FnMut(A) -> R + 'a));

impl<A, R> CallOnce<'_, A, R> {
    fn call(self, arg: A) -> R {
        (self.0)(arg)
    }
}

fn foo(x: impl FnOnce(u32) -> u32) {
    let x_cont = MaybeUninit::new(x);
    let mut wrapper = move |arg| (unsafe { x_cont.assume_init_read() })(arg);
    foo_impl(CallOnce(&mut wrapper));
}

fn foo_impl(x: CallOnce<'_, u32, u32>) {
    dbg!(x.call(3));
}

fn main() {
    foo(|x| x + 2);
}

I would like to make it more generic with some kind of CallOnceContainer that allows the following type of usage:

// func: impl FnOnce(A) -> R
let mut func_cont = CallOnceContainer::new(func);
let func_dyn: CallOnce<'_, A, R> = func_cont.to_dyn();
foo_impl(func_dyn);

to_dyn would be an &mut self method with some kind of lifetime trickery to make it borrow func_cont “permanently”. Does anyone know how to do that?

[ProgramCrafter]

@mattfbacon

I would like to make it more generic with some kind of CallOnceContainer that allows the following type of usage:

// func: impl FnOnce(A) -> R
let mut func_cont = CallOnceContainer::new(func);
let func_dyn: CallOnce<'_, A, R> = func_cont.to_dyn();
foo_impl(func_dyn);

to_dyn would be an &mut self method with some kind of lifetime trickery to make it borrow func_cont “permanently”. Does anyone know how to do that?

It cannot borrow func_cont for ’static because the variable certainly does not live that long, nor can it work on &mut self as described as problematic code would be accepted:

let mut func_cont = CallOnceContainer::new(func);
{
    let func_dyn = func_cont.to_dyn();
    foo_impl(func_dyn);
}  // any borrow by `.to_dyn()` ended at this point
{
    foo_impl(func_cont.to_dyn());  // we can borrow again
}
{
    foo_impl(func_cont.to_dyn());  // and a third time if we want
}

[mattfbacon]

I found it, there is this pattern with a brand lifetime. So here is the final solution:

use std::marker::PhantomData;
use std::mem::MaybeUninit;

struct CallOnceContainer<'brand, A, R, T> {
    f: T,
    // Makes the type invariant over `'brand`.
    brand: PhantomData<fn(&'brand mut A) -> &'brand mut R>,
}

fn wrap<'brand, 'closure, A, R>(
    f: impl FnOnce(A) -> R + 'closure,
) -> CallOnceContainer<'brand, A, R, impl FnMut(A) -> R + 'closure>
where
    'closure: 'brand,
{
    let cell = MaybeUninit::new(f);
    let wrapper = move |arg| (unsafe { cell.assume_init_read() })(arg);
    CallOnceContainer {
        f: wrapper,
        brand: PhantomData,
    }
}

impl<'brand, A, R, T> CallOnceContainer<'brand, A, R, T>
where
    T: FnMut(A) -> R,
{
    fn to_dyn(&'brand mut self) -> CallOnce<'brand, A, R> {
        CallOnce(&mut self.f)
    }
}

struct CallOnce<'a, A, R>(&'a mut (dyn FnMut(A) -> R + 'a));

impl<A, R> CallOnce<'_, A, R> {
    fn call(self, arg: A) -> R {
        (self.0)(arg)
    }
}

macro_rules! dyn_fnonce {
    ($f:expr) => {
        wrap($f).to_dyn()
    };
}

fn foo(x: impl FnOnce(u32) -> u32) {
    foo_impl(dyn_fnonce!(x));
}

fn foo_impl(x: CallOnce<'_, u32, u32>) {
    dbg!(x.call(3));
}

fn main() {
    foo(|x| x + 2);

    // Weird `'brand` magic protects against multiple mutable borrows:
    let mut w = wrap(|x: u32| x + 2);
    _ = w.to_dyn(); // "first mutable borrow occurs here"
    _ = w.to_dyn(); // "cannot borrow `w` as mutable more than once at a time"
}

[nerditation]

@mattfbacon

The closure is already on the stack, why would I need to care about its size?

my brain couldn't process information properly yesterday, maybe it was overheated or something. I thought you were to store the callback which gets called asynchornously at later times.

glad you already found a solution. in the case where the callback is called synchronously (analogue to nested scopes), I happened to know a crate specific for this use case (unboxed dyn FnOnce), and it's using more or less the same technique as your solution with a brand lifetime paramter.

below is the example code in the "allocation-less dyn FnOnce" section of the docs:

use ::stackbox::prelude::*;

mk_slots!(f1, f2);
let f: StackBoxDynFnOnce_0<()> = if some_condition() {
    f1.stackbox(move || {
        // …
    }).into_dyn()
} else {
    f2.stackbox(move || {
        // …
    }).into_dyn()
};
// …
f.call();

[Yandros]

@nerditation

below is the example code in the "allocation-less dyn FnOnce" section of the docs:

use ::stackbox::prelude::*;

mk_slots!(f1, f2);
let f: StackBoxDynFnOnce_0<()> = if some_condition() {
    f1.stackbox(move || {
        // …
    }).into_dyn()
} else {
    f2.stackbox(move || {
        // …
    }).into_dyn()
};
// …
f.call();

With life-span-of-temporaries extension, this can now be written using own_ref!(), which stems from the WIP eponymous crate, which intends to supersede ::stackbox, by virtue of involving a better name than “stack box”, and instead favoring the nomenclature around &own references.

With those, we get the proper, trivial, answer to the OP:

// First, coerce:
&own impl FnOnce() -> &own dyn FnOwn()

Then, enjoy.

Demo (WIP crate):

let f: OwnRef<'_, dyn FnOwn()> = if some_condition() {
    own_ref!(move || { // : impl FnOnce()
        // ...
    })
} else {
    own_ref!(move || { // : impl FnOnce()
        // ...
    })
};
f.call_ownref_0();

• See more: FnOwn in own_ref::traits - Rust

[Yandros]

@mattfbacon

pattern with a brand lifetime

Pedantic nit: this usage is not what is typically referred to as a brand, since a brand involves a unique lifetime for each and every instance.

Here you could have two CallOnceContainers with a 'static lifetime in them (or even whichever other lifetime of your choice if you don't call to_dyn()), and then call .to_dyn() on both of them (e.g., through a helper Box::leak())

I prefer to call this pattern that of the maximal borrow

[mattfbacon]

Cool, can I ask why the crate is WIP? Are you planning to release it soon?

[mattfbacon]

Good to know, thanks.