Scoped closures for owned structs

help
1

This is not a real problem that I have, but I was thinking on how similar functionality can be implemented in Rust.

I have an application struct, which owns other subsystems, such as the Executor. I want to be able to push closures to the executor, which contain references to the application struct. Essentially I want to have my own scoped closures, similar to scoped threads in std. As in the following example.

What are the safe ways to implement this? I want the App to specifically not be wrapped in Rc.
Also I know that this implementation has a flaw that std::mem::swap can, in theory, be abused, to replace the executor with another one, which will take away ownership from the App, also Executor can move the closures somewhere, which will also result in access-after-free.

struct App(RefCell<AppInner>);

struct AppInner {
    executor: Executor,
    ...
}

struct Executor {
    queue: Vec<Box<dyn FnOnce()>>,
    ...
}

impl App {
    fn on_startup(&self) {
        // SAFETY: The executor is owned by the application and is dropped
        // before the `Application` is destroyed. `RefCell` ensures that mutable
        // aliasing will not happen.
        let this: &'static Self = unsafe { std::mem::transmute(self) };
        self.borrow_mut().executor.push_task(|| this.notify_pause());
    }

    fn notify_pause(&self) { ... } 
}

impl Executor {
    fn push_task(&mut self, task: impl FnOnce()) { ... }
}

And even if I am working with Rc, I wonder whether there a way to go from &Self to Rc<Self> in the on_startup function, similar to how shared_from_this is working in C++. I've seen this discussed here[1], but wouldn't mind hearing more thoughts about this.

  1. https://stackoverflow.com/a/79259201 ↩ī¸Ž

2

That would be a self-referencial struct. There's no built-in functionality to do that in a useful way. There are some crates that attempt enabling it, though I don't know if any have achieved actual soundness.

Creating a &mut self invalidates the captured reference.

If you're going to attempt unsafe, at a minimum you should test things with Miri.

3

I don't understand the comparison to scoped threads. Here there is no block/scope that constrains the closure's lifetime, right?

One simple way around the problem, if an &App is always available when executing tasks, is to pass &App to the closure:


struct Executor {
    queue: Vec<Box<dyn FnOnce(&App)>>,
    // ...
}

impl App {
    fn on_startup(&self) {
        self.0
            .borrow_mut()
            .executor
            .push_task(|app| app.notify_pause());
    }

    fn notify_pause(&self) {}
}

impl Executor {
    fn push_task(&mut self, task: impl FnOnce(&App)) {}

    fn execute(&mut self, app: &App) {
        self.queue.drain(..).for_each(|task| task(app));
    }
}
4

This came back again, and I still don't know how to properly implement this.

I have a stack of networking callbacks:

struct GameClient {
    client: NetworkClient,
    ...
}

struct NetworkClient {
    client: NetworkClientCore,
    ...
}

struct NetworkClientCore {
    client: UdpClient,
    ...
}

struct UdpClient {
    connection: UdpClientConnection,
    ...
}

If the game wants to send the network message. It just picks its client and calls send_message on it, which is then propogated down the "stack" up to the UdpClientConnection send on socket.

However, when the socket receives a message, I want to propogate it upwards to GameClient.

In C++ I would have done this with callbacks, where all of those classes accept callbacks from their parents like so:

struct GameClient {
    client: NetworkClient,
    ...
}

struct NetworkClient {
    client: NetworkClientCore,
    on_packet_received: Fn(Packet),
    ...
}

struct NetworkClientCore {
    client: UdpClient,
    on_packet_received: Fn(Packet),
    ...
}

struct UdpClient {
    connection: UdpClientConnection,
    on_packet_received: Fn(Packet),
    ...
}

impl GameClient {
    pub fn new() -> Self {
        let this = Self { ... };

        let mut network_client = NetworkClient::new();
        network_client.on_packet_received = |packet| {
            this.handle_packet(packet) // <- cannot do in Rust!
        };
    }

    pub fn handle_packet(&mut self, packet: Packet) { ... }
}

impl NetworkClient {
    pub fn new() -> Self {
        let this = Self { ... };

        let mut network_client_core = NetworkClientCore::new();
        network_client_core.on_packet_received = |packet| {
            this.handle_packet(packet) // <- cannot do in Rust!
        };
    }

    pub fn handle_packet(&mut self, packet: Packet) { ... }
}

/// etc.

Essentially the idea is for callbacks is to propagate the message up the stack.

With &mut I can deal by storing all clients in RefCell. Shouldn't be much of a performance impact.
But I don't want to deal with Rc. Since ownership here is pretty clear and GameClient owns NetworkClient, which owns NetworkClientCore etc.

Is there a way to do that similar to how scoped threads work? If so, then how?

5

This ties App to Executor, which I don't want to do.
Also, please, check out my second example, where being able to split responsibilities is even more important (GameClient is a different crate, and is not part of NetworkClient logic).

6

In the code you've shown, ownership of a layer object is shared between its child layer's callback and its parent layer. Shared ownership in Rust requires either Rc<RefCell> or Arc<Mutex>. This is the same as when using smart pointers in C++.

To make it simpler you can probably avoid callbacks and store parent pointers at each layer. However, ownership is still shared, so each layer object still needs to be wrapped in an Rc<RefCell> or Arc<Mutex>.

That technique is not applicable because it relies on single ownership and temporary borrowing within a fixed scope/block that is known to the compiler. It is very similar to passing references to a function.

7

ownership of a layer object is shared between its child layer's callback and its parent layer
Well, I don't want to share the ownership. I just want to give a reference to children, since I can statically prove (myself, not the compiler), that children are owned by the parents, so their references cannot escape.

Maybe I can do this somehow with ouroboros crate, maybe by allocatting everything in a single arena (also works). But I'd rather not use Rc.

8

Just curious, can you say why not? Using ouroboros will add as much complexity as using Rc. I assume the number of the layer objects that you create is not huge, so allocations wouldn't be an issue. The overhead of Rc borrowing is minuscule.

But yes, if you're in a position where an arena works, that's an ideal solution.

EDIT: You need to mutate the layer object in the callback via a mut reference, as well as directly via the owned object (I'm assuming the latter is true). So an anena won't help here, nor ouroboros I believe.

Since I'm trying to encourage you to use Rc, here is how it can be done :slight_smile:
playground

9

I do not think your design is even sound.

struct GameClient {
    client: NetworkClient,
    i: usize,
}
impl GameClient {
    fn do_something(&mut self) {
        self.i += 1;
        {
            // only self.client is borrowed here, it is disjoint from self.i
            self.client.do_something_network();
        }
        // the compiler would want to optimize this couple of changes out
        // once it can prove that `do_something_network` does not panic
        self.i -= 1;
    }
}

struct NetworkClient {
    callback: Box<dyn FnMut(usize)>,
}
impl NetworkClient {
    fn new_for(game: *mut GameClient) -> Self {
        let callback = Box::new(move |n| {
            // with RefCell, that would be a `.borrow_mut()`
            let game_client: &mut GameClient = unsafe {&mut *game};
            game_client.i = 42;
        });
        Self {callback}
    }
    fn do_something_network(&mut self) {
        // we did some processing and detected we have to callback
        (self.callback)(42);
    }
}

This code is UB. If you used RefCell, it would panic instead, indicating a double borrow.

The values cannot change from under the GameClient::do_something just because it invoked function on its part. This is something Rust decidedly does not offer, and it seems to save people from a lot of non-trivial bugs.

If you want to change a value, you need to pass a reference to it from whatever function controls (owns or has &mut T) it now. Or to pass the results up, as it happens.

1 Like
10

I mean, it is sound in C++. And I want to have something similar in Rust.

11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.