How to handle memory leaks from long-lived closures

Up until this point, I've been using closures the way it's defined in the wasm-bingen guide:

The wasm-bindgen guide: Web-sys: Closures

But now I have a bad memory leak issue (browser/workstation freezes, OOM within 20 seconds) when I use more than just test data.

Navigation: psuedo-code:

ListView:

  • Deletes content (e.g. DetailView).
  • loop and create a list of HTML links, each one creating a long-lived closure (and capturing it's environment) for opening DetailView.
let callback_enter = {
    let big_obj_copy = big_obj.clone();
    let mut sm_obj_copy = sm_obj.clone();
    move |event: Event| {
        event.prevent_default();
        let anchor = event
            .current_target()
            .expect("Some: `EventTarget`")
            .dyn_into::<web_sys::HtmlAnchorElement>()
            .expect("Some: `HtmlAnchorElement`");
        sm_obj_copy
            .entries
            .insert(String::from("anchor"), sm_objEnum::Anchor(anchor));
        controller::show_detail(1, &big_obj_copy, &mut sm_obj_copy);
    }
};
let enter_element = get_element_by_id("item-XXXX-detail");
let enter_htmlanchorelement = enter_element
    .dyn_ref::<HtmlAnchorElement>()
    .expect("Some: `HtmlAnchorElement`");
let enter_closure =
    Closure::wrap(Box::new(callback_enter) as Box<dyn FnMut(Event)>);
let enter_closure_ref = enter_closure.as_ref().unchecked_ref();

enter_htmlanchorelement.set_onclick(Some(enter_closure_ref));
enter_closure.forget();

DetailView:

  • Deletes content (e.g. ListView).
  • Displays content based on caputured environment from ListView,
  • contains a long-lived closure (caputuring environment), for returning to ListView.
let callback_leave = {
    let big_obj_copy = big_obj.clone();
    let mut sm_obj_copy = sm_obj.clone();
    move |event: Event| {
        event.prevent_default();
        sm_obj_copy.entries.remove("anchor"); // no longer needed
        controller::show_list(1, &big_obj_copy, &mut sm_obj_copy);
    }
};
let element = get_element_by_id("detail-leave");
let htmlbuttonelement = element
    .dyn_ref::<HtmlButtonElement>()
    .expect("Some: `HtmlButtonElement`");
let closure = Closure::wrap(Box::new(callback_leave) as Box<dyn FnMut(Event)>);
let closure_ref = closure.as_ref().unchecked_ref();

htmlbuttonelement.set_onclick(Some(closure_ref));
closure.forget();

Filter psuedo-code:

ListView (list view)

  • loop and create a list of HTML links, each one creating a long-lived closure (and capturing it's environment) for opening DetailView.

(see ListView closure above)

FilterBox

  • On keyup: Deletes ListView. Recreates (filtered) ListView.
let callback_keyup = {
    let big_obj_copy = big_obj.clone();
    let mut sm_obj_copy = sm_obj.clone();
    move |event: Event| {
        event.prevent_default();
        // ... get element references
        // ... update reset button

        // remove list
        remove_elements(vec!["list-wrapper"]);

        // ... update listeners

        // ... add list
        add_list(&big_obj_copy, &sm_obj_copy);
    }
};

If I have less than ten items I dont't notice anything. Clicking back and forth is OK. Filtering the list is OK.

But when I have thousands of items. I realize that on every rountrip to the ListView, or on every KeyUp event, I am creating thousands of new closures that use .forget(). That's my best understanding of the situation at the moment.

So one thing I tried to to change as many closures to Box+FnOnce as possible. But that had almost no effect because they still required .forget() to compile.

The next thing I tried was adding a drop handler based on this link:
[Question] How do I remove event listener?

So I was able to add this:

pub struct Listener {
    element: web_sys::EventTarget,
    name: &'static str,
    callback: Closure<dyn FnMut(Event)>,
}

impl Listener {
    pub fn new<F>(element: EventTarget, name: &'static str, callback: F) -> Self
    where
        F: FnMut(Event) + 'static,
    {
        let callback = Closure::wrap(Box::new(callback) as Box<dyn FnMut(Event)>);
        let cb_ref = callback.as_ref().unchecked_ref();

        element
            .add_event_listener_with_callback(name, cb_ref)
            .expect("Result<(), JsValue>");

        Self { element, name, callback }
    }
}

impl Drop for Listener {
    fn drop(&mut self) {
        let name = self.name;
        let cb_ref = self.callback.as_ref().unchecked_ref();
        self.element
            .remove_event_listener_with_callback(name, cb_ref)
            .expect("Result<(), JsValue>");
    }
}

And use it like this

let leave_el = get_element_by_id("detail-leave");
let leave_tgt = leave_el.clone().into();
let leave_el_ref = leave_el
    .dyn_ref::<HtmlButtonElement>()
    .expect("Some: `HtmlButtonElement`");

// add listener stuff
let leave_cb = {
    let big_obj_copy = big_obj.clone();
    let sm_obj_copy = sm_obj.clone();
    move |event: Event| {
        event.prevent_default();
        controller::stg_init(1, &big_obj_copy, &sm_obj_copy);
    }
};
let leave_ls = Listener::new(leave_tgt, "click", leave_cb);

// drop listener stuff
let leave_drop = move || { drop(leave_ls); };
let leave_cl = Closure::once(Box::new(leave_drop) as Box<dyn FnOnce()>);
let leave_cl_ref = leave_cl.as_ref().unchecked_ref();
leave_el_ref.set_onclick(Some(leave_cl_ref));
leave_cl.forget();

But the issues remain:

  • If I have thousands of links - and only one gets clicked - then only one closure gets dropped. I need all the list closures dropped.
  • The keyup, and navigate-back modules are separate from, and do not have access to, the drop handlers created in the ListView, so how could they trigger them?

Any ideas would be appreciated. Thanks.

If I have thousands of links - and only one gets clicked - then only one closure gets dropped. I need all the list closures dropped.

Fundamentally, you can't solve this problem in a “self-contained” way where each item-with-event-listener cleans up after itself when removed. What you need is a single vector or other data structure on the Rust side — not owned by any of the closures, but owned by whatever your top-level application state is — which contains all the Closures for the active list items, so that when one of the list items is clicked and you discard the list, you can also clear the vector of all of the Closures that are no longer needed.

However, it might be easier and cleaner to use JavaScript event listeners instead. You can use the “JS snippets” feature to add JavaScript from within your Rust code. The reason this can work better is that JavaScript functions/closures can participate in the JavaScript GC, and so don't need to be kept alive explicitly. Then, they can communicate back to your application in a way which, even if it involves invoking a Closure, doesn’t involve creating many separate Closures whose ownership has to be managed.

3 Likes

Yeah, I think I may go with the 'track the closures in rust' route. I'm just trying to think of the best way to retrofit all the existing code to handle this, because this affects every possible click in the app.

To solve this in general, you essentially need a data structure, parallel to the DOM tree, which owns the event listeners.

It doesn't have to represent all nodes, or to have exactly the same tree structure, but does have to represent enough of the structure that whenever you want to delete nodes from the DOM, there's a corresponding part of this other structure for you to drop to clean up the Closures. One could say that its state reflects which events you are expecting, and the requirement to avoid leaks is that you mustn't keep any part of it which you are definitely not expecting any more events from.

At this point I'm stuck because I do not have enough exerience with Dyn, Closure or FnMut items to know how to store and read them. Trying out things like puttting them in a Vec or HashMap in an already existing struct givese me errors about not satisfying the traits for Clone

#[derive(Clone, Debug)]
pub enum ArgsEnum {
    // snip...
    Event(HashMap<String, Closure<dyn FnMut(Event)>>),
    // snip...
}
error[E0277]: the trait bound `Closure<dyn FnMut(Event)>: Clone` is not satisfied
  --> src/types/controller.rs:13:11
   |
10 | #[derive(Clone, Debug)]
   |          ----- in this derive macro expansion
...
13 |     Event(HashMap<String, Closure<dyn FnMut(Event)>>),
   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `Closure<dyn FnMut(Event)>`, which is required by `HashMap<std::string::String, Closure<dyn FnMut(Event)>>: Clone`
   |
   = note: required for `HashMap<std::string::String, Closure<dyn FnMut(Event)>>` to implement `Clone`

If if put it in a new struct without #[derive(Clone, Debug)] then I can get it to complile, but I was trying to attempt this with the structs that are already passed to each function.

For this error:

7  | #[derive(Clone)]
   |          ----- in this derive macro expansion
...
11 |     callback: Closure<dyn FnMut(Event)>,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `Closure<dyn FnMut(Event)>`

I'm looking at this thread:
How to clone a boxed closure

And this linked playground:
Rust Playgound - Box Clone

And It may be what I need to do to add a Vec of Listeners to say, the Model struct below:

use wasm_bindgen::{closure::Closure, JsCast};
use web_sys::{Event, EventTarget};

#[derive(Clone)]
pub struct Model {
    pub events: Vec<Listener>,
    // snip...
}

#[derive(Clone)]
pub struct Listener {
    element: EventTarget,
    name: &'static str,
    callback: Closure<dyn FnMut(Event)>,
}

But I'm not entirely sure yet. If I can get the Vec working, then I can try, in a different view - if I can iterate over the Vec and drop all the stored closures, before setting the closures for the current view.

You definitely shouldn't be trying to clone your functions. The place the Closures go should be a data structure that doesn't try to implement Clone because it doesn't need to. Cloning would just give you more things to potentially leak.

You don't necessarily need to write code that explicitly drops the closures. Just clearing the vector whenever you change the DOM, before you start putting the next set of Closures in, would do. (Assuming that you're replacing everything.)

2 Likes

OK. I'll move the events Vec<Listeners> out of the cloneable Model object and see how far I get.

Edit:

Not very far. Now all my functions take an events object in addtion to the big_obj and sm_obj. The problem arises when I caputure the environment in the closure - the events object is now also one of the things I need to capture (which is done by cloning).

let callback_leave = {
    let big_obj_copy = big_obj.clone();
    let mut sm_obj_copy = sm_obj.clone();
    // let mut events_copy = events.clone();  <<<<< NOT POSSIBLE (Clone not implemented)
    move |event: Event| {
        event.prevent_default();
        sm_obj_copy.entries.remove("anchor"); // no longer needed

        // Callback function with caputured environment
        controller::show_list(1, big_obj_copy, sm_obj_copy, events_copy);
                                                            ^^^^^^ NEED TO ADD
    }
};

Edit:
Some added context. the reason sm_obj and big_obj (and now the events_obj) are cloned and passed to every function is because (as far as I understand it), it is preferred in rust to structure things that way (avoid globals).

E.g. from Stack Overflow

Avoid global state in general. Instead, construct the object somewhere early (perhaps in main), then pass mutable references to that object into the places that need it. This will usually make your code easier to reason about and doesn't require as much bending over backwards.
Look hard at yourself in the mirror before deciding that you want global mutable variables. There are rare cases where it's useful, so that's why it's worth knowing how to do.

Update:

Comparing my MVC project to the one written by the wasm_bindgen team:
Wasm_Bindgen: TODO MVC

I'm reminded to how much I did not understand or thought was overkill for my small CRUD app:

  • Heavy use of Rc and RefCell
  • Reimplenting the JavaScript event loop
  • HTML Templates
  • Lots of shortcut (clever/obscure) coding patterns - things only the authors of the wasm_bindgen crate would know to use.

But...

They seem to be able to drop their listeners/closures just fine:
impl Drop for View (see bottom of file)

They also seem to get around the cloning / capturing-environment issue by only passing messages (strings or booleans) which are match'ed and passed to methods in the controller or view structs - which will trigger the desired behavior. -- In my app, I copy and pass entire structs as arguments - which means I may have to rewrite my entire app from scratch to imitate the wasm_bingden way of handling views / dropping closures/listeners. -- because it may be the only way things can work.

Update:

After rewriting my unfinished code to use some (but not all) of the Wasm_Bindgen: TODO MVC structure. I was able to resolve the memory leaks. Primarily by:

  • Using what they call the scheduler as a Mediator - and making the Model/View/Controller depend on the Mediator:
    let mediator = Rc::new(Mediator::new());
    let model = Model::new(Rc::clone(&mediator));
    let view = View::new(Rc::clone(&mediator));
    let controller = Controller::new(Rc::downgrade(&mediator));
    let mdtr: &Rc<Mediator> = &mediator;
    mdtr.set_controller(controller);
    mdtr.set_model(model);
    mdtr.set_view(view);
pub struct Model {
    mediator: RefCell<Rc<Mediator>>,
    // ...
 }
  • Not having to put copies of the (large-ish) data structs, or data methods, in every closure - just the changes in data.
let my_closure = Closure::<dyn FnMut(Event)>::new(move |event: Event| {
    event.prevent_default();
    let value = event...value();
    if let Ok(mdtr) = &(mdtr.try_borrow_mut()) {
        mdtr.add_message(Message::Ctrl(MsgCtrl::DoStuff(value)));
    }
})
  • Using a template struct + templates folder where each template:

    • Adds any created listeners+closures to a Vec on the Templates struct - instead of using .forget()
    event_target
      .add_event_listener_with_callback("click", my_closure.as_ref().unchecked_ref())
      .expect("Failed: `add_event_listener_with_callback()`");
    
    self.listeners.push((event_target, String::from("click"), my_closure));
    
    • Removes+drops the listeners+closures collected by the previous template - if the DOM elements get removed.
    for listener in self.listeners.drain(..) {
      listener
          .0
          .remove_event_listener_with_callback(
              listener.1.as_str(),
              listener.2.as_ref().unchecked_ref(),
          )
          .expect("Failed: `remove_event_listener_with_callback()`");
    }