Tokio spawn_blocking in endless loop while borrowing self

I have the following async code:

    /// Starts the MemoryCleaner, which will run forever.
    pub async fn start(mut self) {
        info!("MEMORY: Sarting delete_deamon!");

        let seconds = Duration::from_secs(self.interval);
        let interval = self.interval;
        loop {
            info!("MEMORY: Waking up!");

            // If we encounter any errors, we log them and continue
            let join_handle = tokio::task::spawn_blocking(|| self.clean());
            match join_handle.await {
                Ok(res) => {
                    if let Err(e) = res {
                        error!("{e}");
                    }
                }
                Err(join_err) => {
                    error!(
                        "Memory cleaner failed to spawn clean() task with tokio: {}",
                        join_err
                    )
                }
            }

            info!("MEMORY: Going to sleep for {} seconds", interval);
            // thread::sleep(seconds);
            tokio::time::sleep(seconds).await;
        }
    }

    fn clean(&mut self) -> Result<()> {

self.clean is a blocking command (and not async), so I wrap it in spawn_blocking.

I am getting the error:

cannot borrow `self` as mutable more than once at a time
`self` was mutably borrowed here in the previous iteration of the loop

I understand the error, but I think there has to be some way that I am not aware of with which it is possible to do what I want to do. Any ideas?

The problem is that tokio::task::spawn_blocking() requires the closure to be 'static, i.e. not to capture any short-lived references. This is because the spawned task/thread may outlive the caller (thus, capturing references may cause them to dangle).

I don't think there's a good solution to this problem. You can wrap Self in an Arc and move a clone of it into the closure; this will require wrapping whatever state execute() and clean() need to mutate in a Mutex.

3 Likes

I thought, maybe naively, that joining the thread would allow me to just pass the reference, since I am only ever spawning one thread at a time and waiting for it to complete. But I guess the rust compiler doesn't know this, and can't guarantee that I'm not calling another blocking thread with the same syntax e.g. before the loop.

But there's nothing in the signature of spawn_blocking() that would provide type information related to that.

Native std threads have a scoped API that enforces the appropriate lifetime constraints. Tokio doesn't seem to have a parallel.

1 Like

It looks like tokio::task::block_in_place may work here.

1 Like

Would something like this work (untested)?

let join_handle = tokio::task::spawn_blocking(move || (self.clean(), self));
self = match join_handle.await {
    Ok((Err(e), _)) => { error!("{e}"); }
    Ok((Ok(_), self_ret)) => { self_ret }
    Err(join_err) => {
        error!(
            "Memory cleaner failed to spawn clean() task with tokio: {}",
            join_err
        )
    }
}

The best solution may be to use a single plain non-Tokio thread to run the cleaner loop. There is nothing in the loop that really needs to be async, and a single long-lived thread is not a large cost.

4 Likes

Yes, I was thinking of that as well. Is the tokio::time::sleep somehow favorable over a normal thread::sleep, because it is awaited and, therefore, gives resources back to the runtime? The interval (duration of sleep) is usually not that small, e.g. 60 seconds, so I thought this could be useful. But creating a new blocking thread every loop iteration is probably also not ideal.

There is no advantage to a Tokio sleep specifically. The advantages of using a tokio task instead of a thread are:

  • no OS-level context-switch between it and other tasks (less CPU time)
  • no memory tied up in a dedicated thread stack

and you can think of calling the Tokio sleep as “giving the thread's resources to the tokio runtime to run a different task”, but I don't think it's useful to think of that as being about sleep, but rather about using async tasks instead of dedicated threads.

But in this case, these are small benefits when you have only one infrequently-waking task.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.