Tokio JoinHandle and drop

In the following code, is it guaranteed that fut has been dropped once its associated JoinHandle has been awaited?

let jh = tokio::task::spawn(fut);
jh.await?;
// has fut been dropped?

Certainly fut.poll(..) must have returned Poll::Ready, but when does the runtime actually drop fut?

edit: Tokio in particular has this guarantee, but Rust's async in general does not.


Technically, no, you don't have any guarantee that it will be dropped. An executor could in theory poll the Future to completion, and then mem::forget it.

You could wrap fut in your own type that implements Future::poll that drops the inner fut as soon as it's ready (see Option::take, or ManuallyDrop, but be careful about leaks and double-frees on abort). Then you'd have a guarantee that the awaited fut is dropped.

1 Like

This is honestly quite annoying to read. Do you know what the purpose of not specifying that futures must be dropped by an executor was?

A while back I saw someone write that one shouldn't assume that Drops in Rust are run. I thought this was meant as a "haha someone might shut the power off", or "std::process::exit() may be called", but I guess it was meant less tongue-in-cheek than I had originally assumed.

One pattern I've used is to have each receiver Future register itself in a shared map, and once the Future is dropped, it deregisters itself from the map of actively waiting receivers. It really sucks to have to pollute the poll method with logic that I feel belongs in a Drop implementation. (With all that said, I don't think these changes will require a lot of work -- it's just .. ugly).

Out of curiosity: When there's talk about "Async Drop", I always figured this was just referring to that there's no way to .await in Drop implementations (specifically there's risk that Drop implementations can block the executor for an extended amount of time), but does this work also imply that there should be some mechanism to guarantee that some kind of alternative Drop-like method is run for futures?

I'm embarrassed to say that I've written a substantial amount of Futures, several of them with Drop implementations (and taking care to not risk hanging the executor), and I never realized that this was a problem. In my defense, I think that it violates POLS.

Maintainer of Tokio here.

Yes, Tokio guarantees that JoinHandle::await will return after the future is dropped.

The main thing here is reference cycles. If you use Arc to set up a cycle of objects, then the cycle will keep them alive and prevent them from being dropped.

And of course methods like mem::forget and Box::leak will leak destructors.

6 Likes

When you pin something, you promise that you will run the destructor before you repurpose that memory for something else. Since futures are pinned, this means that the only way to not run the destructor of a future is to leak that memory until the process exits. This does not happen under normal circumstances.

3 Likes

Any reasonable executor will drop the future eventually. Just like any reasonable owner of a Rust value will drop it except in special cases that are either intentional or bug-like.

It's just that there is no specification of what “an executor” is, or what it must or must not do. (And futures may be owned by other things than “an executor” such as future combinators.)

If you are concerned about memory leaks, don't be — an executor that did not eventually drop its tasks would be a terrible implementation. If you are concerned about timing of Drop side-effects — consult documentation, and if the documentation isn't adequate, consult implementation and fix the documentation.

5 Likes

To be clear, the futures always get dropped in practice, without any issues. You can use Drop to perform many kinds of cleanup.

But if you're asking about a guarantee, then it has a more specific meaning in Rust. Rust concerns itself with making sound APIs that are completely foolproof and can't cause Undefined Behavior in safe Rust code, even if the safe code is doing nonsensical things. So if you created a Future using unsafe implementation that could crash the program if it wasn't dropped at very specific moment, it would rely on something that Rust doesn't always guarantee, and create a loophole in its safety guarantees.

Rust had to declare that leaking memory is "safe". It doesn't mean it's a good idea to do so, and doesn't mean it happens in practice, but there is a loophole (circular references in recursive refcounted types) that prevents Rust from guaranteeing that nothing ever leaks.

1 Like

If that was required then either Pin would require that drop must run (in which case Box::pin would not be sound) or Future::poll would have to be unsafe, which would make writing an executor without unsafe code almost impossible.

1 Like

@kornel I don't actually agree with this description. The question is about what Tokio's JoinHandle guarantees when you await it. Tokio does indeed guarantee that the future is dropped before the JoinHandle completes.

Like, sure, Tokio could do these things without the unsafe keyword. They would still be bugs in Tokio.

5 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.