Close fantoccini client when an assertion fails inside an integration test

help
1

Hello,

I'm new to Rust and I'm experimenting with it to do a personal web project.
I'm at the stage where I want to implement integration tests. For this I've decided to use the fantoccini crate to drive the browser from my rust test code. This works, but there is the annoying problem that if an assertion fails the browser window is not closed. To close the window I'm using

#[rocket::async_test]
#[serial]
async fn test_firefox() -> Result<(), Box<dyn std::error::Error>> {
   // some setup code to start geckodriver

    let client: fantoccini::Client ...  /* connect to the driver */;

   // test code is here, it uses an assert which can fail if a tests fails

    client.close().await?;
    Ok(())
}

So the close method on the client is async and closes the browser, but if there is a panic (because of the assert failure) it is not executed. The standard Drop is not async and the Async Drop is not stable.

What can I do to implement proper clean up in this situation? I'd prefer if I continue using stable rust!

2

You have two bad choices:

  1. Use std::panic::catch_unwind and std::panic::resume_unwind.
  2. Use block_on from your async runtime in the Drop method, to block the thread completely while the close happens.

Before choosing the first option, you should read and understand the documentation for catch_unwind - there's a bunch of potential issues here, and while you'll not normally see them, you need to know what they are in case they happen for you. You can then catch the panic, do your close, and resume unwinding.

The second option has fewer gotchas, but is still not perfect. You'd use rocket's direct access to the async runtime it uses (tokio), to allow you to grab access to the current runtime and spawn a task. You then need to be careful to move the client into the newly spawned task. This looks something like:

pub struct ClosingClient {
    client: Option<fantoccini::Client>
}

impl ClosingClient {
    pub fn new(client: fantoccini::Client} -> Self {
        Self {
            client: Some(client)
        }
    }
    pub fn client(&self) -> &fantoccini::Client {
         self.client.as_ref().expect("Only taken on drop");
    }
}

impl Drop for ClosingClient {
    fn drop(&mut self) {
         let client = self.client.take().expect("Should not have been taken before drop");
         let handle = tokio::runtime::Handle::current();
         handle.spawn(client.close());
    }
}

#[rocket::async_test]
#[serial]
async fn test_firefox() -> Result<(), Box<dyn std::error::Error>> {
    // some setup code to start geckodriver

    let client = ClosingClient::new( ...  /* connect to the driver */);
    let client = client.client();

    // test code is here, it uses an assert which can fail if a tests fails

    Ok(())
}

This has more panic paths in your code, and can still fail if you somehow move ClosingClient out of the scope of the runtime (e.g. via a channel, or shared memory), but is otherwise more reliable than catch_unwind.

It also has the problem that client.close can finish some time after ClosingClient has been dropped - and this is non-deterministic, so you can run into surprise "drop order" problems, where Drop::drop is called at the right time, but the spawned async tasks finish in the "wrong" order. Be careful to avoid cases where this matters.

3

What do you mean by "be careful to avoid..."? How I can I even know that I'm causing a particular problem. After all most of the code I'm interacting with is hidden/generated/part of the runtime. Will I be seeing panics or I'll be in UB territory and the behavior will be random?

And thank you for the detailed explanation!

4

You are definitely not in UB territory; behaviour is fully defined, just non-deterministic ordering. It also would be extremely surprising if your dependencies (fantoccini, rocket, tokio etc) had unsound code that this trips over.

Note, too, that by definition in Rust, if there's UB, there's the keyword unsafe somewhere, and culturally, we work on the principle that you should make use of module boundaries, crate boundaries etc to ensure that if you have any unsafe in your code, you've made sure it can't cause UB in your callers, unless your caller includes unsafe.

If you have something where the order between two Drop implementations matters to you (e.g. where you care that something else is not dropped until after fantoccini::Client is closed and dropped), then this solution introduces a new problem, where you do not know exactly when fantoccini::Client will be closed and then dropped - only that it will happen at some time after ClosingClient is dropped.

The most likely symptom of this is that you get log messages in the "wrong" order - you expect to see ClosingClient's last log, then the logs related to fantoccini::Client closing and being dropped, and then the logs relating to something else being dropped, but instead you see ClosingClient's last log, then the logs relating to something else being dropped, then the logs related to fantoccini::Client closing and being dropped.

99.99% of the time, this doesn't matter, and you'll know if you're in the other 0.01% where you really, really care that two unrelated things are dealt with in the right order.

5

Thanks for the explanation again.

I don't think I care for the order, but it seems the spawned task is never executed.
I can see that ClosingClient::drop is called, the spawn call is executed, but the task is never executed - the browser window is not closed, no if I put any prints in the task block.

The behavior is the same if there is an assert or not.

Is it possible to see the code generated by a macro like the one used for #[rocket::async_test] or #[serial]?

6

cargo expand will show you the expanded code from a macro; because this is using Tokio, it should wait for the task to finish, unless Rocket's code shuts down the runtime too early (in which case the task will be dropped).

7

This is cool. Thanks.

Using it I can see (in fn async_run<F, R>) that rocket adds a 500msec timeout for the runtime and then terminates it.

I've switched to the tokio::test macro with #[tokio::test(flavor = "multi_thread", worker_threads = 1)], and now the spawned task from the drop is executed/started, but the execution of the client.close() task doesn't finish. And I'm not really sure how to debug it.

Is there some kind of tutorial how to debug these async executions using something like lldb+vscode. I've tried it and the execution jumps to random locations at every step.

8

I've tended to use a mixture of tokio-console (to see what the Tokio runtime thinks is going on) and tracing (to instrument my code, so that I've got clean spans and traces - I really like the way tracing-chrome presents the information).

The key thing to remember is that wherever sync Rust would block the thread waiting for something (e.g. I/O, time delays), async Rust returns to the Tokio executor to await the same thing, giving the executor a chance to execute something else on the same thread (where the "something else" has finished awaiting the thing that would have blocked it).