Passing --test-threads=1 to cargo test by default?

help
1

I'm writing a Rust wrapper for a C library. This C library contains some global state and therefore cannot be used by multiple threads at the same time without precautions. This means that when I run cargo test, I'm getting segfaults because Rust runs multiple unit tests in parallel using threads. To make my tests pass, I need to pass --test-threads=1.

Is there a way, e.g. in Cargo.toml, to specify that this should be done by default when running cargo test? Alternatively are there options that could tell Cargo that tests should be run in distinct processes instead of distinct threads?

1 Like
2

Unfortunately, there is no way to configure the default testing behavior

But, if your tests are segfaulting, then your library is not sound. It should protect the global state, such as with a mutex.

2 Likes
3

Thanks for your answer.

Just to clarify: the global state is deep down inside a dependency of the C library (with global initialization and finalization functions, so if a test calls the finalization function while other tests/threads still use the library, these tests are just going to crash). Besides, this library is not meant to be used in a multithreaded context so even though I could make the Rust binding thread-safe, it would be an overkill (with performance overhead).

4

Even if you can't add mutex to protect bindings (which is definitely a problem IMO, but I don't know all there details) you can add mutex to your tests:

static LIBRARY_USER: Mutex<Executor> = Mutex::new(Executor);

#[derive(Clone, Copy)]
struct Executor;

impl Executor {
    fn run_test(self, f: impl FnOnce()) {
        f();
    }
}

#[test]
fn my_test() {
    LIBRARY_USER.lock().unwrap().run_test(||
        println!("Hello, world!")
    )
}
4 Likes
5

Do what you want with your own code for your own purposes, but if you plan to publish it for others to use, this is not sufficient. A key element of Rust's “fearless concurrency” is that libraries are thread-safe — more precisely, that libraries are sound under all possible interactions with them from safe code (which includes multithreaded code).

10 Likes
6

Mutexes are pretty cheap without contentious. In single threaded context it can be considered nearly zero cost for most use cases. You can acquire a global mutex at the entry of public API or even throw panic if it's not called from main thread depending on the actual requirements the C library has.

7

The help info says:

$ cargo test -- --help

...

By default, all tests are run in parallel. This can be altered with the
--test-threads flag or the RUST_TEST_THREADS environment variable when running
tests (set it to 1).

...

so you can try RUST_TEST_THREADS=1 cargo test.

Another general way is to define a custom command alias via .cargo/config.toml in your project, like:

# in path_to_your_project/.cargo/config.toml
[alias]
t = "test -- --test-threads=1"

then you can use cargo t[1] to run tests.

  1. cargo t is the default shorthand form of cargo test ↩︎

8

Thanks all, all good material to work with :slight_smile:

My problem appeared in a -sys package for which I'm adding tests. This is a crate that just provides the FFI bindings for the C library, not the place to add mechanisms to make it rust-oriented and safe (let alone thread-safe). I do plan to write a safe package on top of that, with a rust-oriented API (using a static Mutex in particular to protect calls to the C API). But my understanding is that it's not the role of a -sys package to provide anything more than the FFI bindings, hence such a package, even published, will not be safe.

9

A lock inside the tests is the best solution.

Multi-threaded cargo test is the standard way of launching the tests, and if that doesn't work properly, you'll have hard time explaining it to everyone and everything that may use the package.

1 Like
10

This still won't work for everyone. In Stakker when not used with "multi-thread" or "multi-stakker" features, I have a global singleton that effectively makes one thread permanently special. This cuts out a lot of checks later on, once that singleton check has passed. For people who only need to run Stakker in one thread, why should they pay multi-threaded costs? To test that I have to force it to run the tests with just one thread. Anyway, I have a default feature which makes plain cargo test work, forcing the multi-thread feature on for that, to meet people's expectations. But right now (1.68.1) RUST_TEST_THREADS=1 and -- --test-threads=1 seem broken. I'm getting panics from the singleton check. Maybe they're doing one thread at a time, over various threads, which is equally bad from my point of view. I think I'll have to write a script to cargo test just one test at a time.

11

If you keep the tests within the library (not in tests/) then you can use #[cfg(test)]/cfg!(test) to enable extra locking.

From the perspective of Rust, single-threaded programs don't exist, e.g. there's no safe static mut.

12

Thanks for the ideas. I'll see if I can find some way along these lines to get my tests to work.

Regarding this single-threaded optimisation, okay, it uses unsafe, but if it is sound then Rust ought to be able to support it. Ignoring single-threaded has caused problems for async/await too, which I'm glad to see they're starting to fix now. These are valid use-cases.

13

There's a bug for it on github [Enable specifying --test-threads in Cargo.toml · Issue #8430 · rust-lang/cargo · GitHub], where there is the following suggestion:

I found .cargo/config.toml can specify environment permanently.

[env]
RUST_TEST_THREADS = "1"

So maybe that will solve the OP's problem.

4 Likes
14

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.