Serializing against tests outside my crate

[nickblack]

Hello there! I'm the author of a character graphics/TUI library, Notcurses. As my first Rust project, I'm wrapping this (C) library via bindgen, and implementing a dialog clone, Colloquy. bindgen is awesome, by far the best bindings generator I've ever had the pleasure to work with. Infinitely superior to python-cffi, which brings me no end of annoyance.

My lowest-level wrapping crate is libnotcurses-sys(*). I've got a few fairly basic tests in src/lib.rs(*). These tests (and most tests I'll be writing) make use of the controlling terminal and stdio. They ought not be run in parallel with one another, or with anything else making use of the terminal. When they are, various failures can occur, usually involving EAGAIN. See my #766(*) for examples.

Having read #43155(*), I used the serial_test crate and marked my tests with serial. This reduced the frequency of such failures, but I still get them sometimes. I believe these to be due to my tests being run in parallel with dependency tests. So far as I understand, the serial_test crate can only serialize within groups of annotated tests. Since the tests of my dependencies won't (can't?) be so marked, I'm unable to defend against this parallelism. I cannot reproduce a failure when running with cargo test -- --test-threads=0, which would seem to support my hypothesis.

Any suggestions for ensuring my tests are serialized against all other tests, not just against my other tests? Thanks!

(*) sorry, I would have had links for these, but new users can only include 2(?).

[OptimisticPeach]

Hmm, I presume you'd like to parallelize the tests as much as possible, hence why you don't just resort to using cargo test -- --test-threads=0.

Here's an approach: what happens if you gate all of your tests with a feature and put them into a separate mod (assuming not all of your tests are parallel-sensitive). Then you run the regular tests with your standard cargo test, and the parallel only tests with the --feature Parallel -- <path::to::my::parallel::tests::mod>. The only thing I'm not 100% sure will work is if you can specify and entire module as the test path.

---

Edit: Ooh, even better: just gate the mod instead of individual tests.

This also doesn't need the serial_test crate anymore since this can use the --test-threads option.

[nickblack]

Hmm, I presume you'd like to parallelize the tests as much as possible, hence why you don't just resort to using cargo test -- --test-threads=0.

cargo test -- --test-threads=0
error: argument for --test-threads must not be 0
error: test failed, to rerun pass '--lib'

In any case, I want this applied as a core property of my testsuite; I don't want users to have to know to run with these arguments, and reverse dependencies oughtn't need deal with this.