How do i write integration tests that are their own crate?

i am writing a crate that is used inside a build script, and i need to write integration tests for it, which each require their own build script and thus need to be individual crates.

i tried running cargo new inside tests, but neither cargo test nor rust analyzer recognize the crate.

You're looking for workspaces:

i'm already using a workspace, and i eventually decided to add another crate as a workaround,
but if the test crate's build script panics it stops the entire build and the tests in other crates from running instead of just logging a test failure and continuing with the other crate's tests (the same applies when the build script generates invalid code).

I'm not really sure what your after here, you are mixing together different requests.

You make cargo recognize crates including having it's tests being runnable by having the crate path be matched by the workspace.members array.

This is how you would get a build script separate to the main crate build script, by just moving the tests to a tests crate, eg "foo-tests" for "foo". Tests themselves are, like examples and bins, only targets within a crate, and don't have a separate build script (strictly, a directory with a Cargo.toml is a "package" which contains targets, and a crate is a built target, but hardly anyone uses that terminology)

You also might simply be looking for the cargo test option "--no-fail-fast" which doesn't stop at the first failing target.

Another option you could look at is to ditch the build script and the default libtest harness, by setting the harness property on the test target to false, see Cargo Targets - The Cargo Book

This basically means your test is just a regular binary with a main function, #[test] is not treated differently, which means you can do whatever setup and execution you like, but you don't get the default test filtering and output.

I'm not really sure what your after here

in my case the test is literally whether the code generated by the build script compiles (there are also a few unit tests that test the compiled code afterwards, which should run if the generated code actually compiled and just be ignored otherwise).

with the "other crate in workspace" workaround, if the build script generates nonsense then nextest (and cargo test, they react the same way) immediately exits (due to all the syntax errors in the test crate), without running any tests in the other crates that do compile.

what i'm after is some way to have "the build script of the test crate generated code with 200 syntax errors" just show up a test failure along with the results of the rest of the tests (i usually use -no-fail-fast to get all results) instead of immediately exiting.

You also might simply be looking for the cargo test option "--no-fail-fast"

-no-fail-fast does prevent failing tests from immediately exiting, sadly it does not prevent crates that don't compile from immediately exiting

I think you might want something like trybuild for your tests. This runs the compiler inside of tests, so that compiler failures are test failures, and Cargo doesn’t see them at all.

trybuild looks promising,
but from the examples it looks like it always uses individual files instead of entire crates (or does it also work with crates but just never say that in the docs?),
and it doesn't look like it has a option to run any tests inside whatever it build if building succeeded.

Perhaps you’ll need to write something like trybuild that fits your needs. I haven’t used it myself yet, so I can’t comment in detail.

it always uses individual files instead of entire crates (or does it also work with crates but just never say that in the docs?),

Every invocation of rustc compiles an “entire crate”; that’s part of the definition of what a crate is. (Packages are often confused with crates, though.)

Every invocation of rustc compiles an “entire crate”; that’s part of the definition of what a crate is.

what i meant is that trybuild is always called on individual rust files, none of the examples show it being pointed at a folder with a build.rs, src directory, etc.
(although looking at how it explicitly mentions using rustc it is likely that half of what i consider to be part of a "crate" is actually a cargo thing and thus entirely incompatible with it as it doesn't use cargo, one of the downsides of cargo is that its so good that i never interact with rustc directly and thus forget it exists)

Perhaps you’ll need to write something like trybuild that fits your needs.

i'll likely end up needing to do that, at which point i'll have a side project of a side project :ferris_melt:
i wonder what the record for most deeply nested side project is

thanks for the help,
imma keep the issue open in case i actually finish a tool that works for this use case (in which case i'll post it here and mark it as a answer for people with the same problem finding this post)