I expect that running cargo test won't have that compile error. However, this condition compilation does not work. I suppose the condition #[cfg(not(test))] was not supported, however
#[cfg(not(test))]
pub fn add(){}
when running cargo doc --open, the documentation does have this item while #[cfg(test)] makes the item hidden in the documentation, this result witnesses this condition compilation should work. Why does it not work for the compile_error!{"some error"} item?
When you are testing a library, the library is still also compiled withoutcfg(test), for use in its integration test targets (tests/*.rs) and dependent packages. (cfg(test) is only active when producing a test binary that runs #[test] functions, not in any other case. That test binary is not a library.)
I’d hope that’s skipped if you have none of those, but perhaps you do, or perhaps Cargo isn’t that clever.