What terminal are you using?
Because I had been idly wondering for a while why XprotectService on my Mac kicked in for a second or two every time I tried to run some Rust binary I had just compiled. That had started some time ago, but I wasn't sure whether it came with an OS update or at some unrelated time.
Turns out that I'm using iTerm2 as a terminal, and in the "Developer Tools" section under "Privacy & Security" in the system preferences, Apple Terminal was listed under "Allow the applications below to run software locally that does not meet the system's security policy ", but not iTerm2!
Added iTerm2 to that list, restarted iTerm2 - and hey, no more XprotectService delays when doing cargo test!