There’s no regular archive for Windows; you’ll have to either use the installer, or get something that can extract the contents of the installer.
Rust installs itself to wherever you tell it to and will (optionally) add itself to the
PATH. The contents of the install directory are all you need to use Rust, and as far as I know, there aren’t any required registry keys or the like.
Cargo does keep a cache of things in your user folder (where it shouldn’t, but it’s UNIX software, so it doesn’t really care), but not having it there will at worst trigger more downloads and rebuilds. Insofar as I’m aware, you can’t change where it puts it, either.