Can I (and how do I) install on Windows as a portable install to a thumb or network drive?
If not, what gets installed where with the .msi installer? I don't want to try without knowing in case the installer puts things in places where they might leave remnants on uninstall. (Like Program Files or shadow directories.)
There's no regular archive for Windows; you'll have to either use the installer, or get something that can extract the contents of the installer.
Rust installs itself to wherever you tell it to and will (optionally) add itself to the PATH. The contents of the install directory are all you need to use Rust, and as far as I know, there aren't any required registry keys or the like.
Cargo does keep a cache of things in your user folder (where it shouldn't, but it's UNIX software, so it doesn't really care), but not having it there will at worst trigger more downloads and rebuilds. Insofar as I'm aware, you can't change where it puts it, either.
About extracting MSI content:
http://superuser.com/questions/307678/how-to-extract-files-from-msi-package
That would require setting environment variables on every machine, which isn't very portable. I suppose you could do a wrapper around Cargo... but that's starting to get into "all software is portable with enough effort" territory (though only just). 