Welp. On windows (native), installing Rust-docs still takes forever. This isn't a good thing....
All right everyone, I tried it using Windows native, not through WSL. Apparently, the compilation speed is much better, taking only 2-3 seconds. Installing/updating rust is still hideously slow though, taking the same amount of time as before.
1 Like
I will second that the other day when I updated Rust on Windows 10 (native MSVC) installing the docs took an incredible amount of time (even on a PC with a fast CPU, SSD, and a 50Mbps Internet connection). I believe this is compounded when you have stable + nightly, etc versions that all need to be updated.
It would be nice if this experience was better out of the box on Windows.
1 Like
Yes. I find it strange that WSL is affected, too. Unless there's something underlying different about Windows itself, then I don't see why it's slow. Except for a few (kinda nontrivial, but mostly trivial) differences, to an application, WSL "appears" just like Linux....
If the docs work generates a great many small files, that’ll impose a substantial difference on Windows as compared to Linux or macOS – from observation, roughly an order of magnitude. The Windows FS is much slower at that specific kind of operation. (This is something I’m painfully familiar with because Ember.js’s build tooling is, well, an order of magnitude slower on Windows for precisely this reason.)
The numbers you’re seeing are still surprising to me even granting that, though: I wouldn’t expect that long even with that known performance issue.
Yeah, same. For builds, I expected something like 10s on WSL, not 25s average. I don't know about rustc too much, but why is it generating lots of small files? What purpose do those files serve? Can they be optimized out? Many compilers that target a similar audience as Rust (ada, for example) have much faster compilers on WSL and very fast compilers on windows, linux & macos native. Rustc is the first compiler I've seen (out of all GNU compilers and some others) that has this issue with WSL. So I'll guess that they can be optimized out in some way.
NOTE - the above is for the compiler. My bad. But docs is taking a crazy amount of time. 20 minutes is way too excessive.
OK status update as of now:
On WSL, updating and building is insanely slow
On Windows native, updating is insanely slow build building is good (could be faster, hello world still takes 3-5 seconds) - gcc installed through mingw on windows takes less time
Oh and lol the clickbait is working, so many views and replies XD This isn't clickbait tho lmao
I've experienced the same thing on Windows. The WSL filesystem implementation is very unoptimized right now, they've said as much.
For Windows native, the rust-docs installation is writing lots of small files, which is generally not as fast on Windows as it is on Linux, just due to differences in filesystem implementations. If you have antivirus in the mix as well, it's even slower. I have Cisco AMP for Endpoints on my work laptop (required by corporate policy) which scans every file access (drives me crazy) and rust-doc installation takes about 20 minutes (Core i7, 32GB, SSD).
EDIT: For comparison, my workstation at home (Core i7, 16GB, SSD) takes 1m25s to download and install stable (including docs) via rustup.
2 Likes
"Not as fast". Assuming that everything except for the filesystem is the same across windows and linux (if they aren't the difference should be small - the hard drive/SSD is the bottleneck here), that still doesn't explain why it takes 25 freaking minutes to install rust-docs. Unless Windows's file system is 16 times slower than linux's filesystem?
Here's an issue filed around slow rust docs installation: https://github.com/rust-lang-nursery/rustup.rs/issues/763. It's due to the anti-virus apparently.
Nope. With antivirus disabled, it's still insanely slow. JUST GIVE US AN OPTION TO NOT INSTALL RUST-DOCS ALREADY lmao.
Well, right now all I can say is that there's an issue for that as well 
: https://github.com/rust-lang-nursery/rustup.rs/issues/906
3 Likes
Just to be sure, are you saying you've disabled Defender for the WSL directory as in Speedup WSL by disabling Windows Real-time protection ?
I've tried both - I've disabled windows defender as shown in the link, and I've just disabled real time protection globally. Nothing causes significant speedups.
I did some testing of Rustup on Windows with ProcessMonitor capturing all file events. The creation of a single file in rust-docs requires 18 "syscalls":
8:56:27.9670280 AM rustup.exe 20964 CreateFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html NAME NOT FOUND Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
8:56:27.9675751 AM rustup.exe 20964 CreateFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: OverwriteIf, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult: Created
8:56:27.9684873 AM rustup.exe 20964 QuerySecurityFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Information: Attribute
8:56:27.9685802 AM rustup.exe 20964 WriteFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Offset: 0, Length: 7,420, Priority: Normal
8:56:27.9687324 AM rustup.exe 20964 CloseFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS
8:56:27.9790328 AM rustup.exe 20964 CreateFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
8:56:27.9793741 AM rustup.exe 20964 QuerySecurityFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Information: Attribute
8:56:27.9794153 AM rustup.exe 20964 SetBasicInformationFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS CreationTime: 0, LastAccessTime: 2018-11-04 3:13:44 PM, LastWriteTime: 2018-11-04 3:13:44 PM, ChangeTime: 0, FileAttributes: n/a
8:56:27.9795474 AM rustup.exe 20964 CloseFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS
8:56:27.9799556 AM rustup.exe 20964 CreateFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
8:56:27.9800764 AM rustup.exe 20964 QuerySecurityFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Information: Attribute
8:56:27.9801085 AM rustup.exe 20964 QueryInformationVolume C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html BUFFER OVERFLOW VolumeCreationTime: 2017-01-18 7:26:52 PM, VolumeSerialNumber: 28BB-82BA, SupportsObjects: True, VolumeLabel: WinՂ
8:56:27.9801349 AM rustup.exe 20964 QueryAllInformationFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html BUFFER OVERFLOW CreationTime: 2018-11-05 8:56:27 AM, LastAccessTime: 2018-11-04 3:13:44 PM, LastWriteTime: 2018-11-04 3:13:44 PM, ChangeTime: 2018-11-05 8:56:27 AM, FileAttributes: A, AllocationSize: 8,192, EndOfFile: 7,420, NumberOfLinks: 1, DeletePending: False, Directory: False, IndexNumber: 0x8000000202cb1, EaSize: 0, Access: Read Attributes, Synchronize, Position: 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Long
8:56:27.9801636 AM rustup.exe 20964 CloseFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS
8:56:27.9804884 AM rustup.exe 20964 CreateFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Desired Access: Write Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
8:56:27.9806919 AM rustup.exe 20964 QuerySecurityFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS Information: Attribute
8:56:27.9807270 AM rustup.exe 20964 SetBasicInformationFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS CreationTime: 0, LastAccessTime: 0, LastWriteTime: 0, ChangeTime: 0, FileAttributes: AN
8:56:27.9807689 AM rustup.exe 20964 CloseFile C:\Users\gordon.tyler\.rustup\tmp\tnchg8cf2r9owb5i_dir\rust-docs\share\doc\rust\html\std\macro.vec.html SUCCESS
Each of these calls is probably being intercepted by AV, which just magnifies the problem.
2 Likes
Nope. As I said, I globally disabled the antivirus. Still doesn't fix the problem. But nice job getting the log!
I too like using WSL. I did the update to 1.30 just now to check. Rust docs takes 10x as long to install as rustc or anything else. The first half involved some disk active times, but not more than 40MB/s. The CPU was running at 33% the whole time (corresponding to using one thread on the quad core plus overhead). That suggests that some kind of inefficient decompression is taking up most of the time?
It's strange to see only one thread being utilised during the installation process. Is it single threaded?
At one point it was debated whether the docs should even be installed by default. I went back and forth with how I felt about it, but for Windows users there is definitely more weight towards considering not including it by default.
Needing 5x CreateFile for each file definitely sounds bug-worthy...
1 Like
I know Microsoft are still working on the performance of the WSL filesystem, they acknowledge that it's way too slow. Personally I use Rust for Windows when I'm working on Windows, and run proper Linux Rust either in a Hyper-V VM or on my dual-boot laptop. When I excluded the .rustup and .cargo directories from Defender, installing the 1.31-beta4 win-x64 release via rustup seemed to be significantly faster. I know that doesn't help you, but this thread pointed me in the direction of setting those exclusions, so thanks!