I downloaded the application described here (there's a link to his GitHub.
It's a .NET Core console application. I'm running on Windows 10. It is a concocted multithreaded application comparing C# and Rust execution speeds.
I'm aware it's non-deterministic, so timings will vary but why are both C# and Rust slower on their initial runs than subsequently? And why is Rust slower then C# initially and then 6-7 times faster subsequently?
I tried this exercise yesterday and today with similar results. E.g., in milliseconds
1st run
Execution time in C#: 32
Execution time in Rust: 45
Subsequent runs
Execution time in C#: 19
Execution time in Rust: 3
Is this just an artifact of Windows? Would similar happen on Linux or macOS?
No, this is a poorly designed benchmark and the discrepancy you are seeing is almost certainly a direct result of that.
First, you can't just run the function-under-test once. There's a strong chance that your times will be thrown completely off by factors external to the function (things running in the background, GC runs, the thread scheduler, etc.), so you should be repeating it several times and do a proper statistical analysis on the timings.
Related to the previous point, you need to run the function several times before you even start measuring. This allows the various caches on your computer (L1, OS pages, etc.) to properly "warm up" and make sure timing isn't thrown off because data/machine code is moved around at an inopportune moment.
C# is also a JIT-compiled language, meaning it often won't translate a function into executable machine code until just before it gets used for the first time. P/Invoke (what powers the DllImport attribute) requires doing a lot more work than just JIT-compiling some C# because it needs to find the RustLibrary.dll DLL, load it into memory, and figure out how it'll marshal arguments between the C# VM and native code. I wouldn't be surprised if subsequent runs were faster because the C# runtime was caching this work across subsequent runs of the program.
Also, your C# and Rust functions aren't identical, so you are actually comparing different things. The C# example emits one print at the end of each task but before the tasks are joined (Tasks.WaitAll(tasks)), while the Rust code prints the message when joining the threads. This difference in order may favour Rust over C# (e.g. because it allows more parallelism or better cache behaviour).
This wasn't my example. I wasn't especially concerned about the differences in implementation between C# and Rust. The question related to the differences in timings between the initial and subsequent runs and the similar timings in the initial run.
There's a strong chance that your times will be thrown completely off by factors external to the function (things running in the background, GC runs, the thread scheduler, etc.)
C# is also a JIT-compiled language, meaning it often won't translate a function into executable machine code until just before it gets used for the first time. P/Invoke (what powers the DllImport attribute) requires doing a lot more work than just JIT-compiling some C# because it needs to find the RustLibrary.dll DLL, load it into memory, and figure out how it'll marshal arguments between the C# VM and native code. I wouldn't be surprised if subsequent runs were faster because the C# runtime was caching this work across subsequent runs of the program.
Yes, those were the explanations I was looking for. Thanks.
