In a recent talk Andrew Kelley about why zig lang is more speed than Rust.
In his own words
I was going to say Iâm going to be Rust now. So for that one, Iâll circle back to the other example you brought up, which was self-hosting the Zig compiler and the speed-ups that it brought. So to be clear, the speed-ups that it brought are almost entirely from the fact that now I know what Iâm doing, right? And itâs the second version of something. Youâre always going to do better on the second version of something.
Thatâs where those speeds come from.
But I can tell you what I did, right? Now, what is the better thing than I did, and therein kind of lies the justification for my claim?
So what I did is I learned about data-oriented programming. I developed a better intuitive model about how the cache system of CPUs work. And the fundamental premise, or observation I should say, is that if you touch less memory, then the CPU cache will get to have more hits for the memory that you actually do touch. Thatâs the whole observation. So based on that observation, you can make a bunch of code changes and then your code gets faster.
So one of the ways that I did this in a self-hosted compiler was I found the place where we created a lot of objects on the heap and the objects that we were creating on the heap were IR instructions. So this is a part of a compiler that is an intermediate representation that you use to pass from one component of the compiler to another component of the compiler. You admit these intermediate instructions for the second component to consume. And itâs code, right? So youâre generating many of them in memory based on the code that the user types.
So based on the observation about CPUs and this knowledge, my hypothesis is, okay, if we make these objects take up less memory, then not only will that just use less memory in the compiler, which is good, it will reduce the pressure on the cache of the CPU and therefore make the code faster. And this turned out to be completely true and ended up being something like a 35% speed-up. But not only that but that pattern-
Yeah, it was huge. And that pattern also existed in three other places. So I got that big of a speed-up three other times for doing the same strategy.
Itâs always amazing to me the impact of increasing that cache hit rate or cache locality,
Itâs all about algorithms and whatever, but man, that is a substantial speed-up.
Yeah. But let me make my point with Rust though because I do have a way to tie it in.
So in order to do this optimization, this data-oriented design reworking, one of the core components is an untagged union. And so in Rust, these are enums, but theyâre tagged. There was a tag for this data structure, but you put it in a separate array and thatâs part of the strategy. So you have one array up here and itâs each one is a byte, and the byte says this is the kind of instructionâitâs like add, subtract, store, whatever. And then in a different array, each element contains the instructions data and theyâre a fixed size. So if you want to find out the tag, youâll look up index number one and the tag is in this array. If you want to find out the data, you go look in the other array, index one, thereâs the data. These are called instructive arrays.
And the point of this is that if you tried to put all this data in one array, you would be wasting seven bytes of padding because if you only need one byte for the tag but then you have like a pointer size field here, you have to have padding there. So you just put them in different arrays and then like poof, the padding goes away and thatâs better for the cache, right? Youâre not putting padding in the CPU cache.
Total waste. So, the problem is that you canât model this in Rust without unsafe. You have to use unsafe, which is fine, you could do unsafe, right? But no one wants to do that in Rust. No one wants to use unsafe because people come to yell at you and make you feel sad. So they donât.
Yeah. So people donât do it, right? In practice, people try to write safe Rust code, but you canât fully exploit the hardware of your computer is fully safe Rust code. So, in practice, these strategies are available to Zig programmers to write faster code. And hereâs the kicker, this code is safe in Zig because we have safety on untagged unions. Like we have both. Itâs safe
So this is an example. I mean, itâs one little use case. I mean, itâs not contrived, itâs a real use case. Itâs the self-host compiler, but itâs a use case where itâs safe in Zig, but if you tried to write this code in Rust, it would be unsafe or it would perform worse.