I wrote a terse HTTP webserver using tokio. It’s hard for it be simpler (it cheats and doesn’t pay attention to request headers; and it doesn’t send a Date header back ). And yet, actix-web is outperforming it by a decent margin in benchmarks. Here’s the whole code:
The actix benchmark code from TechEmpower’s benchmark gets
50k req/s in my tests; my code gets
43k req/s. This is on cloud instances, 4096 keep alive connections,
wrk, 2 vCPUs per machine.
Can anyone think of where this difference could come from?
Because of the use of keep-alive connections I think that none of the performance difference is due to the way the listener is setup. So the fact that actix-web uses a different backlog setting and possibly multiple listeners doesn’t seem to matter (though I did try a few modifications of my code to test and didn’t see much difference). So it must be something else. actix appears to mostly use mio directly rather than tokio’s runtime and reactor. Does using tokio have that much overhead?
Here’s the full text of the benchmark results:
## actix, no pipeline: Running 15s test @ http://10.142.0.3:8080/plaintext 2 threads and 4096 connections Thread Stats Avg Stdev Max +/- Stdev Latency 10.09ms 505.14us 23.99ms 70.69% Req/Sec 50.47k 2.10k 55.84k 65.33% Latency Distribution 50% 10.15ms 75% 10.46ms 90% 10.63ms 99% 11.09ms 1506079 requests in 15.05s, 186.72MB read Socket errors: connect 3077, read 0, write 0, timeout 0 Requests/sec: 100059.44 Transfer/sec: 12.41MB ## tokio-raw, no pipeline: Running 15s test @ http://10.142.0.3:8080/plaintext 2 threads and 4096 connections Thread Stats Avg Stdev Max +/- Stdev Latency 11.52ms 7.57ms 237.87ms 63.74% Req/Sec 43.86k 3.55k 49.92k 62.67% Latency Distribution 50% 11.40ms 75% 17.27ms 90% 21.29ms 99% 23.96ms 1309184 requests in 15.05s, 97.39MB read Socket errors: connect 3077, read 0, write 0, timeout 0 Requests/sec: 86962.04 Transfer/sec: 6.47MB
You can also see that the latency distribution for actix is flat, while there are a number of slower requests for my code.
I’ve tested tokio-minihttp, hyper, and the HTTP example in the tokio repo. All perform even worse than my code, let alone actix.
TechEmpower’s benchmark confirms this: https://www.techempower.com/benchmarks/#section=data-r16&hw=cl&test=plaintext You can see that on Cloud hardware actix-raw outperforms tokio-minihttp. (NOTE: This is not the case on Physical hardware benchmarks, but I suspect that this is due to the hardware used in Physical tests being more powerful, thus the benchmarks hit the network bottleneck and so you can’t see their difference in performance anymore).
In practical terms, I’m not concerned about performance at these levels on artificial benchmarks. But I am curious as to why my exceedingly simple code is being outperformed by code that is doing an order of magnitude more work.