Why Did Rust Actix Come in Second?

I noticed that Actix came second, after Drogon C++ Framework.

Rust over-performed C++ when it comes to JSON, 1-query and plaintext responses.

But C++ came first in serving 20-query and fortune responses, as well as updates.

In particular, Actix's 20-query responses and updates are quite lower, compared to Drogon's.

Is there any particular reason why? - and what can be done to get the top again, in your opinion?

Source: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=composite

2 Likes

It's probably because the original author of actix-web — who was super focused on winning these benchmarks — has left.

6 Likes

Personally I don't think those few percent difference make a difference worth losing sleep over. For what I do the speed of the web frame work is almost certainly not the bottleneck. I suspect this is commonly the case.

Could we have a similar set of charts indicating the number of bugs and security vulnerabilities in the various offerings?

11 Likes

I think it's that coupled with the fact that the people who replaced him as maintainers seem to be a lot more focused on removing unsafe code from the actix code base, whereas the original maintainer considered unsafe code fine if he thought it was both sound and more performant than the safe alternative.

However, when I think about the fact that I could figure out how to make my actix server compress all its responses with DEFLATE just by looking at the types and docs, and in less than 10 minutes from idea to commit, that is so amazing that it absolutely trumps a few % difference in performance for me.

5 Likes

Please: Focus on facts - not myth!

That Actix was "fast because of unsafe code" and that it is "optimized for benchmarks" are assumptions that people have started on Reddit, and which never have been proven by any detailed audits or benchmarks. Bringing those things up here again does not help the discussion.

If someone is interested in why Actix got slower, try to repro the setup using the 2 different tested commits, and see whether it really got slower (or whether Drogon got faster). If you discovered it got slower, add tracing to see in which area. Then try to improve that area.

The ones where Actix wins don't use a database, the other don't. Therefore it seems like the DB integration in the C++ project performs better.

11 Likes

I'm not suggesting Actix got slower, and I don't want to even touch the topic of unsafe code.

Let's be clear though that original author has explicitly stated that he pays attention to that benchmark and worked to make actix #1. He had a goal, worked towards it, and got results. That's orthogonal to everything else about safety and general usability of actix-web. We'll see if the current maintainers take a note of actix dropping to #2 and "fix" that.

20 Likes

I think people lost sight of the forest for the trees when it comes to Actix.

What should have been done before the controversy came to a head is to fork the codebase to actix-web-safe or actix-web-lts. Whether or not it’s theoretically necessary, people’s time is finite and the original development strategy was working for getting top performance out of it, even over far more established frameworks. That’s what attracted so much attention to it.

I’m not sure if that’s a viable strategy anymore, but that’s how I can see people having their cake and eating it too.

Someone may have a good argument against this, but I’d expect Rust with some unsafe blocks here and there is still better than an entire codebase of unsafe C. Even if there is a safe way to remove the unsafe blocks in Rust, and the unsafe way was what you had time for.

Perhaps the question that should be asked for this project in particular is why there seems to be a zero-sum game between performance and safety. Are unsafe solutions more discoverable, or are safe solutions vastly more difficult, or is there some theoretically counterintuitive performance cost in this practical (well as practical as benchmarks get) situation? Those aren’t literal questions, just prompts for additional study...

3 Likes

We were using Actix in our company because at the time (1 yer ago) it seemed like the most developed and feature-packed framework. Plus Actors, I really like Actor model to write performant concurrent code.

@spease You mentioned the are more established frameworks. Apart from Rocket, which I am already aware of, which ones are you thinking about?

2 Likes

Take a look on www.arewewebyet.org

There is Gotham, Iron, Nickel, ...

I was actually referring to the other contenders here:

The fact that Rust was shooting to the top of the list was getting it attention from a wider audience than just the Rust community. Sure benchmarks don’t represent production usage, but I think everybody on that list is pandering to the benchmark in one way or another.

If you want to grow the language and get adoption, that’s the kind of metric that matters. I’ve heard people claim that really performant Java is also non-idiomatic and bypasses the garbage collector by implementing memory allocation within the application itself (eg with a giant array). I don’t think people really consider that sort of thing when they’re looking at the metrics, they just see it as a theoretical high-bound for performance of the language and framework.

2 Likes

The original author of actix-web still helps out with benchmarks in techempower. I think the case is drogon gets faster but I believe the bottleneck may be in the database side on multiple queries. Even may-minihttp (rust) is getting a lot faster and came third this time.

3 Likes

Probably preaching to the choir, but the idea of writing anything but the most trivial web app server in C++ without the compile-time checks from Rust (or any bounds checking GC-ed language for that matter) honestly freaks me out. There's a reason folks back in the day chose Perl over C for cgi_bin scripts, and it certainly wasn't because Perl was faster or more memory efficient. The one big unsafe block that is C++ just seems like too big a risk, benchmarks or no benchmarks. Good for a performance target that others use to evaluate themselves, but as an actual development platform? No, thank you.

2 Likes

Yep.

So all the web jumped on PHP. Perhaps that did not crash your server but it's was an ongoing security problem ever since.