Say you're the CEO of a large corp, and a bunch of VPs report to you and you assign tasks to them. Say there are two types of VPs:
Type 1: At some point, this VP says: "too much work, I can't handle this new task"
Type 2: This VP always says "sure, I can do that", but, at some point, when overloaded, then just quit abruptly without warning.
We would both agree that Type 1 > Type 2. However, with my current knowledge of Rust, servers that I write tend to be of form Type 2 -- if, at any point, there are so many requests that we run out of memory -- the system just quits and dies. In my current code, I am never considering 'what if vector.push_back fails, what is hashmap.insert fails, what if FooBar::new() fails ... '. I do not think this is a problem the type checker / borrow checker can solve, because the programs are perfectly typed, it is just a matter of "$&@#*# out of memory"
Instead of 'sudden death', I would prefer a system that (1) handled as many requests as possible while sending out "hey, temporarily overloaded" to other requests
I believe the Erlang solution to this problem is that instead of having a few VPs, we have hundreds of thousands of cheap-to-spawn VVVVVVVVVVVPs assign them precisely one task, and if any crashes/overloads, just respawn a replacement.
I know that Rust has efforts like
but it seems to me, the fundamental issue is, the only solutions to this problem involves one of:
only using fixed amounts of memory -- some how write program in a way that, regardless of load, always uses some (large) fixed amount of memory
really really damn good memory monitoring / prediction -- i.e. somehow able to upper bound "I can guarantee, in the worst, case, handling these n additional requests will use at most M memory"
localize damage / crash oriented computing / cheap respawn -- i.e. some type of VM / virtual threads where virtual threads can independently crash without damaging the rest of the system, and really really fast/cheap respawning
some cool technique I am completely unaware of
If you have personal experience (or know of good books/blog posts) on building Rust systems that, under pressure, 'perform at peak throughout, then drop connections', instead of 'suddenly dying', I would love to hear more. Thanks!