Game dev in Rust - some notes on the mess

community
22

I was playing Ark: Survival Ascended [1] the other day and it crashed [2] three times on me in a single play session. It also tops out at about 45 FPS at 1440p on an RTX 3090 + Ryzen 9 5900X. Clearly playing loose with memory safety is a cause for concern for game developers and players alike, and eschewing memory safety isn't a magic bullet for performance.

Personally, I would prefer a 10% perf hit (40 FPS) for a memory safe implementation that doesn't crash. And if I had to generalize, I think it would also be an acceptable tradeoff for everyone else who just wants to play games instead of deal with constant unnecessary interruptions.

The problem also isn't with shipping quickly. Games take a lot of time to build. The real problem being highlighted in this hot Internet take is that a certain subset of game devs are struggling with complacency.

And these development timelines are not outliers. This is just the nature of big projects regardless of the industry or product. That is to say it's a matter of scale. Nothing specific to games. The rest of this thread goes into some interesting details which serve as examples of why big projects take a long time. The bigger the project, the more of these details there are to work out. Sometimes you can leverage someone else's time (UnrealEngine) because they've already worked out many of the same details.

On the other hand, I can empathize with these frustrations. As an early adopter of bevy, I found it far too immature for making a large-scale game. To the point where I abandoned the entire game 2 years ago, after 3 months in development. This was a good "fail fast" experience for me, and I didn't trap myself into any kind of sunk cost bias.

I picked Godot/gdnative for my next project. And while it fixed some of the issues I had with bevy, I quickly discovered that Godot 3 is also not suitable for a large-scale game. More mature toolchain, different set of problems blocking the same goal. Godot 4 looks better still, but also needs work with e.g. reversed depth to support large sprawling scenes. I have not yet tried to port my old game to it.

I would argue that game dev in Rust is not any messier than game dev in C++ (it's actually much cleaner!) or even compared to other available game engines. You just trade one set of problems for another set of problems when migrating between any of these.

  1. I'm picking on Ark here because it's a recent example that is fresh in my mind, but these instability problems plague gaming. If even half of them are caused by memory safety problems, that's a statistically significant proportion which could be avoided with a memory safe language. (The data suggests the number of security vulnerabilities caused by memory safety violations is between 40%-70%. But that's a different metric from stability bugs, and I don't know what the stats are for that.) ↩︎

  2. This reddit thread is not mine, but it has a screenshot of the crash reporter. ↩︎

23

I have that same outlook, and while memory safety is usually whats brought up when talking about Rust, this quote of the week to me describes what to me is really its greatest safety feature:

24

Technically only a candidate for QOTW that didn't get chosen.

25

I don't know much about 3D in Rust. But I have to say kudos for what you've done. It's cool stuff! Maybe worth writing a series of blogs for documenting / future references, and very possibly inspire others to join force in this area.

EDIT: looks like you've already written "3D Rust - the missing intro". Nice!

26

The tools I listed do roughly the right things. They're just not solid enough yet.

27

Why not? Language and day to day game dev are different things. I think it is crazy to suggest 3D or 2D game dev needs Rust, same goes for it needs C++, Zig, Odin, C, C#, Objective-C and so on.

The most useful tool is the one you know how to use and have readily available. Game dev does not discriminate.

A language will not make you a game developer, not 2D or 3D.

Making games is hard regardless of the language. It has unique problems that a simple choice of language will not solve. A C++ game can perform horribly, very easy to do, hard to make performant.

28

Funnily enough, after reading this post the day before yesterday I started getting crashes in Cyberpunk 2077 with error messages like this one (this one is verbatim):

Error reason: Unhandled exception
Expression: EXCEPTION_ACCESS_VIOLATION (0xC0000005)
Message: The thread attempted to read inaccessible data at 0xFFFFFFFFFFFFFFFF.
File: <Unknown>(0)

Imagine if there was some system to prevent that.

29

Thanks for a good laugh. I would not call that genius, but rather I call it poor decision making. He is not going to choose Rust, he has and makes that clear. Jai is the language and his new unreleased game, game engine, and language remain to be proven. There is no magic he is performing by going down that long road. Other game developers have done it. It is not pragmatic and is selfish, about control. If it makes him happy, then great, but that in no way is special.

30

That is a bit harsh is it not? What is selfish about somebody spending their own time an resources creating software for themselves the way they want to?

If such people did not do such things then Graydon Hoare would never have started on Rust. Bjarne Stroustrup would not have started C++, Gary Kildal would not have created PL/M, Dennis Ritchie would not have started C, etc, etc, etc.

As for "control", I don't know. He has a vision of how he wants it to be. He wants to make it like that. He does not want hundreds of chefs spoiling the broth. Sounds reasonable to me. By contrast C++ is totally out of control and an ever growing mess. A fate which I hope never comes to Rust.

Perhaps there is nothing magic or special about what Johnathan is doing. Perhaps not genius. After all people have created languages and games before. No need to bash on it though.

Johnathan does have some negative views of open source for various reasons that don't add up to me though.

31

Thanks ZiCog. Just to be clear I don't mean selfish, like he is a bad person. I mean it "lacking consideration for others" as defined, he looked around and decided nothing fit his needs, or so he perceived, so he went to building it. Maybe that is a good decision, but I'm not convinced. It certainly was very costly, maybe too high a price to pay.

32

I think the issue is this:

To the best of my knowledge, Jonathan Blow has sold more games than all of us combined. If I am wrong, please correct me.

Thus, who are we to judge his technical decisions regarding game programming language design. :slight_smile:

33

If you want to read about his language, here is a primer. You don't have to watch hours of videos.

It's basically yet another improved C. Unusual features:

  • Any code can be run at compile time. This is useful for games which do a lot of preprocessing. Games tend to have very elaborate content preparation. Take a look at Unreal Editor. It also lets the program be its own build system.
  • Good facilities for cache locality. One keyword can turn an array of structs into a struct of arrays.
  • Data types for 2, 3, and 4D vectors and matrices are built in. It's a huge headache when this isn't standardized. I'm currently trying to get the author of the "glam" crate to advance to version 1.x.x, because too many graphics crates are tied to specific versions of "glam", and don't play well together.
  • Consistent syntax for functions, lambdas, and internal functions.
  • Enough data is stored for full reflection.
  • Yet another approach to polymorphism.

That's most of it. Nothing exotic there. No improvements on the safety front. Just convenience features.

34

I do not understand the point you are making. The point I am making is:

If the point of Game Dev is delivering entertainment value to end users, Jonathan Blow has more credentials than most (all?) of us here combined. As such, who are we to judge his decisions regarding Jai. We argue from this theory and that theory, this opinion and that opinion. JB actually delivered. It's like someone playing boxing-simulator-video-game and then criticizing Mike Tyson's training routine.

35

I can imagine that Johnathan as an old school C/C++ programmer does not prioritise memory safety as much as many of us might.

He does value simplicity, small size, and fast compilation speed.

I don't get the "just convenience features" statement. Convenience is the entire reason for being of high level programming languages. Else we would all still be writing our code in binary instructions like the good old days.

36

IIRC, there was a live demo where Jai was compiling 100k loc FROM SCRATCH in < 1s. I do not recall if it was 1 core or multicore.

37

I'm pretty sure Mike Tyson's training routine wouldn't help him to win in a boxing-simulator-video-game. Worse: if you say that boxing-simulator-video-game is poor simulation for real world fight then I would point out that any half-decently trained marine would turn Mike Tyson's into pile of blood and bones before Mike Tyson would even be able to blink. Sure, he would use his trusty M27 for that, not his fists, but why is that an issue?

Similarly with gamedev: sure Jonathan Blow is very persistent guy and he did things that made him very well-known, but… what exactly have he invented that made him a genius who may alter course of game industry? Bill Budge who was my teammate a few years back wasn't arrogant enough to [try to] make yet-another-language-which-would-be-forgotten-soon.

Heck, I'm not even doing any gamedev but at my $DAYJOB I fixed bug that prevented game with 100 million active users from being available on some obscure platform and that act, alone, enabled more users than buyers for all Jonathan Blow games, combined. Does it make me genius? No, but it gives me more isight into the way games are done today than Jonathan Blow would ever have.

It's like comparing McDonalds logistic to work of three star restaraunt chef. These are entirely different industries and the requirements they put on participants are not even remotely similar.

The last thing I would do in the topicstarters place is to ask Jonathan Blow for advice, sorry. In fact as sceptical as I am about @John_Nagle work if you ask me who have a larger chance to affect the gamedev I would say it's not even a contest: chances that Jonathan Blow would change the gamedev industry is precisely zero while @John_Nagle does have [a small] chance.

steffahn 38

Let’s try not to take the discussion too far off-topic. Between Jon Blow’s achievements or standing in game development, and training routines or McDonalds logistics, none of this seems particularly related to game development in Rust anymore :wink:

14 Likes
39

I think this is amazing what you've accomplished so far, and that you should keep at it. I can tell you're a bit disillusioned, but tbh I don't think the "meta verse" came and went, I think zuck tried to usher it in way too quickly just like second life in the 90s.

I think you're right, and the tech in general just isn't there. Honestly, if you keep at this you could be the first to market (albeit with a lot of time and effort) with an updated game tech stack that may actually allow a more proper large, open, interactive, virtual world.

I'm very interested in game dev in rust. I'm very interested in helping do the low level work.. I'm just not that experienced yet but I think I see the potential.

Without the people like you in the trenches the transition to game dev in rust will be even slower. I know it's frustrating, but for ideas like this you gotta be in it for the long haul. Best of luck! I hope you continue with it.

40

As a fan of Stephenson's Snow Crash, I see Zuckerberg's adoption of the moniker as indicative of the lasting staying power of the original vision. To see us endeavoring to make the metaverse a reality more than 30 years after reading it in science fiction, I would say it is premature to conclude that it "came and went".

Likewise, as a devoted fan of this language, I have seen @John_Nagle posting on this forum for years. When I was a raw beginner, he was pushing the bleeding edge of Rust 3D. Now I am a seasoned... beginner. I can no more contribute to the Rust 3D ecosystem than when I first signed on 5+ years ago. However, I can now use Rust in my day job to make small, practical CLI applications that make me more productive at my duties. The progress OP has made in an equivalent time seems to bury my own, and their work seems pretty amazing. I know I have a bias to overestimate how much I can accomplish in a given time frame, and feel frustrated when I fall short.

Some of the frustration that is dawning on OP may be a reflection of the fact that John is a "leader in the field". The principal cause of concern appears to be a feeling of bait and switch between the hype that Rust would be an exciting field for game development, versus the reality that the ecosystem is still immature. I think the big takeaway from his experience might be "don't believe the hype".

41

I'm not really a graphics programmer. I've never written a shader. I've never used anything more than basic OpenGL graphics in other projects. The crates I'm using are doing the 3D graphics. I rely on them. Most of my discussion here is about where those crates are strong, and where they are weak.

A metaverse client is halfway between a web browser and a game engine, and has some of the problems of both. Plus time constraints. It's getting a continuous stream of short messages over UDP telling it to do things. It makes requests of asset servers for content. The UDP messages tell it what to ask for. It's moving objects on screen, following velocity info from the server. The "game" part is all server side. Movement, physics, etc. are either done server side or done by the client following instructions from the server. All this is asynchronous. It's not frame-oriented, like most games. This is about 80% data and network management, and only 20% graphics.

Big open-world MMO games have many of these problems, so this isn't unknown territory. A metaverse of user-created content is a less controlled environment than a MMO. There's more content and less instancing. My observation is that a metaverse requires about 3x the GPU memory for that reason. Second Life has over 50,000 different user-created chairs, and about 3 petabytes of content on the asset servers, for a world the size of Los Angeles. Some of the content has errors, and a small fraction is hostile. Client side validation is necessary. That's much like web content.

This is where Rust helps. Wrangling all the data with good performance requires several CPUs working hard, doing totally different things in a coordinated way. Trying to make all that work in C++ would be really hard. In safe Rust (I don't use "unsafe" in my own code at all) it's not bad.

The existing C++ clients are async-oriented. Lots of internal co-routines, all stealing time from refresh. So, the frame rate suffered whenever a lot was happening. The point of this Rust viewer is to get all the updating off the critical path of the refresh thread and maintain a constant frame rate.

Here's a list of the main threads in my own program:

  • Refresh. Draw, draw, repeat. Do as little else as possible. Highest priority.
  • Per-frame update. Runs in parallel with rendering, locked to the frame cycle, mostly moving objects which are in continuous motion. This is a trick borrowed from Unreal Engine to get all those little move updated out of the refresh thread. Locks individual objects, not the whole object tree.
  • Event processing. Fed by UDP messages and some other sources. Locks the object tree as needed. Locks individual objects as needed.
  • Asset decoding threads, about 2x the number of CPUs. Run at low priority. Fetch content (meshes and textures) from servers. Decompress and preprocess content. Also locks the object tree for brief periods. Locks individual objects as needed. Loads content into the GPU in parallel with rendering.

That gives a sense of how the work is split across CPUs and the GPU.

I need the lower levels of graphics to Just Work, so I can work on the higher level problems. They almost do. Almost. That's why I'm writing this. If you try to use the graphics stack I've been discussing for a simple application, it will probably work. You might hit some problems. Typical problems: full screen doesn't work right on some platforms, filling up the GPU can cause crashes, etc. Many users will never hit those problems.

I encourage people to try. Do some 3D stuff. Load up some pre-built glTF scenes and walk through them to get started. Rend3's "scene-viewer" can load standard glTF scenes, and they look pretty good. You can make you own glTF scenes in Blender or Maya. So it's not that hard to get started on a full 3D game.