Game dev in Rust - some notes on the mess

community
1

Ref: Reddit - Dive into anything

For three years now, I've been developing a metaverse client (Second Life, Open Simulator) in Rust. Here's some demo video. This is high-detail, big-world, high-perfomance, highly concurrent 3D graphics - the kind of stuff Rust is supposed to be good for. Existing single-thread C++ implementations are too sluggish. The Rust implementation can fix that.

The trouble is, the graphics crate ecosystem still isn't ready for prime time. There was more enthusiasm for Rust game dev in 2020 than there is now. The last monthly update on the rust game dev web site was in July 2023. There are few posts on Reddit any more. I have one of the very few high-end 3D projects in Rust.

Trying to get the ecosystem working is like chopping a path through the jungle with a machete. I write programs to reproduce problems, file issues on Github, talk to developers, and gradually get things fixed. Over half my time goes into dealing with ecosystem bugs. Development is just too slow. The whole metaverse era came and went while I was fighting with the Rust ecosystem.

We either need more people doing serious 3D in Rust, or we need to admit that 3D game work in Rust was a flop.

Comments?

15 Likes
2

So we are firmly in the Trough of Disillusionment stage?

That's fine and kinda expected.

Yes, but it wouldn't be good as long as foundations are not there. And game industry is one of the most conservative industries that exist.

Games were using MS-DOS for years after everyone else switched to Windows, for crying out loud! Remember the history? Quake in 1997, Grand Theft Auto in 1998… all MS-DOS games in the initial release.

Sure it does some crazy and innovative things, too, but many if these are to stay in the past! Like use of unreal mode to squeeze advanced game in the confines of MS-DOS.

That's why it's hard to predict what would happen in the future: unlike switch to Windows, where Microsoft was emphatically very interested in moving gamedev to that platform Rust doesn't have any “stick”. And “stick”, in some form, is the only way to move game industry anywhere.

Games haven't adopted Windows because it was better than MS-DOS, Games adopted Windows because most 3D cards weren't supported in MS-DOS!

And I don't really see what may push them into Rust realm today. More likely they would continue to stick with C++ till it would be abandoned by the rest of the industry and only then would upgrade, kicking and screaming.

It would take years, if not decades.

Nope. It wasn't a flop and games would arrive there. They just wouldn't arrive there before the rest of the industry.

Heck, can you name anything that game industry abandoned quickly? Not “added to the pile of hacks that they already collected over time”, but abandoned?

I couldn't recall anything.

And switch from C/C++ to Rust is very much abandonment of years of investment into C/C++ engines.

This wouldn't happen quickly.

20 Likes
3

My main comment is that you obviously did the actual work, and therefore it would be foolish to discount your experiences.

That said, I think the "problem" is that you're an early adopter in something where some of the foundations are still being built. Things will get better, but it sucks that you had suffer the productivity cost of being early adopter.

16 Likes
4

I'm three years into being an "early adopter". That's too long. The entire metaverse thing came and went while I was struggling with Rust crates.

4 Likes
5

May I ask why you did it in the first place?

2 Likes
6

Hmm... As I remember it there was no point in games moving to Windows for a long time as it offers no advantages and only wasted memory in the machine. Memory was very expensive then. Windows was not even 32 bit till Windows 95 and that ran on top of DOS like a game itself.

Not until 3D graphics cards hit the consumer market, with their Windows drivers, was their benefit in games moving to Windows. The first was the 3DFX Voodoo in 1996/7 as I recall. It was a glorious day when I first got my hands on one of those. The drop in memory price helped as well I guess.

All in all it seemed to me that games followed the development of Windows into the 32 bit and accelerated 3D world, which was not so quick.

When it comes to Rust, I'm not sure one should expect the modern world of games engines, VR, whatever to appear any time soon. My understanding is that these things are huge, they have been developed in C++ over many years. Not much chance of all that getting overthrown and written in Rust in a hurry.

Of course there are the geniuses like Johnathan Blow who spend a decade building a game including their own game engine. (He is not likely to likely to adopt Rust as he is now building his own language as well) They are few and far between.

5 Likes
7

I don't think Rust will really shine in that space any time soon, some external factor would probably need to happen in order for the industry start paying attention to it, not to mention, at least on the internet I see game devs often sneering at Rust, as they claim that game dev requires playing loose with safety in order to gain performance and to ship things faster.

But then there is the matter of the graphics ecosystem, which is not necessarily a game thing and I really see no reason why it wouldn't thrive for other reasons.

I'm not too familiar with these things though, so this is just conjecture.

1 Like
9

Why? Bored during COVID lockdown, actually. Then it started working. Here's two years ago:

Looks nice, but sluggish, no lighting, no normals, static scene, no dynamic content loading.
Looked like it was going to work. So I kept at it.

Technically, here's the problem I'm working on:

  1. High-detail shared virtual world with good graphics.
  2. Seamless big world, not little rooms with portals.
  3. User-created content which can be changed in the live world.
  4. Consistently high frame rate. (60 FPS or better.)

This is the base feature set for the "Ready Player One" / "Snow Crash" metaverse.

Many systems have three of the four. Nobody has all four. Second Life and Open Simulator have 1), 2), and 3), but the client is single-threaded C++ and does too little on the GPU, so it's sluggish and can't do 4) without reducing the draw distance and giving up 2). There are lots of systems where you have little closed worlds, maybe with portals, but no big landscapes. Sinespace/Breakroom are good examples. If you reduce the detail to cartoon level or below, you can get 2), 3), and 4). That's Decentraland and CryptoVoxels. If you preprocess all the content and do a lot of optimization during game development, you can make a good AAA title, but you give up 3).

User-created content means far less instancing, because each creator makes their own stuff. So you need to have a lot more content live to display a scene. It's about 3x over a commercial game, in practice. If you look closely at, say, Cyberpunk 2077, you'll see the same railing showing up all over the place. In Second Life, there are over 50,000 different chairs. So you need about 3x the GPU memory. And you need a lot of network bandwidth for dynamically loading content. Both are available today, and most Steam users have them.

To juggle all that data, you need heavy concurrency. You really need all those CPUs a modern desktop has. This is where Rust shines. If you try to do something this concurrent in C++, it's really hard. The Unity dev team had a big group working for years on that. Unreal Engine depends heavily on preprocessing and optimization of the content in Unreal Editor, which is not compatible with dynamically created user-created content. (Tim Sweeny, CEO of Unreal, was talking about supporting all that a few years ago, but we're not hearing that any more.)

The Rust ecosystem has most of the necessary crates. The trouble is that they're nowhere near finished. I've accomplished a lot, but, as I said, half my time goes into fixing or working around problems of the ecosystem.

A few more reasonably competent game devs using the same stack, so that this stuff gets pushed to the point where it Just Works, would bring the 3D crates up to a usable level.

At the bottom is Vulkan. That's maintained by Kronos and GPU makers for different platforms, and that's pretty solid. Just to be different, Apple has Metal.

Atop this, in the Rust world, we have WGPU. That's a cross-platform graphics system. So cross platform it does phones and web as well as desktop and laptop. Using WGPU, you still have to manage the buffer space and data structure allocation in the GPU yourself, which is a huge headache. But at least you get portability.

Atop WGPU, we have Rend3.
The Rend3 level provides management of all the graphics objects. The application creates meshes, 2D textures, materials, and objects, and Rend3 takes care of getting them into the GPU and displaying them. It's about at the level of classic OpenGL. It's all "retained mode" - you load content into the GPU and it goes, with minimal help from a CPU. That's how you get good performance.

Rend3 is basically a library - you call it. (Well, there's a "framework", to manage refreshing, and it owns the event loop, but you don't have to use it.) There's also Bevy, which is a full game engine, but an unfinished one undergoing a lot of churn.

Other key components are winit, the cross-platform window manager, and egui, which is a 2D overlay for menus and dialogs. They're mostly OK. Egui requires a lot of code for each GUI widget. It needs some kind of tool to generate widgets from JSON or XML or something.

All these crates are tightly coupled. All advance in version lockstep. So, every month or two, there's a big upgrade, it's necessary to deal with the API churn, and it's time to debug stuff that used to work. None of this stuff is documented beyond Rustdoc.

I never wanted to be on the cutting edge of Rust 3D development. I'm trying to solve a new problem, using what I though was a more mature ecosystem already being using by game devs. That's what the Rust hype said. That hasn't been updated since July 2023, by the way. I needed Rust because I knew this was going to need a lot of CPUs working hard to be fast enough. Rust does that part reasonably well.

I have a whole series of other problems to solve, related to that four-item list above. There's high-performance concurrent content wrangling, LOD change management, seamless crossings between regions, automatic impostor generation ... That's what I need to be working on, not fighting the low-level stuff.

If this stack had a few more projects using it for reasonably hard 3D work, most of the dirty laundry problems would get fixed. Most Rust game dev is simple 2D stuff, for which you don't really need Rust.
A few people have done 3D work in Rust, but they often use some C++ engine (DX11, Unity, etc.) for the graphics, so they can actually ship a product. Few are using the whole stack described here.

That's the problem. Comments?

15 Likes
10

Given you've spent 3 years on this, have you considered just writing your own shaders directly on wgpu ?

11

It's not shaders that are the problem. What I get from WGPU and (soon) Rend3 is safe concurrent updates to the GPU content. I'm dealing with a flood of incoming content, and the updates slow the frame rate if the content can't go from network to GPU without bothering the rendering thread too much.

It took the WGPU people 9 months to get the locking right, and now Rend3 is being updated to use it properly. The Rend3 API is already ready. You add a mesh or texture and get a handle back. Any thread can do this, and my application already does that. But most of the copying work is currently queued for the main thread. That's being fixed.

I'd have to write all that if I used WGPU directly.

3 Likes
12

Is this the hype you're referring to?:

While the ecosystem is still very young, you can find enough libraries and game engines to sink your teeth into doing some slightly experimental gamedev.

My reaction when reading this is that it sounds immature. It is probably trying to sound hopeful as well, but that's something that can't be relied on. Are you sure you weren't overly optimistic yourself?

In general I think your posts here are very thoughtful and could inspire others to help with the missing pieces. I hope it works out!

8 Likes
13

TBH, my first reaction here is that there might be a reason it's not been done. If everyone can build and then take down a high-detail CN Tower (1) on their own land (3), the LoDing and culling and such for (2) means that I'm really skeptical that (4) is feasible.

Distinct areas with portals seems like the logical solution for the problem space, when arbitrary custom content is allowed. (Yes, sometimes it's a hack with poor player experience -- Starfield -- when the content is under one team's control. But budgeting between unrelated players that's fair but not punishingly low is really, really hard.)

"Player-customizable shared-by-everyone durable world" is one of those nirvanas that just never seems to work out well, as endless MMOs encounter. As in any Common-Pool Resource problem, the bigger the number of players the more likely that one will cause issues impacting the others.

1 Like
14

My first reaction to this is if it's possible, it's by doing things at the extreme bleeding edge of game engine tech: GPU driven rendering, an equivalent to the UE5+ feature set (Nanite, Lumen, VSM, unified shader model, etc), very careful cascading of fallbacks for shadows and reflections, etc. Things that are so far past what's provided by most any commercial engine and even when they are present with dedicated effort in constrained content they can't consistently hit 60.

Could Rust be helpful to build something like that? Maybe, but I'd also want to have a few hundred devs and the 5-10 years Epic took for UE5.

If you prove me wrong, amazing! I'd love to be proven wrong, but this seems like the exact definition of unrealistic expectations right now.

9 Likes
15

That's the conventional wisdom. It's wrong. I'm far enough along to have gotten past many of those problems. Here's a short version of how it was done.

  • The Second Life / Open Simulator world are divided into squares 256m on a side. (Open Simulator allows larger regions, at a cost in items per unit area.) Displaying the nearest four such regions at a reasonably high level of detail and frame rate can be made to work. See the videos I linked. It takes a GPU in the 6 GB - 8GB range. 8GB is now NVidia's entry-level gamer graphics card.
  • Most GPU space is used by textures, not meshes. So textures have to be frantically resized and swapped in and out of GPU memory to maintain roughly one texture pixel per screen pixel. All that is working in the videos. Did you notice? (This is not mip-mapping. That's a rendering speed optimization, but doesn't help with GPU memory.) Keeps a few CPUs busy, mostly doing JPEG 2000 decompression. Currently, this impacts frame rate, but once the Rend3/WGPU/Vulkan stack mods underway are complete (WGPU is done, Rend3 in progress) the threads doing that work should not be generating render thread work as they do now.
    All this updating needs to be prioritized based on how much screen space is being occupied by an object. As the camera moves, priorities change. So a priority queue system with efficient priority changing is needed. That's the Rust crate priority-queue.
  • Beyond the four nearby regions, we're going to need region-sized impostors. Many games do this, but they typically hand-build the impostors. Take a really close look at GTA V, and you'll see the transitions to 3D impostors at 200m-300m from the camera. For a user-modifiable world, we need automation. Fortunately, the needed technology has been developed by the geospatial people. I'm looking into using Open Drone Map to construct those. That takes in still pictures and generates textured 3D meshes, like Google Earth. Such impostors would be reconstructed by a server once a week or so, maybe more often if a region gets a major overhaul. This is still at the design stage with some simple tests.

There's more, but that's the big stuff. It's all conventional technology applied in new ways.

If you tried to do this in C++, you'd need a large team. Keeping the pointers straight and fixing the race conditions would be tough. Rust is a big win here. One group I know of started a C++ project to do texture loading with dynamic priority, and they gave up. Some failed code exists. In Rust, even though it is fairly complicated, it works reliably. The "fearless concurrency" thing is quite real, although few push it all that hard.

So it can be done. Now if we can just get more of the low-level dirty laundry in a wide range of crates fixed...

13 Likes
16

Another reason to get Rust's 3D act together. Unity just had a big layoff. 1500 people. Many smaller Unity offices closing.

Unity used to be the go-to engine for projects below AAA title level. Easy to use, good terms. Then Unity changed their pricing to make more money, which annoyed many smaller game devs. Now this.

Epic had an 800-person layoff last year. Apparently Fortnite revenue isn't enough to heavily fund the UE engine dev any more.

So there's a window of opportunity for good open source game engines.

1 Like
17

If I understand the examples you have raised correctly:

  1. You have shown that there is a significant gap between Unreal Engine vs open source Rust engines.

  2. You have NOT shown a significant gap between open source NON-RUST engines vs open-source RUST engines.

So this really seems like an issue of "UE beats all open source projects" (the reason the company exists) and less an issue of "Rust game dev is behind other open source engines"

18

Very cool project!

But other languages like C++ have a longer history then Rust and it's used by a lot of big gaming studios.

For someone to take Rust to the same level, wouldn't there be a necessary incentive to do it? Either a few of the big players in game development need to see a big advantage in switching from e.g C++ to Rust. Or someone with a lot of time and passion for gaming would have to write something coming close to the current adopted options.

I think it is just a matter of time? I don't see how in theory Rust couldn't be equally good or even better then the languages used today in gaming. So I don't see how, because it isn't there yet, you'd already label it as failed.

19

I know it is a bit off topic, and it doesn't help you, but as an early visitor to SecondLife in 2003 I would like to pay my respects to you. I didn't know it even still existed, but seeing your demo video made me recognize it immediately. I am sorry for your frustration with Rust and/or this project, but I salute you nevertheless!

2 Likes
20

While I have a great deal of respect and appreciation for the incredible technical work that you've done on this, I also think you may be making a fundamental error of kind when you advocate.

Unity, the game engine, is the product of a single incorporated entity, built as a business proposition: "if we make a high-quality engine and attract developers, we can profit from it in a number of ways." We can see that thesis playing out in Unity's licensing choices, in the asset marketplace, in the pricing of Unity itself, and in numerous other ways. It has clear goals (nontechnical though they may be), and can be evaluated as to how it meets or fails to meet those goals.

Rust Gaming, as you describe things, is very much not that. It is a collective and largely inadvertant outcome, consisting of a very non-uniform collection of tools and libraries, made by individuals and unrelated teams, each working on smaller, more local problems to meet their own goals. There is nobody setting a vision for Rust Gaming generally, nor is there a single consensus on what the mission of Rust Gaming should be; those indivudals and teams, instead, have their own goals, and will pull in their own directions.

It is reasonable to suggest that there is an opportunity for Rust as a community in Unity's layoffs. That's a good observation. I think it's a mistake to characterize that opportunity as a mandate, or to characterize missing that opportunity as a failure. It can only be a failure if it leads to a goal being missed, and there's no consensus that absorbing those developers serves a goal.

Neither kind of thing - a single business making a product in order to raise revenue, or a community making tools to meet idiosyncratic and local needs - is better than the other. All I'm saying is that they are different, and that advocating for one to behave like the other requires either overlooking, or eliminating, those differences. Personally, I would not want any part of the Rust community to be shoehorned into the behaviours of a for-profit enterprise, but I also would not Rust to categorically exclude those enterprises.

For what it's worth, I do feel for you that the tools you are using are not meeting the needs you have. That's a frustrating place to be, especially when you can, as you clearly can, see a possible future that does meet those needs and are struggling to persuade others of it.

21 Likes
21

To try to get more people doing 3D in Rust, I've just written 3D in Rust, the missing intro. This is a brief note to tell people what to read to get started, and the obstacles ahead.

21 Likes