Please don't put ECS into your game engine

Please forgive me this provocative title and allow me to share some of my frustration with Rust game development.

A while back I saw a talk on how Object-Oriented Programming makes our lives harder by making us thing of everything as objects. As example it was described how OOP made us think of Conway's Game of Life as of a program made out of Cell-objects that had methods and properties. In this talk a point was made that all that is actually required to implement Conway's Game of Life is a hash-set.

The exact same thing is true for game engines, except for some objects are a requirement, rather than an option, and it is horrible.

For instance to implement Conway's Game of Life in Unity you would have to make a grid of phony GameObjects that contain phony MonoBehaviors, all of which are acted upon by another phony GameObject with another phony MonoBehavior, with actual logic divorced completely from the scene and the rest of the framework regardless of whether you want to use hash-set or not.

Why is that?

Game of Life is a brilliant example of how our ways of making games make development hard, complicated and unreliable. The Game of Life actually requires you to make state persistent (sort of). If all cells would be updated in-place then already updated cells would affect update results for the rest of the cells. In other words already updated data conflicts with not yet updated data. And that is true not just for The Game of Life -- just as "updated" cells conflict with "stale" cells, "updated" entities conflict with "stale" entities. As a result, the state is only "correct" before and after entire update, and everything in between is a minefield of bugs, because state is inconsistent, it's a conflict of "past" and "present". And when update order is not guaranteed it gets even worse. And we have to deal with that.

I would chose Löve over Unity any day of the week. Despite Unity's rich feature set, scene editor, asset management, etc etc. Because Löve allows you to just sidestep that problem, it doesn't impose any rules on your state management (and thus game logic) at all. You are free to implement whatever you want in whichever way you want. And engine does for you exactly what you need -- call API when you want sprite drawn or sound played or something. It's "more with less".

And I would love to see more (with less) of that in Rust and in three dimensions as well, but it seems first thing people get in engines is ECS, and that makes me very sad.

TL;DR I just want a 3D game engine but I can't build my own and everyone else who knows how it's done builds them with ECS which makes terrified to use them for reasons stated above.

I am writing it here to see if anyone cat relate to what I'm saying and, hopefully, suggest something.

3 Likes

I know little about game programing, but what would you put in instead?

1 Like

The last time I looked at Löve, it seemed like an extremely thin wrapper around SDL. If that's all you need, you can use Rust's SDL wrappers directly. Fundamentally, game engines are opinionated pieces of software: they have an idea about how some games should be built, and try to make that approach easier to implement.

What it really sounds like is that you want the datastore to include some kind of transactional feature. That's orthogonal to whether it's object-oriented or an ECS: either one can be designed with both a current and pending state and a way to switch between them.

1 Like

That's the thing -- it depends on the state of the game, so, I suppose, I would put in exactly what is required. There are a lot of quite valid game states with which ECS does not help at all. For instance what use is ECS if player is in the main menu or loading screen?

Every games has their unique set of requirements so there's no single engine that fits with all of them. To compare it with web server frameworks, some people only wants basic HTTP protocol parser and writer, some wants routing, some wants async support, some wants file system structure convention, some wants full ORM. You shouldn't yell at django to have ORM just because you don't need it.

ECS pattern is to handle growing complexity of the game logic. For example let's imagin a not-so-creative RPG game. Some entities have rigid body and block others to have colliding position. Some entities have health point and can be damaged by some other entities. Some entities have inventories that can be interacted with some other entities. Some entities have moving rules which are activated if some conditions are met. Historically game developers solve this issue by make an entity class inherits from RigidBody and Damagable and InventoryHolder and ConditionalMovable etc but it wasn't scale well. So they invented the ECS.

The game of life, while being very inspirational game, doesn't have really complex rule - and that's the reason why it's so inspirational. Each cells are too simple and many to be a proper entity in the ECS pattern. Yes, it's one of the game that you should not use ECS for it.

If you like the Löve framework, there's a ::ggez crate which calls itself a game framework inspired by Löve. It gives you a game loop, simple graphics tool, audio management and input event, that's all.

14 Likes

A transactional feature would be nice, I even wanted to say a couple of words about STM, but, decided not to, since my main point is that it's better not to try to guess what would fit developer's needs.

The one time Unity Technologies tried to make a video game they got all the usual unity-game's ad-hocs:

  • custom update-loop
  • that is executed from singleton MonoBehavior
  • that marks itself as not destructible
  • that is the only actual class using engine-provided Update hook

And all of that because they thought they know what kind of architecture is required. Good thing they now know they were mistaken.

Yes, there is no single engine that would fit all the requirements of every game. And when engine enforces restrictions on your code for it's systems to work -- that's reasonable. But when those restrictions exist just because of engine's design decisions about your code -- to me that is purely contrived. Why just not provide programmers with tools and let them decide for them self whether they want an ECS or not. It's not like an engine needs you to use ECS to draw a mesh or to play a sound.

I know basically nothing about game development, but from my memory I think the piston family of crates are designed to follow the UNIX philosophy. Maybe they would suit your needs.

I think game engines are opinionated because game design is so complex that making an AAA game from scratch requires too much mental bandwidth. That's why lots of people use unreal engine - not only does it take care of lots of the hard bits, it also makes it easier to work in a team of designers, programmers etc, where each person has tools that they can use to be productive.

2 Likes

Unity's main architecture (MonoBehaviors, etc.) is not ECS at all. What do you think is wrong with ECS specifically?

I feel like grid-based games are inconvenient to implement in the majority of game frameworks. That's because their implementation could be much simpler, and the general complexity of the framework gets in the way.

2 Likes

GoL makes a good case study of nice ways to update things in Rust. Notably, if you have &muts to everything, it's hard to look at other things, and even harder to parallelize that.

But one thing that's common: the size of the updates are smaller than the size of the state. So I pattern I first saw in mir-opt is to do a pass over the read-only state and write out a separate bag of updates to make. Conveniently, that's trivial to parallelize while still being able to look at the whole world. Then you can do a separate pass to actually apply those updates, which should be pretty simple and thus fine do do single-threaded. (Or one thread per component type, or whatever.)

8 Likes

I do agree that ECS is hard to get used to and adds a lot of complexity. I recently decided to recreate snake in two different frameworks: coffee and bevy. It only took me a night to implement it in coffee because it's so simple and it took me a week to do it in bevy. Bevy was hard to work with, even though it clams to be a simple ECS. The things that would have helped me would have been if the book went over some of the built in components. Also they need more complex examples. Ones that demonstrate how load different levels and transition between selection screens and game play. I also think that it's hard to figure out how each component is used. Rust docs don't help with this at all, because there is no way to search systems that use a component.


That being said I do think that part of the problem is that I am trying to implement a simple game using a complex engine. The main benefits of ECS come from the modular design. This doesn't really show itself in such simple examples.

I made the same discovery when I used Elm GUI programming for the first time. It does not require you to update any widget text or conditionally add listeners to a button. Instead, it uses an immutable architecture, the Model-View-Update pattern. I assume this would not yield acceptable performance for games, but showed me that there might be more useful approaches than the existing ones.

I think a problem with ECS is that it does too many things at run time, delaying errors, and harder to debug than compile errors

Yes, that's one of the cool benefits of persistent state. In Clojure (with persistent data structures in use) that's whole 10 fps out of a single keypress

Whether it's true or not, my main point all along is that making ECS the only option to manage state to make it more fit for large-scale projects comes at cost of making it a lot less versatile. And yet it still would make sense for people to chose Unity or Unreal over it.

Because it's not architectural decisions that make engines attractive. Source Engine was used to make large-scale projects not because of its architectural decisions, but, I speculate, despite them.

Rust is already hard to program with, because of all of its compile-time magic. And when ECS with all its restrictions is a necessary part of the equation, it would make perfect sense to chose some other engine, which architecturally would be exactly the same (an thus come with same problems), except it would "benefit" from using some other language. And that engine could be rich with tools and docs as well.

And all along ECS was and still is completely optional, possibly even completely independent, with a lot of implementations to chose from. Yet people make their own, put it in their engine, build everything around it. All for what? For a large-scale project that is not going to come? Because ECS is not enough. Because large-scale projects at the very least require some engine-specific game design tools (and there are none).

Rust has one of the best infrastructure I've ever seen (rustup, cargo, build scripts, macros) as well as yielding terrific performance and being compile-time-proven-safe, even if safety comes at some price.

Löve has basically nothing. Even luarocks doesn't count because it's a mess and Löve uses LuaJIT anyway (it's only compatible with Lua 5.1). It even seems reasonable to write native libs for it and then use those e.g. to parse xml.

Unity Technologies do their own thing (they always do) with custom compilers and package management systems (so good luck I guess).

Unreal Engine uses C++, which, I suppose, was not bad enough so they decided to, sometimes, generate it, resulting in it being such a mess that there are paid tutorials on how to delete a class (I am dead serious).

Among all other options, somewhat reasonable Rust game engine with simple API would be a bliss. Because of language design and infrastructure, because of performance.

Sacrificing versatility and usability in exchange for scalability via ECS is just not reasonable -- there is nothing to scale yet. And even if there were -- ECS could be optional and completely independent. Because when developers manage state on their own -- they can implement whatever they want however they want, and that includes ECS as well.

1 Like

You’ve spent a lot of words describing things that a game engine shouldn’t do, but haven’t said anything about what it should do. In your mind, what is the role of a game engine? Is it just a display library?

1 Like

Technically you could call it that (but that's not to say that feature set should be modest).

The whole "game engine" notion is arbitrary.

Löve is a good example of a game engine that everyone calls a game engine, that is, in essence, "just a display library". By the same token a "game engine" could be a framework. My whole argument, in general, is just that a Rust game engines should (at the very least for now) avoid being frameworks.

For the time being, AAA-developers, it seems, are not likely to show up, and, even if they do, they, I suppose, would start from scratch.

The best we could do now is to provide Indie-developers with some features and good API to use them. And they even would not mind limited customization (many of them are perfectly happy with built-in Unity shaders for instance).

So yeah, technically speaking, just a library, or, maybe, a set of libraries.

I haven't written any games, but I do think about software design a lot.

I think this is an example of the general problem that in Rust architecture is hard because the compiler is strict.

ECS enthousiasm came along because they found OOP was hard with Rust ownership and single point of mutability. I do think that it is overrated. It doesn't really solve any problem, it just turns it sideways. Instead of having access to a number of different properties that go logically together, it allows access to one property for a number of logic entities. It doesn't really add any flexibility per se, just structures things at a 90° angle.

In the stuff I do, I use the actor model. It solves all ownership and mutability problems easily, but for games the issue here might be performance. And actors are particularly overkill for things like getters and only really useful when you do async io rather than calculations. Not a great fit for games.

All in all, games are relatively simple:

  1. collect input
  2. re-calculate game state
  3. re-render game state

Thus it becomes quite reasonable to see game state as a global singular object and let logic always have access to all information. I think that is a more important part of the design. Where OOP groups some state with some logic, this separates state from logic. It completely does away with information encapsulation.

Note that encapsulation was mainly intended for libraries. Have a clean API and hide internals so people don't start depending on them. For an application, you can afford having to update several parts if the data format changes. Tight coupling of components still isn't a good thing, but the simplicity of separating data from logic might well be worth it.

Once you have that global state, it might as well be a HashMap tbh. It really doesn't matter how you lay it out.

You can now implement transactions like scottmcm described above or you can clone the whole thing and read from one and write to the other (if everything is layed out sequentially in memory, that's probably faster and simpler anyway). I've yet to look into it again, but I recall that the approach of druid with lenses sounded interesting as well.

As for game engines I would think the best is that "state handling" libs and rendering libs be orthogonal and usable independently, or in other words the good old libraries vs frameworks. Or at least build your frameworks from libraries that don't depend on the framework.

3 Likes

Doing "architecture" is equally as hard in C++. The only difference is that when you make a mess in C++ your program randomly crashes and burns or produces the wrong results at run time. Where as the Rust compiler will not let one make such a mess. I know which I prefer.

As far as I can tell this is not true. ECS has been in use in C++ for ages. For example in Minecraft: GitHub - skypjack/entt: Gaming meets modern C++ - a fast and reliable entity component system (ECS) and much more

The motivation for ECS is performance, plain an simple, to quote the entt author:

The proposed entity-component system is incredibly fast to iterate entities and components, this is a fact. Some compilers make a lot of optimizations because of how EnTT works

Apparently it solves a performance problem in appropriate situations.

The question then is, does one actually have a performance problem and can it be solved with ECS?

4 Likes

I'll note that nothing I said required "persistent state". One of the great things about rust's ownership model is that you can get the best of both worlds: reliable immutability when it's helpful, but also a guarantee that you can just edit something without affecting others when you own it.

So this doesn't need you to use a persistent data structure for the board, nor the double-buffering approach. You only have one board, and you update it every step. That means no memory allocation most updates (only when the updates-per-frame count outgrows its existing buffer), unlike what usually happens with persistent data structures.

1 Like

I'm glad the conversation finally came to this point. Personally I don't see anything wrong with seeking alternative ways to manage state in a game engine. That seems like a perfectly reasonable request! Another way to look at it is that requesting everyone stop putting ECS into their game engine "because reasons" is counterproductive.

As stated by the two comments you quoted, ECS does start to come in handy at scale. To take your main menu example from earlier, if all buttons were entities and you wanted to add an easter egg that turned the main menu into a classic game of pong (or pick your favorite; Pac-Man, Tetris, it doesn't matter) then it's a "simple" task of adding collision and some behavioral components to these entities and you suddenly have a compelling easter egg.

This is a contrived example of course, but it illustrates the problem that ECS tries to solve. You cannot start a new game project understanding every minor details of how every little thing will interact with every other minor detail. If that was the case, every game would take 2 months to build by a single developer and it would be bug-free.

Most of game development is experimentation. And being able to dynamically add behavior and state to "entities" is where ECS shines. It can be a useful tool for rapid prototyping. Traditional approaches like object-oriented designs tend to lead to leaking state and behavior (or worse, the diamond problem). Especially as you attempt to share and reuse state/behavior among entities that independently need to pick and choose smaller parts of a larger class.

So that is really the case for ECS. It doesn't mean you should be forced to use it if you don't want it. But most game developers will benefit somewhere in the process of finding the right combination of things that makes their game fun to play. And that also makes the case for including it by default in a game engine; provide tools that most developers will use at some point.

I guess the only place to really go from here is defining a reasonable alternative and pitching a proposal to game engine authors?

7 Likes