Please don't put ECS into your game engine

41

Well, as I said

Although it might not seem like it, I actually like ECS. I am arguing specifically against ECS being at the bottom of everything.

Yes, but the engine has to allow some freedom to do that. Amethyst's state transitions (the State trait in general) is the best option that I've seen, though it is a little restrictive, given that only one type of data could be shared between states, and, I expect, could lead to the acorn-behind-the-tree ad-hocs.

I am currently looking deeper into what Amethyst has on offer, it seems to reflect exactly what I'm talking about.

42

Coroutines are a completely separate topic, which involves concurrency. State transitions are done in Unity (or UE, or Godot, or pick your favorite engine) with scenes. You can think of a scene as a container for a scene hierarchy.

It's a hierarchy because that's how humans typically think of the structure that makes up the game world. And that makes editing quite intuitive. That hierarchy just gets flattened when entities are added to ECS at runtime.

A crude example of such a transition is just clearing the ECS world and populating it with all entities from the next scene. What's interesting about this approach is that it is purely data-oriented. There is no explicit "state machine" beyond the hypothetical state machine that is created by various entities and systems loading each new scene. In other words, you don't write nested loops like you've shown. You just have a main loop that drives the state machine through systems.

Sharing state between scene transitions, I don't see why that can't just be another component perhaps using continuation-passing style. Something like the load_scene function accepts an argument for state to carry over.

43

I thought the whole motivation behind ECS is that there are no nested game loops. There is only one. It is unavoidable. It is dictated by the physical reality of the frame refresh rate. At 60fps you have 16.6ms to figure out what it is you want to display and get it rendered.

Given that hard constraint and the desire to have hundreds/thousands of objects in a scene it turns out representing those objects in typical OOP style does not meet the requirement. Typical OOP style splatters your data all over memory in a cache unfriendly and slow way.

Enter ECS. It organizes the data of your "objects" into nice neat arrays in memory that your processor can rip through applying updates and transforms very quickly. Avoiding the huge penalty of cache misses.

What am I missing here?

I am no game writer but this is all reminiscent of real-time control systems. Think aircraft control systems. Where one typically has a hard deadline on performing the critical work. 10ms or whatever. If there is other less time sensitive work to it gets done a bit at a time over many iterations of that time tick. Those jobs may well have state machines of their own.

44

Although I've called them coroutines they are actually a generators (they're called coroutines in some places for some reason), that are invoked every frame. They are going to be added to Rust at some point it seems.

That feels like constructing a Rube Goldberg machine.

45

Do you mean Heath Robinson? W. Heath Robinson - Wikipedia :slight_smile:

Perhaps so, but the solution is driven by the problem. If your game does not demand a 60fps update and rendering of thousands of objects perhaps you don't need ECS.

46

I don't think that there should necessary be a trade-off. What could actually really help is just a place for logic overarching the ECS. Amethyst, as it seems, provides exactly that.

47

Call it what you will, it's a very common way to manage scene transitions in modern game engines.

UE4's level streaming is overkill, but it's the same core concept at an "open world" scale.

1 Like
48

I can feel the same frustration. I came from Unity at that time when I still considered me a indie game developer. I just loved the ECS system that Unity had, and so, I tried to re-implement it in MonoGame/.NET, and SDL/C++.

But once I got my first contract as a game developer, I ran into optimization issues. I was developing a little idle, clicker RPG beautifully (I thought) based in ECS. The game ran as hell.

After doing my research, I discovered the flaws such update-per-entity-system Unity had. And it was with almost every ECS implementation across every engine/framework.

I had to re-write entirely all the game mentioned early, costing me an expensive amount of time. I made use of Unity co-routines and .NET delegates (callbacks) to avoid Update-callbacks. Where it would only be changed a group of specific game properties when a event happened. I could improve by near 90% of the game's optimization, showing how a mess my code was prior.

After that experience, it was broken for me all the purpose the ECS had: reliability.

Afterward, if I used more than one GameObject, it was because Unity needs them for render graphics, play sounds, do animations, and such stuff commonly not related to the game logic. And so, I just used ScriptableObjects to make data persistent across a game session, not as another way to abstract my game components.

I believe that Entity-Component representations should be used as an abstract reference, not the implementation. Somewhat you would look at the game specification document, or level designs.

The problem with ECS and OOP paradigms directly implemented into software is that they tell us we have the necessity and mighty to re-use code everywhere, and every time. We even believe we can do it with several kind of software. But it ends costing much more than re-implementing solutions.

Now I know that a software's code only belongs to it, and it shouldn't be intended to be reused later. The only software that should have such purpose are APIs. And their interface shouldn't be as specific as an application would need. Video games are not APIs.

Instead I prefer to make use of the past knowledge and improve implementations the next time, or even better, find new ways to do it. Focus on the software functionality, effectiveness, and maintainability will be a better use of my time, rather than designing a perfect, and totally unnecessary re-writing of my perspective of life into code.

49

Could you talk about what exactly this flaw is?

1 Like
50

There's nothing forcing you to use ECS for everything in ECS-enabed engines.

I worked on a grid-based game utilizing Amethyst , similar in mechanics to game of life, and grid tiles were in a Vec<TileTile> etc. I only used ECS for stuff that made sense.

51

I wrote this to help explore what I feel are some under-examined assumptions leading people to talk past one another in this thread. It ended up way longer than I expected, sorry!

First, it’s probably a good idea for general purpose game making tools to separate rendering and state maintenance/update. There’s no avoiding the tension between “every game is unique” and “lots of games do similar things”, so we ultimately want to structure our code so that developers can opt in/out and ideally control the game loop themselves either by writing it or by configuring the engine. Even if we are forced into “everything is via ECS”, there’s nothing stopping you from having one entity and one component and doing what you want in there, just like we often do in Unity games because it so tightly couples the rendering and simulation.

That said, It’s also worth noting that there are two things called entity component systems:

  1. Unity-style (really, Common Lisp style or Smalltalk Morphic or Self-style) entities each of which has a Vec<Box<dyn Component>>; we update each entity which updates its components, which can freely query state from other components within the entity or on other entities. Expressive, convenient, super flexible at runtime, utterly un-optimizable (every dispatch is dynamic and through a pointer lookup, every function call blows out the instruction cache, every object is big), hard to integrate with other schemes unless you’re ok with giant über-entities.
  2. Specs-style (for example) “data oriented” ECS systems, where we group together the data belonging to components of each type, and often require that accesses to other components are committed to in advance (similar to indexes used for joins in databases). Entities don’t exist per se (usually they’re just an ID tag). Can admit some adding and removing of components to entities at runtime but often at an efficiency cost, requires more up-front commitment, but when done carefully can eliminate needless cache misses and even branches from hot code.
Life example

For the game of life example, I’d say the data oriented way is a pair of Vec<u8>, old and new, and we swap them at the end of the frame; indeed no ECS required. You could have a LifeComponent with a pair of Vec<u8> like that rather than have each cell be an entity (so the whole grid would be one entity, maybe useful if this is inside a larger videogame or something), or you could give up some performance and allow for each cell to be an entity (e.g. if transition rules can change at runtime per cell) and still come out ahead of the Unity style ECS, since you’ll process all the cells using rule A then all the cells using rule B and so on.

It would make sense to put the global old/new state vectors into a resource or something rather than spread it through the components, but you could still do it with components if you allow more than one system per component; then the ProcessRuleNSystem updates its own state based on the OldComponent and the CopyRuleNSystem copies its calculated state into the OldComponent. Then just ensure the Copy systems all run after the Process systems. Interestingly this may turn out faster than the original loop over every cell if we don’t need to branch inside of the loop to pick which rule to apply. Of course we could replace the original loop with a loop over the cells involved with each type of rule and win back the difference and avoid needless copying.

Nested loops example

To get through the nested game loops situation, it’s definitely a useful game design observation to say it’s nested loops; but a key insight of data oriented programming is that your problem domain is not necessarily the way you should structure your program, because your program is running on a specific type of computer. Also, your problem domain may change over time and that can cause a lot of code churn. There are lots of good solutions ranging from different isolated ECS worlds (updated based on a state machine) to activating and deactivating different sets of systems (based on a state machine) to blowing up most or all of the entities and components and making new ones (when a state machine transitions). Sure, if you’re making a board game you’ll want to be careful about how you express turns and phases and that generally wants to be global somehow, but this may mean that you have ActivePiece and InactivePiece components you add and remove appropriately, or you give up and do some branching inside your Piece component based on global data about whose turn it is.

I think the main insight of data-oriented ECS is that we want straight-line code, with for loops over dense arrays in a pinch, and so we want to structure our whole world to avoid recursively defined data or data with discriminants we need to branch on. The performance benefits are one thing, but it turns out flat data of known size are easier to serialize and deserialize, to perform all kinds of automated translations to, and so on. So there are maintainability benefits too.

5 Likes
52

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.