Help with an ECS based project

help
1

I'm looking for some high-level guidance or a long term mentorship (1 month+) for a project I'm currently working on.

Inspired by Arcs and GatsbyJS, I'm attempting to build a static site generator.

The way I've visualized this is:

  • each "Entity" would correspond to a page
  • "Components" would be plugins/pieces of shared data
  • Providing data to a render template would be created by composing "Components" on top of the "Entity". (As opposed to in Gatsby, where JavaScript's dynamic ness allows any new data to just be added to the Node object.)
  • The "System" logic would be for each plugin or necessary combinations of plugins. (This logic would be implemented by the plugin)
  • A "plugin" would only have access to the other components it needs.
  • the rendering to be up to the user. Use ructe, Teras, liquid, etc. as long as it can produce a string (or write to a Writer), the generator will accept it.

There are a couple parts I'm having trouble working out:

  1. Is ECS even the right way to go about this? I've played around with Shipyard, specs, and legion but I'm not attached to any one in particular
  2. How would I restrict what a plugin can do in terms of messing with the "World"/"System?" The ECS crates I've looked at assume "Systems" or mutations on Entities/Components won't be in a hostile environment (which makes sense for a library) but won't be true with a "plugin."
  3. It would be ideal for the ECS library this generator uses to be an implementation detail. The user would only have to implement the rendering logic, but the type-heavy way that components are passed around and queried in the crates I've found have made it very difficult for me to reason about how to even go about creating this abstraction.
    - Especially for plugins, providing a trait that would allow the plugin to specify any dependent "Components", which may or may not be different for all plugins

Any snippets of wisdom, code, or references would be greatly appreciated!

2

I think you're asking a good question there. Why do you think you need an ECS?

As far as I know, a main use of ECS is in game development, and one of the benefits is that grouping data in a different way (structure of arrays instead of array of structures) can have benefits in terms of raw performance, cache-friendlyness, etc.

All this seems quite far from your use case for a static site generator. I would recommend starting by designing your data structures (Rust structs) in the way that models directly your domain in the most straightforward way and evolve it as your start to understand the problem better. It's possible an ECS might be useful too, but as you seem to have many questions about what to use, do consider the simpler approach as well.

1 Like
3

This isn't really my interpretation of how an ECS would be used... If you are familiar with OO languages (e.g. C#), think of each Entity as the equivalent of an object and a Component being a property attached to an object. Similarily, a System can be thought of as a method which operations on object fields in bulk.

Keeping that in mind, you'd have entities representing things like pages, templates, etc. and if a page has some associated data (e.g. author, content, edit history) then they'd be attached as Components.

From there, you can think of the rendering process as a series of Systems which progressively manipulate the world (load files from disk, run early plugins to manipulate which files are used, apply templates, run post-processing plugins, write the files to disk etc.) until a bunch of HTML pages pop out the end.

Something to note is that a Plugin contains behaviour, so it doesn't actually belong as a Component. Instead they would be part of a System... or more likely your own custom type implementing System and wrapping the plugin as a trait object.

This sort of information hiding isn't really a thing in an ECS. Unless you make it so plugins can't refer to the Component type (e.g. it isn't exported by the crate), there's no real way to prevent access. Keep in mind this architecture is intended for applications where you prefer flexibility and rapid iteration over security or

Instead, I'd take a convention-based approach. For example, if you provide plugins with a set of utility functions and helpers then they'll use that 95% of the time but still have access to the internals if they need an escape hatch. Kinda like Python's naming convention that items with a leading underscore are private.

This could be implemented by making sure plugins can only interact with the world through a trait which operates on the components and never actually getting access to the World or entities and their storages. It might look something like this if you were using specs:

/// The interface to be implemented by our plugin.
trait Preprocessor {
    /// Update the content for a page before it gets rendered to HTML.
    fn update_content(&mut self, name: &Name, content: &mut Content);
}

struct PreprocessorSystem<P> {
    plugin: P,
}

impl<'world, P: Preprocessor> System<'world> for PreprocessorSystem<P> {
    type SystemData = (ReadStorage<'world, Name>, WriteStorage<'world, Content>);

    fn run(&mut self, data: Self::SystemData) {
        let (names, contents) = data;

        // imagine we created a bunch of pages which have components like this:
        // let page_entity = world.create_entity()
        // .with(Name::new("Overview"))
        // .with(Content::from("Lorem Ipsum ..."))
        // .with(Author::new("Michael Bryan"))    
        // .build();

        for (name, content) in (&names, &contents).join() {
            self.plugin.run(&name, &content);
        }
    }
}

You can see that the Preprocessors have no idea we're actually using specs under the hood, and have no way of accessing the world or breaking things. However, for more sophisticated plugins you'll probably leak some implementation details, because the world has a way of adding random edge cases which force you to compromise on the ideal design.

1 Like
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.