Programming simulations usually is no simple task. Especially when it's about creating a computer game, which has a lot of data to crunch and a lot of different logics to run. Organization is the most crucial part, and traditionally, every game studio has their own secret sauce. However, times are changing, we see computers with more and more cores, a flood of Indy Games, and a blossoming opensource world. All of that led to more uniform approaches, and also very elaborate systems to handle the problem. Lately, for example, one rather old idea is making the round and seeing implementations in different game engines, like Unity (DOTS), GameplayKit, Amethyst, and many more. I am talking about Entity Component Systems. I created my own library, and here is why and how.
Content
- tl;dr
- What is an ECS?
- Why create this project?
- From inspiration to implementation
- About challenges and hurdles
- What's next?
TL;DR
Don't want to read all the details? Read this section for the minimal gist. If you are interested in the tech and want to learn something new, skip this chapter and read on about "What is an ECS?!"
I created a ECS in TypeScript, mainly because the existing projects are dead, missing features, PoCs, or I don't like how they solve certain problems. You can find my project here: sim-ecs. It's an ECS focused on fast iteration speed and ease-of-use. I was mainly inspired by SPECS and Legion, two modern Rust ECS libraries, but handled some things differently and tailored for JS. Having a great ECS library for JS allows me to quickly build systems and PoC features I want to put into my game without spending a lot of time writing compile-able Rust code or introducing technical debt to my main code base.
If you are interested in SPECS and Legion, how they work, compare to each other and what kind of differences they have, I recommend this great article by Cora Sherratt, which compares them in detail.
What is an ECS?
First of all it is important to understand, what an Entity Component System actually is and what it is used for. In an ECS, we have three different parts. Firstly, we have systems, which are blocks of logic, which operate over data sets. Secondly, there are components, which are blocks of data. And thirdly, there are entities, which act like glue for components, bundling them into packages of information. The clever thing about an ECS, however, is the way how these three parts work as a whole. An ECS can be thought of as a big database table. The columns of the table represent possible components, and the rows are component combinations, indexed by an entity.
There are several advantages to having an ECS as the backbone of your simulation. The one, why the Rust simulation community is so into it is because it works very well with the ownership model of Rust, which, while eliminating a lot of runtime problems, is also difficult to work with (because the solved problems are difficult in the fist place...). We don't use Rust here, however having a good separation of data and logic, and doing composition of objects as opposed to inheriting classes, resulting in strange workarounds, is a boon no matter how you look at it. Additionally, when using an ECS, logic is automatically split up into easily parallelize-able chunks, so we can make use of a lot of CPU cores for free! Note: For JS, this means leveraging WebWorkers in the browser, or the Cluster module in NodeJS.
If you want to know more about why the ECS architecture is awesome, I recommend reading the RustConf 2018 Closing Keynote by Kyren from Chucklefish. It is a lengthy read, but it's a good read!
Why create this project?
One could argue that creating an ECS for JS is stupid. It would be a lot better to write a simulation with WASM as a target. Plus it is difficult to harness the power of several CPU cores from JS. Also, let's not forget that there probably already are a good number of libraries available solving the same problem. Why not just slap one of those into my applications?
While it is true that I used this project to learn more about ECSs, it certainly also is no excuse to that end when I say that the available ECS systems are not TS (I want type-support!), dead projects, PoCs themselves, or solve the problem in a way which I don't like and might require a lot of overhauling to implement features I want. For example, if I want to PoC a feature for my game, written in Rust with modern, fully featured ECS systems, I need ways to juggle worlds, components and entities, while still having solid performance (as much as you could expect from pure JS anyway). Ideally with a similar API and feature set, and the level of ease-of-use and DX I am used to from the Rust libraries. By the way, that's also the reason why I want an implementation for TS, instead of slapping something into the browser with WASM. WASM cannot handle many things about JS, like usage of Prototypes - which would result in a lot of number-workarounds or string shuffling, which also is not very performant to do between JS and WASM.
Since writing this ECS took a long time and lots of thinking out of the box, I also hope that I can help other game devs, who just want to create a small HTML5 game, but also want to have the full potential of big systems directly in their hands. Most folk who want to write a game do not want to learn a difficult language, but might already be familiar with webdev.
From inspiration to implementation
Speaking of the big systems, I did not pull most of how sim-ecs looks like and works out of thin air. My two major inspirations are the modern Rust ECS libraries, SPECS and Legion. I especially love the classic ECS interface provided by SPECS, however I also like how Legion makes an ECS look more like a DB, and adds things like cached queries and archetypes, which help a lot with performance. So, I decided to mimic the way of SPECS' API. For example, SPECS can be used like this:
use specs::prelude::*;
#[derive(Debug)]
struct Velocity(f32);
impl Component for Velocity {
type Storage = VecStorage<Self>;
}
#[derive(Debug)]
struct Position(f32);
impl Component for Position {
type Storage = VecStorage<Self>;
}
struct SysA;
impl<'a> System<'a> for SysA {
// These are the resources required for execution.
type SystemData = (WriteStorage<'a, Position>, ReadStorage<'a, Velocity>);
fn run(&mut self, (mut pos, vel): Self::SystemData) {
// The `.join()` combines multiple component storages,
// so we get access to all entities which have
// both a position and a velocity.
for (pos, vel) in (&mut pos, &vel).join() {
pos.0 += vel.0;
}
}
}
fn main() {
// The `World` is our container for components and other resources.
let mut world = World::new();
world.register::<Position>();
world.register::<Velocity>();
world.create_entity().with(Velocity(2.0)).with(Position(0.0)).build();
world.create_entity().with(Velocity(4.0)).with(Position(1.6)).build();
world.create_entity().with(Velocity(1.5)).with(Position(5.4)).build();
world.create_entity().with(Position(2.0)).build();
// This builds a dispatcher.
let mut dispatcher = DispatcherBuilder::new().with(SysA, "sys_a", &[]).build();
dispatcher.setup(&mut world);
// This dispatches all the systems in parallel (but blocking) once.
dispatcher.dispatch(&mut world);
}
Quiet a bit going on there, however it is a simple piece of code which defines two components, Position and Velocity and a system, SysA, which iterates over the entities that have to be created at runtime in main(). In the SPECS API, there is some baggage, which is unnecessary, because it could be done in constructors (see the call to setup()), or automatically recognized by the system (component registration, for one). Also, one major flaw of SPECS is that it has to do the join of components, which creates a set with the needed components, each time a system is executed (each time SysA::run() is executed). That's a terrible overhead, and it shows in benchmarks. Which is why I decided to take some under-the-hood-inspiration from Legion, which will replace SPECS in Amethyst for performance reasons, among others. The result is a cleaner API, which only exposes the least necessary actions to a library user. Writing the same example as SPECS uses, just for sim-ecs in TypeScript, yields:
import {ECS, ISystemActions, Read, System, SystemData, Write} from "sim-ecs";
class Velocity {
speed: number = 0;
constructor(speed: number) { this.speed = speed; }
}
class Position {
x: number = 0;
constructor(x: number) { this.x = x; }
}
class Data extends SystemData {
pos = Write(Position);
vel = Read(Velocity);
}
class SysA extends System<Data> {
readonly SystemDataType = Data;
async run(actions: ISystemActions, dataSet: Set<Data>): Promise<void> {
let pos, vel;
for ({pos, vel} of dataSet) {
pos.x += vel.speed;
}
}
}
main: {
const ecs = new ECS();
const world = ecs.createWorld();
world.buildEntity().with(new Velocity(2)).with(new Position(0)).build();
world.buildEntity().with(new Velocity(4)).with(new Position(1.6)).build();
world.buildEntity().with(new Velocity(1.5)).with(new Position(5.4)).build();
world.buildEntity().with(new Position(2)).build();
world.addSystem(new SysA());
world.dispatch();
}
Alternatively, you can check out the counter example, which includes a lot of descriptions and additionally features a continuously running world and resources, which are pretty useful.
Anyway, as you can see above, I had to add a constructor to the Components, because Rust works a bit different, not having constructors, but requiring fields to be filled by name or position on initialization. That's not a big issue, though, and as a bonus, I got rid of a few annoyances while maintaining the important features, namely leveraging TypeScipt's type system (you get auto-complete for the types in a system's run() method) and JavaScript's Prototypes, which foregoes all the string-matching, which are a possible error-source. For example this means that you get additional build-time checks on component names, because you have to use the actual classes instead of class names (which was one of the downsides of existing libraries). All in all, I am quiet happy with how I can use the library.
By the way, you can test sim-ecs yourself. Either get the package from NPM
$ npm install sim-ecs
or pull it directly from GitHub (in your package.json):
{
// ...
"dependencies": {
// ...
"sim-ecs": "github:NSSTC/sim-ecs#0.1.0"
}
}
If you use the github version, make sure that you use the latest tag or master branch, else you will get stale, outdated code ;)
About challenges and hurdles
Implementing the above, however, was not easy. It meant exploring the limits of JS and TS. For a long time, I wasn't even sure if I could copy the API the way it was, because Rust has a ton of cool and novel features. Creating sim-ecs was no walk in the park, and here are two of the things which took me over the limits of what I knew was possible.
Manual run-time object-to-type assignment
This was the thing which had me floating the most. From the beginning I spent wondering, if it is possible to declare a datatype and have it filled with the correct components, and on top of that get auto-completion working for that. Auto-completion was a big issue, because it is a valuable tool when developing an application, and I did not want to miss it. So, I didn't implement it for a long time, and instead just passed an array of entities to the run() method, hoping some day I would have a Heureka moment. Spoiler: It did come. However, it only came after I created the rest of the ECS using lots of Prototypes everywhere, so much that I became very comfortable working with them. All I had to do was declare the fields of a class the type of components I wanted to use, and then somehow also pass that information in a JS-useable way without having to initialize them - which happens to be the prototypes of said components. Under the hood, the ECS would match the prototype names of all components against the prototype information of each field of the Data object in order to identify the entities which may go into the data set of a system - and then create a new instance of Data, which however is filled with the entity's component refs. Does this sound like an awful lot of loops to you?
CRUD operations on entities and components are slow at run()-time
The other big issue I had was that while I stored all entities (and later information chunks) in a field on each system, which meant simple iteration over continuous data, actually changing anything was - and still is - an expensive task. That's why, in the beginning, I introduced -Quick-() methods, which would change arrays in the world object, but not start re-sorting the systems. I also only had the maintain() method, which would do all the sorting and updating, which means that all the things were sorted, even if they did not need to be updated. At some point, I came up with the idea of restricting the passed world object and scope it to what a developer should do at a certain time, leaving out dangerous operations. For example, when inside a system's run() method, a developer should not need to add resources, and definitely should not change components or entities, because that would mean resorting, and that would change the data of other systems, which may or may not have run, yet, which would be pretty much the same as undefined behavior. Danger zone! However, implementing these scopes also let me proxy certain calls, like adding entities, so that I could actually control precisely what is updated, in what way and when - which means only doing what is needed on the live-system. Instant performance-boost, and a safer API!
What's next?
For now, I have a few things left for the 1.0 milestone. However, they mostly come down to under-the-hood improvements and paper work. I need better docs, benchmarks, a CI pipeline. In addition, I want to implement change detection on components and improved archetype handling, which might change the API slightly (see that dispatcher builder in the SPECS example? It makes a lot of sense for a lot of reasons). Also, I need feedback from other people, since I want this to be an active project, which can be used by other people. Is the API intuitive? Are there any big errors on my side? I am mostly concerned with the API, since once I release 1.0, I can easily fix things under the hood, but I cannot easily touch the interface anymore.
Once I have confidently published 1.0, I want to start working on optimizations, which push sim-ecs further, making it really fast and leveraging everything JS VMs are capable of. This mostly comes down to multi-threading and better algorithms for scheduling and execution planning, which are the most powerful in a multi-threaded environment.
If you like what you see, I want to invite you to test sim-ecs out, or contribute. There are a lot of tasks for beginners and also harder tasks in the issue tracker. Even if you just want to write a bug or finding, I would be very happy about you speaking up.
Thank you for reading my article, and see you around!
Unbezahlte Werbung durch Nennung und Verlinkung von Produkten, Personen, Organisationen oder Unternehmen.