All about Entity Component System


All about ECS!


Foreword

Disclaimer: Interested in a ECS library that I recently made? Check out Jecs, a stupidly fast ECS with entity relationships as first class citizens for powerful query capabilities. But keep in mind that this post will not go over the usage of it!

I am basically going to copy the exact format of magnalite’s post All about Object Oriented Programming because it is the most known post of a similar topic. At times, I will even be using the same exact wording to make a reference, so I expect you to have read the aforementioned post.
For the tutorial, I will be using Matter because I believe it is the best available ECS, you can follow this page to get instructions on how to install it. I want to give acknowledgement to other great alternatives:

Table of Contents

What is ECS?
How does it help me?
How do I make this work in Roblox?
Sources

What is ECS?

The Entity Component System (also known as ECS) provides infrastructre for representing distinct objects with loosely coupled state and behaviour. Many modern games developed outside of Roblox in the outer game industry uses ECS in some shape or form, namely Overwatch, Minecraft, Hearthstone, Rust and many more! An ECS world consists of any number of entities (unique ids) associated with components which are pure data that contain no behaviour. The world is then manipulated by systems that implement self-contained behaviour, each of which access a set of a specific component type.

How does it help me?

Most who are new to programming find it difficult to express what they want to achieve with their code because of their lack of proper and fundamental approach to logic. Most believe that it is hard to break things down to step-by-step.
OOP tells to think in terms of objects with layers of abstractions that handles underlying processes. For example, let’s say you’re making a racing game. It would be useful to be able to do car = Instance.new(“Car”, Workspace) and then be able to find useful information such as car.Position or use methods such as car:Respawn(). This is a problem because you are tying behaviour to a specific construct, which means you cannot use that behaviour without using that construct.
In simpler terms, you cannot use ::Respawn() without car unless you inherit those traits and behaviours. So, to make objects, you do inheritance. The following snippet shows what has been described.

Car = require(game.ReplicatedStorage.Car)
Truck = {}
Truck.__index = Truck
function Truck.new(position, driver, model, powerup)
    local newtruck = Car.new(position, driver, model)
    setmetatable(newtruck, Truck)
    newtruck.Powerup = powerup
    return newtruck
end
local truck = Truck.new()
truck:Respawn()

To demonstrate the issue with this pattern, look at this example where both electric and gasoline cars inherits properties of Car.


Once you add a new type of class Hybrid that inherits from both the Electric and Gasoline classes. Notice how in the diagram, this forms this kind of diamond shape

The hybrid car is a subset of both the electric and gasoline car, which results in a duplication of paths which is impossible to express with a prototypal-like language, i.e Luau.
OOP claims that inheritance can be seen everywhere in the world, e.g. a child inherits traits such as hair-, eye- colour, skin tone, etc from their parents. But after acquiring traits from their parents, they also develop their own personalities on top of those traits. Think of it as a top-to-bottom hierarchy.
ECS does this the opposite way, it makes us think of what specific data a thing contains instead of what that thing is.

Pretend you have a game which uses ECS, and this game contains planes and ships, they may share a few traits such as position (where the game object is in the world), velocity (how fast it is), and etc. One day you want to update the game and add floatplanes, which can both fly in the air with its wings and yet float in the water with its hull. This is not a problem with an ECS, if you want your entity to fly, you just add a Fly component or maybe a Gravity component with negative values. If you want it to float, you just add a Float component

image

Due to the nature of the ECS paradigm that allows for new components and systems to be added without conflicting with existing logic makes it suitable for games where many layers of overlapping behaviour will be defined on the same set of game objects, i.e. new behaviour defined in the future.

How does it work in Roblox?

Let’s begin with creating our world first!

local world = Matter.World.new()

In Matter, we construct components using the component function of the Matter namespace.

local Velocity = Matter.component("Velocity")
local Renderable = Matter.component("Renderable")

But these components don’t contain any data yet! When we spawn our entity, we need to call these components again with some data.

local arrowModel = game:GetService("ReplicatedStorage").Assets.ArrowModel
arrowModel.Parent = workspace
local arrow = world:spawn(
    Velocity({ speed = 50}),
    Renderable({ model = arrowModel })
)

Now we want to give some behaviour to our arrows, because right now they would be stationary. So we have to query the world for entities that both contain the data we are looking for, namely velocity and a renderable. We are going to use the component tables to query the world with. Behind the scenes, it is going to look for matching archetypes or a similar set of components and return every entity matching that description.

local function arrowsFly(world)
    for id, vel, render in world:query(Velocity, Renderable) do
        local currentPosition = render.model:GetPrimaryPartCFrame()
        render.model:PivotTo(CFrame.new(currentPosition + Vector3.new(vel.speed, 0 ,0)))
    end
end

The problem right now is that it is going to move 50 studs every single frame, you can imagine it is going to be incredibly fast. Luckily there exists a very useful hook function named useDeltaTime which gives us the time between frames.

render.model:PivotTo(CFrame.new(currentPosition + Vector3.new(vel.speed * Matter.useDeltaTime(), 0 ,0)))

Now let’s make it hurt some people on contact! We want to use the .Touched signal to evaluate whenever the arrow has gotten any contact. Here we will use another useful hook useEvent.

local function arrowsHurt(world)
    for id, vel, render in world:query(Velocity, Renderable) do
        for _, hit in Matter.useEvent(render.Model.PrimaryPart, "Touched") do
            if Players:GetPlayerFromCharacter(hit.Parent) then
                hit.Parent.Humanoid:TakeDamage(5)
            end
        end
    end
end

Lastly we need to run these systems! We will be using the loop export from the Matter namespace. The Loop class accepts multiple arguments which will be passed into every single system (look at each of our functions, they have a world parameter).

local loop = Matter.Loop.new(world)
loop:scheduleSystems({
    arrowsFly,
    arrowsMove,
    arrowsHurt
})

loop:begin({
    default: RunService.Heartbeat -- you can set it to whatever signal you want but commonly you will use Heartbeat
})

Sources

Matter is a great ECS but Jecs design goal is to prioritize performance first. Its query iterators are as fast as iterating over arrays directly without overhead. If that is something that intrigues you, check out the benchmarks under the README: GitHub - Ukendio/jecs

85 Likes

the funny thing is, i was planning on making my own OOP plane(not my first time using OOP but with this type of use case). So this is really useful!

2 Likes

Wish matter was better documented. If you’re coming in from OOP with no prior experience with ECS, the documentation reads like hieroglyphics. From what little I can make out of it, it doesn’t feel as intuitive to use as objects, at least on Roblox.

9 Likes

I pretty much agree, I don’t think this is really necessary for roblox development, but I haven’t had enough experience with it to draw a strong conclusion. I guess it depends on the scope of the project.

2 Likes

Awesome explaination on ECS & Matter, personally Matter has been super beneficial to my development from it’s ease-of-use, decoupled design & in-built debugger.

Matter is a great library, and ECS is a great way of structuring your game. I do wish that more people would use ECS over OOP though.

1 Like

Does Matter have automatic support for parallel luau so we can take advantage of multi threading?
Is this ECS beneficial to performance or is it just serving as another way of structuring games?

Does Matter have automatic support for parallel luau

No, multithreaded Luau can’t work in how you’re thinking it should work in other languages. Luau is a dynamic language and the actor model was made with that in mind to forbid unsynchronized access to garbage collectable tables to prevcnt race conditions.

Is this ECS beneficial to performance or is it just serving as another way of structuring games?

It is not the fastest ECS at the moment, but it is definitely catching up to others. However, the real sell of Matter is for organizational benefits and the ergonomics of its usage. You can read more here. ECS in general will extrapolate problematic performance issues making them easier to find and debug, and Matter also provides a debugger that shows you the performance of each system.

4 Likes

I’m not sure if anyone is actively talking about Matter, since this topic was from like a year ago. But, I have a question. How do I maintain a hybrid use of both RenderStepped and Heartbeat for Systems?

1 Like
  1. Matter is still very active, in fact we have moved it into an organization where multiple people are maintaining the project for it longevity. It is here: https://github.com/matter-ecs/matter/

  2. You need to specify the different RunService phases in the Loop::begin function, and then you have to specify which event that each individual system listens to, otherwise they will default to the “default” field in the table u passed to Loop::begin. You can read more about this here: https://matter-ecs.github.io/matter/api/Loop/#begin

  3. If you have other questions about Matter, there is a channel in the ROSS server for specifically Matter. Roblox OSS Community

1 Like

Thanks for the information! I found what I was looking for anyways in the roomba’s place provided on the repo.

What would this “bottom up” approach look like without using a preexisting framework?

Identify the basic data elements required for the system. Semantics aside, it is really just about how you organise your data, i.e. Struct of Arrays (SoA) where each field of an entity is stored in separate arrays or “columns” vs Array of Structs (AoS) where each struct is stored as elements within an array, arranged in rows that is how classes inherently structure their data members.

1 Like

This is my interpretation of your thread, please give me feedback if I’m incorrect about anything.

You create a two-dimensional array for entity data utilizing two tables:
SoA: Table for Entity Data (Stats, etc)
AoS: Table for Entity Spatial/Instance Data

While OOP is great for reusability and eliminating overcomplicated code by establishing a baseline for data before utilizing it in a framework, making it “top down”, your “entity component system” is pretty much the opposite, adding data as you see fit for each index, therefore making it “bottom up”.

The issue is that with the bottom up approach you need to do a lot more of the heavy lifting, organizing your code so it doesn’t get messy, and that’s where the library you utilized comes in.

I do like the concept, however I’m not personally a fan of relying on unnecessary libraries that exists as a shortcut for organizing my code in an efficient way.

ECS comes in when you have to deal with many objects that have converging behaviour. The traditional approach is to use object-oriented techniques to construct massive and complicated inheritance paths to share a set of behaviour. However, with the additional indirections from code paths, the reusability of code decreases and performance suffers.

Making an ECS is actually really simple, because inherently that is already very similar to what you probably already have, and you just have to rearrange the structure of your data. I am publishing a paper in May on the implementation of an ECS in scripting. Heres a small section of that paper that goes over Luau code.


The ECS utilizes several key data structures to organize and manage entities and components within the ECS framework:

  • Archetype: Represents a group of entities sharing the same set of component types. Each archetype maintains information about its components, entities, and associated records.

  • Record: Stores the archetype and row index of an entity to facilitate fast lookups.

  • EntityIndex: Maps entity IDs to their corresponding records.

  • ComponentIndex: Maps IDs to archetype records.

  • ArchetypeIndex: Maps type hashes to archetype.

  • ArchetypeMap: Maps archetype IDs to archetype records which is used to find the column for the corresponding component.

  • Archetypes: Maintains a collection of archetypes indexed by their IDs.

These data structures form the foundation of our ECS implementation, enabling efficient organization and retrieval of entity-component data.

Functions

The ECS needs to know which components an entity has and provide an interface to manipulate it and search for
homogenous entities from a set of components quickly.

get(entityId, …)

Purpose: The get function retrieves component data associated with a given entity. It accepts the entity ID and one or more component IDs as arguments and returns the corresponding component data.

local function get(entityId: i53, a, b, c, d, e) 
    local id = entityId
    local record = entityIndex[id]
    if not record then 
        return nil
    end

    return getComponent(record, a), getComponent(record, b) ...
end

local function getComponent(record: Record, componentId: i24)
    local id = record.archetype.id
    local archetypeRecord = componentIndex[componentId][id]

    if not archetypeRecord then 
        return nil
    end

    local column = archetypeRecord.column

    return archetype.data.columns[column][record.row]
end

Explanation:
This function retrieves the record for the given entity from entityIndex. It
then calls getComponent(record, componentId) to fetch the data for each specified
component (a, b, c, d, e) from the entity’s archetype which is returned.

entity()

Purpose: This function is responsible for generating a unique entity ID.

local nextId = 0
local function entity()
    nextId += 1
    return nextId
end

Explanation:
Generates a unique entity ID by incrementing a counter each time it is called.

add(entityId, componentId, data)

Purpose: Adds a component with associated data to a given entity

local function addComponent(entityId: i53, componentId: i53, data: unknown)
	local record = ensureRecord(entityId)
	local sourceArchetype = record.archetype
	local destinationArchetype = archetypeTraverseAdd(componentId, sourceArchetype)

	if sourceArchetype and not (sourceArchetype == destinationArchetype) then 
		moveEntity(entityId, record, destinationArchetype)
		-- update query cache
	else 
		-- if it has any components, then it wont be the root archetype
		if #destinationArchetype.types > 0 then 
			newEntity(entityId, record, destinationArchetype)
		end
	end

	local archetypeRecord = destinationArchetype.records[componentId]
	destinationArchetype.data.columns[archetypeRecord.column][record.row] = data
end

Explanation:
This function first ensures that the record exists for the given entity using
ensureRecord(). It then determines the destination archetype from the
current entity archetype and new component using archetypeTraverseAdd().
It will move the entity to a new archetype or if the entity does not have a record yet, initializes
the record by calling newEntity(). Lastly it updates the data for the component in the
corresponding column of the archetype’s data.

If you want to learn more, go to the Roblox OSS discord server and ping me @hi_marcus in the #Matter channel and I will be there to answer any questions.

7 Likes

I still don’t exactly get it, why would I want to create an entire component in my class that only stores a few values and no functions?

If I have a gun, and it has a magazine, why would I create a whole magazine component with a value like AmmoCount = 30, when I could just declare that value in the main gun class.

I feel like ECS is very specific and niche, If I do use it, it would only be on a few things, such as a HostileAI and a NeutralAI or something similar, but I wouldn’t try and make a whole gun system out of it, especially since only other guns will use gun components. It makes sense using position and velocity, but I couldn’t imagine making a component that would only be used in one entity/object ever.

ECS is more effective than traditional OOP if you think your game would have a lot of complexity that is difficult to achieve with OOP.

Inheritance is terrible because it’s easy to abuse it.

Not only is it slower, it’s easy to put yourself in a design problem where you can’t decide whether:

  • a certain class should be derived from other classes (which gives you a big inheritance tree that is difficult to manage and debug),
  • or a separate class of its own (which causes organizational issues).

Back to the gun example, what if you now have a gun that has an added mechanic (eg, shoots 3 rounds instead of 1, overheats, amp-up time, etc)?

What I would typically do is each gun has a config file, with a type such as “Single, Auto, Burst” etc
And in the main class I would just check which one I should be using.

if self.Type = "Auto" then
    while self.MouseDown do
        self:Fire()
    end
else
    self:Fire()
end

That’s still not a very good way to illustrate how ECS gets more effective as your classes get more complex, and you’re missing my point.

You can achieve complexity within your classes using standard OOP, but that usually comes at a sacrifice of performance and manageability.

For example, what if you now have a gun that overheats?
Or what if a gun has some sort of charge-up, where it doesn’t shoot until you hold down the trigger long enough?
Or if a gun takes in multiple ammo types?
Or if a gun is a hitscan and not projectile-based?

  • Do you put all that logic into your main gun class, and also inadvertently put all those mechanics into every other gun that you don’t intend to have those mechanics on?
  • Or do you create a new gun class, even though now both gun classes have about the same logic otherwise?
  • Or if you’re a bit more clever, you use inheritance, but also cause a lot of headache for yourself when you plan on debugging your inheritance tree in the future?

OOP + composition can accomplish anything ECS can. ECS just seems more annoying and also… it runs in a loop? lol wtf

If I have a gun, and it has a magazine, why would I create a whole magazine component with a value like AmmoCount = 30 , when I could just declare that value in the main gun class.
I feel like ECS is very specific and niche, If I do use it, it would only be on a few things, such as a HostileAI and a NeutralAI

ECS is not a god-pattern, it does not fit every game. I should say that your example is rather trivial with ECS though, you do not necessarily need a magazine component, how you structure your logical components is still completely up to you. The point of ECS is to allow you to search for any arbitrary combinations of said components and iterate them in the fastest way possible.

Also I don’t really think it is true that you will only ever use a gun component for a single entity/object. You may have multiple types of weapons such as an assault rifle, pistol, sniper etc. They could all have specific data such as them being guns. You may have a system that needs to check that every gun needs to reload once they deplete their magazines. You have now generalized this behaviour for free because everything is reusable via composition.

If you have more questions, don’t hesitate to ask them in our ECS community: Matter ECS, we don’t bite!

1 Like