Hello All!
I’m not a fan of object oriented programming, so I’m practicing using other paradigms. Currently I’m trying to make a game using data oriented programming with an Entity Component System (ECS) framework to help reuse code. The question I have for you today is this: how would you connect the server’s ECS to the clients’ ECS, and why? I’ve intentionally not described how I setup the ECS framework to not limit your responses: I’m open to changing it!
Thanks!
The way I have mine set up currently:
The server and each client each possess an instance of a class called EntityManager. EntityManager is a singleton - its main purpose is to contain the entity system’s data structures, but it also provides methods for getting collections of entities and/or components.
This class has different methods available depending on if its loaded on the client or the server. On the server, there are methods for adding/removing/changing entities/components on specific clients; on the client, it merely has a method to request a component change.
There are other things I’m glossing over, but that’s the gist of it - you can check out my implementation here. The version on github is a bit different than my current test build so some stuff is probably wrong/badly factored, but you should be able to see the overall architecture
Looks great! I have four questions for you (please don’t feel obligated to answer them, consider them prompts for my selfish benefit 
) :
Currently I’m experimenting with using tags as components on entities (but not limited to just tags, they then just don’t replicate) and remote events to synchronise extra data. In this case my entity id’s are just roblox instances which then maintains the same id across server/client for you. Seems to be working quite nicely so far.
The server is responsible for loading entities to clients with LoadForPlayer(); this allows the server to keep a record of what networked entities actually exist on each client. When a server side system calls KillEntityForPlayer(), UpdateComponentForPlayer(), or KillComponentForPlayer(), these methods first check if the passed entity has been loaded for that client, and then fire that client’s RemoteEvent (which is referred to as the client’s “EntityUpdater” internally).
In my implementation, systems are just functions. When StepSystem() is called by the main loop, it passes a reference to EntityManager to the system in question. The system is then responsible for getting whatever data it needs via GetAllComponentsOfType() or GetComponentAddedSignal(). The main drawback with this setup is that EntityManager is stamp-coupled to each system, but I have yet to determine if this will actually cause any problems
I’m going to be honest with you - I haven’t. However, it’s almost certain that I will as I continue to eat my own dog food
One big strength: the server is always in charge. Clients have zero knowledge of entities or components that they haven’t explicitly been given by the server. This could also be a weakness, depending on your perspective; it means that most (if not all) of the networking that Roblox normally does for you, you have to do yourself.
edit: I completely forgot to mention why I did it this way in the first place. Essentially, this setup gives a robust way to do selective replication - something I needed for an upcoming project
Do you also use an explicit method to update components, or do you perhaps have a method of passively annotating what should be kept up to date and have a system that handles replication?
I just wanted to pop in again and say that if you haven’t already, you definitely should read Adam Martin’s series of blog posts (including all of the comments) - I found it very helpful when I was initially building WorldSmith
The nice thing about components is that they are supposed to be super dumb. Any logic which is changing the values of components should be held in systems which can then also handle all of the replication. In my opinion this creates a very intuitive way to handle replication as you are held back from making something which auto-replicates all values (which can be both inefficient and difficult to code) and all replication logic is also separated/encapsulated within each system.
Oddly enough I don’t find too much need to replicate component values. The most important part is often if a component exists or not (which roblox can handle for us via the tag system). The actual component values are often only really relevant on the server or client and don’t need to be replicated. This is primarily because a lot of the effects from systems (when you are using roblox instances as entities) will apply changes to roblox instances which will also replicate for you.
A super quick example of this could be a DamageComponent which is used to apply damage to an entity. Lets say you shoot someone client side and this damages them. It’s tempting to apply a DamageComponent with a value of 10 to that entity. You would then need to replicate this value of 10 along with the assignment of the DamageComponent to the entity.
Conveniently this method of setting up ECS (roblox instances + tags) makes this very frustrating to do. This is convenient because it’s making bad practice frustrating as this is akin to trying to just damage players directly from client-side. To do this we would have to have some method of telling the server to add the tag/component (as the server maintains authority over tags) as well as the damage value; for the server to then immediately apply the damage and remove the component. If we are doing this then why not just skip the component entirely?
Yes! This is the essence of an ECS, but what I believe what @IdiomicLanguage is suggesting is some sort of “Networked” component which some NetworkSystem is interested in. This system would presumably just send the state of an entity with a Networked component to client(s) over the network at some rate to keep stuff in sync.
I think this is a bad idea for a few reasons:
What these observations should mean to us is that the concept of networking doesn’t neatly fit into a simplistic ECS architecture; the way we’re thinking about the problem is wrong. A “bigger” solution is needed!
This sounds a bit similar to how the first version of my ECS operated. In that model, components were represented by Folder instances filled with ValueBase instances, and entities were represented by referenced instances (tagged “component” and “entity,” respectively). For comparative purposes, here’s what the client did (the server did something very, very similar):
function WorldSmithClientMain:_setupEntityComponentMap()
CollectionService:GetInstanceAddedSignal("entity"):connect(function(entity)
self._entityComponentMap[entity] = {}
end)
CollectionService:GetInstanceRemovedSignal("entity"):connect(function(entity)
self._entityComponentMap[entity] = nil
end)
CollectionService:GetInstanceAddedSignal("component"):connect(function(component)
local entity = component.Parent
if self._componentEntityMap[component.Name] == nil then
self._componentEntityMap[component.Name] = {}
end
if self._entityComponentMap[entity] then
self._entityComponentMap[entity][#self._entityComponentMap[entity] + 1] = component
self._componentEntityMap[component.Name][#self._componentEntityMap[component.Name] + 1] = component
end
end)
CollectionService:GetInstanceRemovedSignal("component"):connect(function(component)
for _, v in ipairs(self._componentEntityMap[component.Name]) do
if v == component then
self._componentEntityMap[component.Name][v] = nil
end
end
end)
local assignedInstances = CollectionService:GetTagged("entity")
local componentTags = CollectionService:GetTagged("component")
for _, entity in pairs(assignedInstances) do
self._entityComponentMap[entity] = {}
for i, v in ipairs(entity:GetChildren()) do
self._entityComponentMap[entity][i] = v
end
end
for _, component in pairs(componentTags) do
if self._componentEntityMap[component.Name] == nil then self._componentEntityMap[component.Name] = {} end
self._componentEntityMap[component.Name][#self._componentEntityMap[component.Name] + 1] = component
end
end
(if you want to see the whole thing, here it is; again, keep in mind that there are many things wrong with it)
This is fairly nice. When a server side system changes the state of a component (i.e. a ValueBase within a Folder), it is automatically replicated to all clients. When a client changes the state of a component, it only occurs for that specific client. This is good - it’s what we should expect from this implementation. However, there are some problems with this approach:
YieldUntilComponentLoaded()
Can we do better? Hell yes.
So, we’ve identified that we want the client to construct its own state, and we want to be able to update game state for only specific clients. To do this, we’re going to have to dispense with the notion that an entity is equivalent to an instance. If an entity isn’t an instance, then what is it?
An entity is a number (or a string, or any other type which can be made unique). It’s just a way to tag each game object; a way to tell our ECS that “this object’s state is located here in our master lists.” Instances do indeed give us a way to do this, but we’ve already decided that the way they behave is undesirable for our purposes.
With this in mind, we’re going to place in each instance some unique-valued ValueBase which marks that instance as a coarse game object (we’ll also tag the instance itself with CollectionService so it’s easy to find). We now have a way to associate a single, unique value with an instance within our ECS (but remember that this instance is not an entity!).
One interesting consequence of this: entities no longer have to be associated with instances at all. This allows for the creation of what I like to call abstract entities; that is, entities that do not correspond to anything in the instance tree (my current implementation doesn’t support these, but if I find a use for them it definitely will).
Sounds scary, right? It needn’t be. We’re going to use what Adam Martin refers to as the “CTRL+D” method.
The server will simply create a copy the current game state (or part of it, if that’s what we want), and send it over the network to the client. The client’s ECS then puts all of this data into its master lists. In my implementation, the server sends over an instance or instances (containing the previously created unique-valued ValueBase) and either a partial or complete copy of the game state. Then, the client loops over those instances to internally associate them to the unique value, and creates the corresponding data structures in its master lists.
The client still needs some way to reconcile its own state with the server’s state. We’ll supply the client with a method to request a state change. In my implementation, the server’s systems are responsible for handling these requests via GetClientRequestSignal(). Clients have no way of affecting the server’s state unless this method is implemented by at least one server-side system. Thus, the server is in complete control of whether the request is actually honored - if the data is bad, we can simply ignore the request and send the correct state to that client. Again, this is all handled within the systems; there is no magic networking layer at work here.
Thank you! I have an implementation question for you:
So if the server’s entity IDs and the clients’ entity IDs are out of sync, then how does the server communicate command to remove existing entities or components? Perhaps a networkID component which is simply an ID which is only set by the server?
Also, here is my implementation so far:
One of the first things you’ll note (since the file has been truncated there as if for emphasis xD) is that I used a very data-oriented approach to the point where this could almost be made into a database with each method turned into a SQL query.
Ah, very pure - I particularly like your implementation of assemblages (something I haven’t quite got around to doing)
In the client-server model I’ve implemented, the server has two jobs:
The server has no knowledge of and cannot interact with entities that clients have created locally, only the ones it has loaded to them. A client could remove an entity that the server has loaded to it and therefore be “out of sync,” but this isn’t necessarily a bad thing, and Roblox indeed already works this way (with network filtering)
The key thing that makes all this work is the fact that the entity IDs are globally unique; the use of HttpService:GenerateGUID() virtually guarantees that the server’s authoritative IDs (i.e., the ones that exist on server init, or the ones it creates during runtime) will never be duplicated by a client when it creates local entities. My ECS went through an iteration where I used a plain integer in place of a real GUID, and this also happened to be the version I began to implement networking… needless to say, “dupes” are very very evil and that version has never seen the light of day 