Harmony - A fully typed replacement to Replica with additional compression, in order to better facilitate replication

In my opinion, Replica is a bad module. There are a lot of things which are in my opinion unnecessary and overcomplicated. So, I’ve redesigned a similar module in order to achieve some things which Replica is not able to:

  1. Being fully typed - By far one of the biggest improvements in Developer Experience, using Harmonic is as simple as modifying a table and the changes on that table will be replicated to all targetted clients.
  2. Compression is built in - using a dictionary based approach in order to compress paths to a minimal string of 2 bytes, its possible to save bandwidth without modifying developer experience and maintaining expected functionality. This is most apparent when attempting to modify deeply nested tables.
  3. In general, its more intuitive. Additional bloat of firing remotes and such has been discarded as it is not the aim of this module to provide that ability. This is a simple module which aims to replicate your data in a simple way.

Use cases & Examples:

-- Server
-- Define a struct that describes the structure of player data
local PlayerStats = Harmonic.struct("PlayerStats", {
	Health = 100,
	Coins = 0,
	Inventory = {
		Items = {},
		Equipped = false,
	},
})

-- Create a data entity for a specific player
game.Players.PlayerAdded:Connect(function(player)
	local dataEntity = Harmonic.new(PlayerStats, Harmonic.ReplicationTarget.PLAYER(player))
	
	-- Just modify the table — replication happens automatically
	dataEntity._trove:Add(task.spawn(function()
		while player.Parent do
			dataEntity.Coins += 1
			task.wait(5)
		end
	end))
end)

-- Client
local PlayerStats = Harmonic.forward("PlayerStats")

-- Listen for any new data entities created for this struct
Harmonic.GetDataAddedSignal(PlayerStats):Connect(function(dataEntity)
	print("Received PlayerStats dataEntity!")

	-- Observe live changes
	dataEntity.Changed:Connect(function()
		print(`Coins: {dataEntity.Coins}`)
	end)
end)

Need to extend a struct? You can use extendStruct:

-- Server
local BaseEnemy = Harmonic.struct("BaseEnemy", {
	Health = 100,
	Damage = 10,
})

-- Create a new type of enemy derived from the base
local BossEnemy = Harmonic.extendStruct("BossEnemy", {
	Health = 500,
	SpecialAttack = true,
}, BaseEnemy)

-- Create an instance for replication
local dataEntity= Harmonic.new(BossEnemy)
dataEntity.Health -= 100
-- Client
local BossEnemy = Harmonic.forward("BossEnemy")

Harmonic.GetDataAddedSignal(BossEnemy):Connect(function(enemy)
	print("Boss enemy replicated with", enemy.Health, "HP")
end)

Have a special Replication Target? You can add filters:

-- Only replicate to players in a specific team
local TeamAFilter = Harmonic.ReplicationTarget.CONDITION(function(player)
	return player.Team and player.Team.Name == "TeamA"
end) --> CONDITIONs will recheck the replicant list every Stepped. If this behaviour is not desired and you would prefer to manually update it, use LAZYCONDITION instead.

local MatchState = Harmonic.struct("MatchState", {
	Timer = 300,
	WinningTeam = false,
})

local dataEntity = Harmonic.new(MatchState, TeamAFilter)

I have used the Harmonic.forward in many of these cases in order to compress down the examples, however, this comes with voiding the typed data, so the best practice with this module is to create a module list with ALL harmonics and then to use those in order to preserve the data type.

-- HarmonicStructs.lua
local BaseEnemy = Harmonic.struct("BaseEnemy", {
	Health = 100,
	Damage = 10,
})
local BossEnemy = Harmonic.extendStruct("BossEnemy", {
	Health = 500,
	SpecialAttack = true,
}, BaseEnemy)
local MatchState = Harmonic.struct("MatchState", {
	Timer = 300,
	WinningTeam = false,
})
local PlayerStats = Harmonic.struct("PlayerStats", {
	Health = 100,
	Coins = 0,
	Inventory = {
		Items = {},
		Equipped = false,
	},
})

return {
	PlayerStats = PlayerStats,
	BaseEnemy  = BaseEnemy,
	BossEnemy = BossEnemy,
	MatchState = MatchState 
}

and then use it like so:

game.Players.PlayerAdded:Connect(function(player)
	local dataEntity = Harmonic.new(HarmonicStructs.PlayerStats, Harmonic.ReplicationTarget.PLAYER(player))
	
	-- Just modify the table — replication happens automatically
	dataEntity._trove:Add(task.spawn(function()
		while player.Parent do
			dataEntity.Coins += 1
			task.wait(5)
		end
	end))
end)
-- Client
Harmonic.GetDataAddedSignal(Harmonies.PlayerStats):Connect(function(dataEntity)
	print("Received PlayerStats dataEntity!")

	-- Observe live changes
	dataEntity.Changed:Connect(function()
		print(`Coins: {dataEntity.Coins}`)
	end)
end) --> properly preserves the type of dataEntitynow.

All Replication Targets:

Harmonic.ReplicationTarget.PLAYER(plr : Player) --> replicate to one person. This will also destroy all bound data entities upon the disconnection of this one plr.
Harmonic.ReplicationTarget.PLAYERS(plr : {Player}) --> replicate to people in persons (changes to the plr table is supported)
Harmonic.ReplicationTarget.LAZYPLAYERS(plr : {Player}) --> replicate to the plrs in the table, and freeze it. No changes supported
Harmonic.ReplicationTarget.CONDITION(cond : (Player)-> boolean) --> replicate to the plrs which satisfy the condition. Supported when condition state for player changes
Harmonic.ReplicationTarget.LAZYCONDITION(cond : (Player)-> boolean) --> replicate to the plrs which satisfy the condition. Does not detect changes in condition state for players.
Harmonic.ReplicationTarget.ALL() --> 

Additionally, you should destroy your data entities after you are done with them:

dataEntity._trove:Add(function() print("NOOOO, SOMEONE LEFT YOUR BEAUTIFUL GAME!") end)
dataEntity:Destroy()

Things to consider:

  1. Harmonic does not support Harmonies which have keys set to nil. For this, use false. If many people finds this as a downside, then it is possible to implement an inherent nil type like Harmonic.None() in order to solve this.
    Harmonies.harmonic({ Key = nil }) wont work. Key will not be changeable and throw an error saying that the field does not exist. Instead change Key = false

  2. Harmonic does not support cyclic tables.

Find the module here: https://create.roblox.com/store/asset/71924586834389/Harmonic-v001

This module is in beta, report any bugs which you have found, or any feature requests you have,

13 Likes

Fixed distribution status… sorry.

Fixed a bug where if ReplicationTarget was not provided the function would error. (SORRY!)
Added new function to resonances:

Resonance:Overwrite(NewData);

Completely overwrites the data in memory with the NewData, if a field in NewData is not provided it will instead use the default values from the harmony.

New update introduces partial array sending (I forgot to add this before, now instead of resending the entire array, it will send the updates parts)

Fixed some bugs (kinda forgot what some were.)
Added new ResonanceSettings to give developers better control over how often the updates are broadcasted to clients (set at a default 20hz)

Some things to note:
The way partial array sending works is definitely slow - it basically figures out the difference between the old and updates arrays. This is intended, It was possible for me to code metatables in order to track changes reactively, however I was discouraged from this for two main reasons:

  1. Setting a property / field to nil will not trigger a metatables __newindex method call, making it impossible to track when a field has been removed.
  2. I want some sort of gurantee that no matter what changes are made, they are replicated. And in the case that you want to try to print out one of the “end nodes”, they are given to you in a proper - non metatable way in order to avoid confusion. (Also in general I find that passing around metatables without knowing or it being obvious / relevant that they are metatables sometimes sucks.)

However, there is one major drawback to this method, which is that we’ll have to find the DIFFERENCE between two tables, which does get slow when starting to go above 180k list items (>=3ms on my machine), so for giant arrays this may cause some lag.
In order to somewhat mitigate this issue, the new ResonanceSettings should also allow you to limit how often the mutation checker (thats what im calling it) should be called. (which’d then improve performance).

In the future, there might be additional ways of inserting to a table the non-roblox way in order to save more performance but sacrifice typing or having to learn some new syntax, which’d bypass the mutation checker, but for now its here to stay, and shouldn’t cause any major problems.

New functions added which allows you to specify if a field should be replicated or not,
and removed some bugs as I was working with it,

Do:

Resonance:SetReplicationDisabled({
     path = {
            to = true;
     }
}) --> will not replicate the path.to variable (helpful to save bandwidth by not replicating irrelevant fields)

…And yet, I can hardly understand your module’s interface compared to Replica. What’s a “chord”? A “harmony”? A “Resonance”?? All of this is in terms of music. A good metaphor can be nice, but these are all metaphors and it turns out more complicated than if you just called them by what they are.

I may not be speaking for everyone, but I do find it intriguing that nobody else has responded to a seemingly well put together resource. Maybe nobody understands it?

1 Like

Does this have server changed events?

1 Like

I suggest you add a Resonance.Data property pls :folded_hands:

I can definitely see the issue with the naming convention and honestly i do agree - i am bad at naming things and thats not a real surprise I could potentially change / add aliases such that backwards compatability remains but i still wouldnt really know what to call my methods - if you could pitch on some ideas id be more than interested in implementing them and if you do not understand what something does i could also explain it

You can get the current data using Resonance:__get(), the main reason why its done like this is to prevent access to the raw table unless for debugging

Not yet since i havent needed it but is it a simple addition. Ill get it done whenever i get home

Regarding the recent concerns - i’ve made some changes, and changed the link of the Harmonic module to https://create.roblox.com/store/asset/71924586834389/Harmonic-v001 due to hopefully avoiding and backwards compatability issues with some of the changes i’ve made in this update - First of all:

  1. All methods have more descriptive and less ethereal function aliases now:
Harmonic.struct == Harmonic.harmony,
Harmonic.extendStruct == Harmonic.chord,
Harmonic.GetDataAddedSignal == GetResonanceAddedSignal
Harmonic.GetDataAddedSignalN == GetResonanceAddedSignalN
  1. Some updates to how the functions Harmonic.GetDataAddedSignal and Harmonic.GetDataAddedSignalN work:
  • Now, instead of returning a signal to bind to, you’ll have to give it a callback. This fixes a problem where when an data entity is already created but you are connecting to the added signal, that data entity is missed. So, this should help solve that as it makes sure the callback runs all existing data entities aswell.
--- BEFORE ---
local Connection = Harmonic.GetDataAddedSignal(Structs.PlayerData):Connect(function(DataEntity)

end) -- this'd miss some entities out.

--- AFTER ---
local Connection = Harmonic.GetDataAddedSignal(Structs.PlayerData, function(DataEntity)

end) --> this function will be called with preexisting data entities

-- returns exactly the same thing!
  1. Server didn’t have Changed functionality.
  • I didn’t realise that this was a vital component so I’m sorry for not including it in the release, this is because my game which is built on top of the module uses another of my modules to track data changes on the server and didnt need this functionality. I’ve added the Changed functionality to data entities created on the server. Note that setting replication disabled on a path will make it so the changed event does not fire when you change that field (this is due to optimisations, this could possibly be changed but i think it is currently fine)

The main post will shortly be rewritten in order to fit with the current module’s best practices, and show the new usage of the dataaddedsignal

2 Likes