RemoteTable - Replicate tables over the network!
RemoteTable is a special type of network library which uses proxy tables to detect changes to a table and send only the changes over the network to reduce network usage. It uses Packet as the networking library to optimize data usage via buffers.
Features
- Automatically replicate changes via proxy tables
- Hash paths to reduce network and cpu usage
- Batch updates and rate limit
- Handle and optimize dynamic indexes created at runtime
- Compatible with ProfileStore
- Strictly Typed
- Multi script support
Limitations
- Not very efficient for relatively large (20-30) item arrays that needs to preserve order
- Keys for tables can only be number or string and values can only be types Packet.Any supports
- Can’t use
table.insertandtable.removeor other standard table library functionality
[Instead useRemoteTable.Insert,RemoteTable.Remove,RemoteTable.FastRemove] - Only 255 tokens can be registered at the same time. [This can be changed by editing Packets module]
- Using ChildChanged signal on an ordered array is really unreliable as ChildChanged relies on paths getting destroyed or added. [Feel free to reach me out on why it is this way]
Quick Start
Setup
- Get the module here.
- Put it under replicated storage.
TIP: You can link your own Packet module to further benefit from batching.
Advanced Setup
You can edit you config file to further customize RemoteTable.
Config file is just a module that returns a table and is located at RemoteTable.Shared.Config.
Documentation for each table property:
Packet -> require(Path.To.Your.Packet)
UpdatesPerSecond -> How often changes are sent.
HashTableSize -> The hash table size for RemoteTable to operate on.
INFO: Each path gets a unique id from the hash table. Default value for the hash table space is 2000 and it is safe to use up to %70 (1400) of this hash table space. For most use cases 1400 paths will be enough for something like user data but incase there is an inventory system with a lot of items you can change this value using this formula
max_paths * 1.4.
WARNING: This unique id is unfortunately per RemoteTable module NOT per created remote table. Fortunately players usually get the same user data template so each identical path can use the same id not adding to the total unique path count.
Server.lua
local RemoteTable = require(game.ReplicatedStorage.RemoteTable.Server)
-- or alternatively require(game.ReplicatedStorage.RemoteTable).Server
local ExampleTable = {
Table = {
Key1 = "Path of this value is 'Table\Key1' ",
},
Array = {"A","B","C"} -- number indexes
}
game.Players.PlayerAdded:Connect(function(player: Player)
--Create a unique token for each player
local token = "PlayerData"..player.UserId
--Initialize a new remote table with the given token
local remote_table = RemoteTable.ConnectTable(ExampleTable, token)
--Give client permission to listen to the remote table
remote_table:AddClient(player) --or RemoteTable.AddClient(token, player)
task.wait(2) -- wait for client to test if changes replicate
remote_table.Data.Table.Key1 = player.UserId -- should replicate
end)
game.Players.PlayerRemoving:Connect(function(player: Player)
--Release token for future players
local token = "PlayerData"..player.UserId
RemoteTable.ReleaseToken(token)
end)
Client.lua
-- // Services
local Player = game:GetService("Players").LocalPlayer
local ReplicatedStorage = game:GetService("ReplicatedStorage")
-- // Modules
local RemoteTable = require(ReplicatedStorage.RemoteTable.Client)
-- or alternatively require(ReplicatedStorage.RemoteTable).Client
-- // Globals
local TOKEN = "PlayerData"..Player.UserId
local data = RemoteTable.WaitForTable(TOKEN)
-- Listen to Data.Table.Key1
RemoteTable.GetValueChangedSignal(TOKEN, {"Table","Key1"})
:Connect(function(new, old)
print("Value changed from", old, "to", new)
end)
--TIP: Enable "Show Tables Expanded by Default" for a better visual feedback
while task.wait(0.2) do
--Data changes automatically
print(data)
end
TIP: It is usually recommended to have 1 remote table token per player but the library is designed in a way that is more flexible. Multiple players can listen to the same token.
Bonus: Example with ProfileStore
DataManager module
local DataManager = {}
-- // Services
local Players = game:GetService("Players")
local ReplicatedStorage = game:GetService("ReplicatedStorage")
-- // Modules
local ProfileStore = require(game.ServerScriptService.ProfileStore)
local RemoteTable = require(ReplicatedStorage.RemoteTable.Server)
local Signal = require(ReplicatedStorage.RemoteTable.Shared.GoodSignal)
-- // Globals
local PROFILE_TEMPLATE = {
Cash = 0,
Items = {},
}
local PlayerStore = ProfileStore.New("PlayerStore", PROFILE_TEMPLATE)
local PlayerDatas = {} :: {[Player]: {
Profile: typeof(PlayerStore:StartSessionAsync()),
Data: typeof(PROFILE_TEMPLATE),
}}
DataManager.PlayerDatas = PlayerDatas
DataManager.DataLoaded = Signal.new()
function DataManager.GetDataFromPlayer(player: Player): any
local remote_table = RemoteTable.GetRemoteTable("PlayerData"..player.UserId)
if not remote_table then return nil end
return remote_table.Data
end
local function PlayerAdded(player: Player)
local data_token = "PlayerData"..player.UserId
local profile = PlayerStore:StartSessionAsync(`{player.UserId}`, {
Cancel = function()
return player.Parent ~= Players
end,
})
if profile ~= nil then
profile:AddUserId(player.UserId)
profile:Reconcile()
profile.OnSessionEnd:Connect(function()
PlayerDatas[player] = nil
player:Kick(`[{script.Name}]: Profile session end - Please rejoin`)
--Release the token
RemoteTable.ReleaseToken(data_token)
end)
if player:IsDescendantOf(Players) then
-- Connect and add client right after player data loads
local remote_table = RemoteTable.ConnectTable(profile.Data, data_token)
remote_table:AddClient(player)
-- Override profile.Data with the tracked read-only table
-- THIS IS READ ONLY DO NOT EDIT
profile.Data = remote_table.ReadOnlyData
PlayerDatas[player] = {
Profile = profile,
Data = remote_table.Data -- Data safe to edit
}
print(`[{script.Name}]: Profile loaded for {player.DisplayName}!`)
DataManager.DataLoaded:Fire(player, remote_table.Data)
else
profile:EndSession()
end
else
player:Kick(`[{script.Name}]: Profile load fail - Please rejoin`)
end
end
for _, player in Players:GetPlayers() do
task.spawn(PlayerAdded, player)
end
Players.PlayerAdded:Connect(PlayerAdded)
Players.PlayerRemoving:Connect(function(player)
local profile = PlayerDatas[player].Profile
if profile ~= nil then
profile:EndSession()
end
end)
return DataManager
Server
-- // Services
local ServerScriptService = game:GetService("ServerScriptService")
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local Players = game:GetService("Players")
-- // Modules
local DataManager = require(ServerScriptService.DataManager)
local RemoteTable = require(ReplicatedStorage.RemoteTable.Server)
DataManager.DataLoaded:Once(function(player: Player, data)
-- Add 100 cash each join
data.Cash += 100
-- Grant a ticket to player everytime they join
RemoteTable.Insert(data.Items, {Name = "Ticket", Value = 10})
end)
Client
-- // Services
local Player = game:GetService("Players").LocalPlayer
local ReplicatedStorage = game:GetService("ReplicatedStorage")
-- // Modules
local RemoteTable = require(ReplicatedStorage.RemoteTable.Client)
-- // Globals
local TOKEN = "PlayerData"..Player.UserId
local data = RemoteTable.WaitForTable(TOKEN)
--TIP: Enable "Show Tables Expanded by Default" for a better visual feedback
while task.wait(0.2) do
--Data changes automatically
print(data)
end
Documentation
Server
--Requiring the module
local RemoteTable = require(RemoteTable.Server) -- or require(RemoteTable).Server
RemoteTable.ConnectTable
- Creates a new remote table and initializes it
@param tbl: Table to be tracked
@param token_alias: String token_alias for the token
@param players: A player or a player array to be added to the remote table
@return RemoteTable<T>: Newly created remote table object
RemoteTable.AddClient
- Authorizes a client to listen to a token
@param player: Client to be added
@param token_alias: String token_alias for the token
RemoteTable.RemoveClient
- Disconnects a client and removes permissions to listen for changes.
@param player: Client to be added
@param token_alias: String token_alias for the token
RemoteTable.ReleaseToken
- Releases the token and disconnects the remote table associated with the token.
@param token_alias: String token_alias for the token
RemoteTable.GetRemoteTable
- Gets the remote table from token alias.
@param token_alias: String alias of the token
@return RemoteTable<T>?: returns nil if remote table does not exist
RemoteTable.GetProtectedTable
- Gets the protected read-only data to be used as rvalue
@param value V: ProxyTable to be used to retrieve the protected value
@return value V: The read-only data
RemoteTable.Insert
- Same as table.insert
@param tbl {V}: Table to insert to
@param value V: Value to be inserted
RemoteTable.Remove
- Same as table.remove
- WARNING: Very expensive to execute and replicate to the client
@param tbl {V}: Table to insert to
@param pos number?: Position to be removed from
RemoteTable.FastRemove
- Removes an element from the array and replaces it with the last one
- NOTE: Recommended when order of elements does not matter
@param tbl {V}: Table to insert to
@param pos number?: Position to be removed from
Client
-- Requiring the module
local RemoteTable = require(RemoteTable.Client) -- or require(RemoteTable).Client
RemoteTable.WaitForTable
- Returns the table if available, waits for it if not.
@param token_alias: String alias of the token
@param timeout: Timeout in seconds. Returns nil after timing out
@return data: Ready-only replicated table.
RemoteTable.GetValueChangedSignal
- Gets the signal that fires when value of the path changes.
@param token_alias: String alias of the token
@param path_list: A string array representing the desired path
@return Signal: Signal that fires (new, old) data
RemoteTable.GetChildChangedSignal
- Gets the signal that fires when a child is Added / Removed from the table.
@param token_alias: String alias of the token
@param path_list: A string array representing the desired path_list
@return Signal: Signal that fires ("Added" | "Removed", key, value) data
RemoteTable.DisconnectValueChangedSignal
- Stops listening to value changed events for that path.
@param token_alias: String alias of the token
@param path_list: A string array representing the desired path_list
RemoteTable.DisconnectChildChangedSignal
- Stops listening to child changed events for that path.
@param token_alias: String alias of the token
@param path_list: A string array representing the desired path_list
Versions
1.0
- Initial Release.
- Added XXH32 as the main hashing algorithm. Thanks to @XoifaiI.
1.1
- Fixed simple typo which allowed everyone to listen to any table.
- Simplified netcode and client/server communication.
- Added universal WaitForTable. Can be used from anywhere to get a remote table.
- Fixed re-hashing for each connected client.
- Fixed client not being able to listen to same path from different tokens.
1.2
- Fixed release token trying to release the non-existent token with id 0.
- Added ValueChanged and ChildChanged signals to listen to.
1.2a
- Removed dependency to promise (custom timeout function).
- .Changed events now .WaitForTable by default.
1.3
- Fixed Remove, FastRemove and Insert to function properly with nested proxy tables. Thanks to @artspb111 for finding the bug!
- Added .GetProtectedValue which allows users to get the protected value that RemoteTable works with under the hood.
- Small fix to license. Same GPLv3 applies.
1.4
- A bit of code refactor
- Fixed timeout logic. Thanks to @taazereb for finding the issue
- Fixed an issue where if player left while connection is being established it wouldn’t release the player properly for other listeners.
- Few improvements to race conditions. More consistent .WaitForTable behavior.
- Fixed .WaitForTable not being able to wait for data again.
- Changed license to LGPLv3.
