Chilly's Datastore "Framework"

This datastore system is very different to anything (I’ve seen) on the forum (ie ProfileStore/ProfileService, Suphi’s Datastore Module & Datastore2)

As a heads up, this community ressource is not for beginners. The module provided is like a foundation, and you will be expected to build onto it, in your own way, to fit the needs of your game

I also encourage anyone who wants to take this and turn it into an abstracted module to do so. I do not plan on doing that myself, as it would be time consuming, and I want to offer a module that isn’t limited by abstraction

(I also did not hold back writing it, you will see some unconventional scripting)




Section 1, The what and the how:

As a heads up, if you like breaking rules, you will love this :P

The idea of this datastore system is to, rather than overwrite the data (ie :SetAsync(Player.UserId .. "_Bobux"), or the :UpdateAsync() equivalent), we can update the data.
(And :UpdateAsync() is made just for that, but you need to actually use the passed argument lol)

The most basic example:

local function UpdateBobux(BobuxDiff)
	Datastore:UpdateAsync(Key,function(Data) 
		Data = Data or {}

		Data.Bobux += BobuxDiff
			
		return Data
	end)
end

Ignoring the lack of pcalls, what differs here from the usual use of :UpdateAsync() is that, instead of sending the player’s balance to the function, the amount of money earned or lost (positive or negative value) is sent, and added onto the previously saved value

> In theory, this structure allows for:
No session Locking, If data is saved in the wrong order, nothing is overwritten, so it’s fineeee
Playing/Saving before the data loaded, For similar reasons, the data you got before your old data loaded will simply be added to your old data when saving/loading. (though, this doesn’t really work for games with plots and such)

This feels so wrong, and I absolutely love it

This approach does have some downsides though. For example, updating plot data will be much harder than simply saving the whole thing, and some things might be impossible to handle through updates (such as keybinds or settings, but these are simply overwritten as dataloss is inconsequential)

Theory is one thing,
> In practice:

Let’s simulate a worst case scenario. Two players sharing the same datastore, playing at the same time.
I’ve creating a test place with a “forbidden datastore” (That’s how it’s called in the test place lol). It’s a typical Currency & Inventory datastore, but every player shares the same datastore key

Note that the data has to be saved or loaded before the other player sees the changes. The system is not made for such cases, as they aren’t expected to occur often. MessagingService could be used to let servers tell others to update if needed

You might also notice that the amount of items ended up in the negative. This is expected. Obviously desync between servers (here it’s the same server, but shhh), will always exist. What this datastore system does is, making sure things are not duplicated, even if that means getting into negative items. If someone manages to buy more items than they should have, they will end up with negative money, meaning they will need to regain that money before being able to buy anything again
It could be possible to add a condition function that is evaluated when the data is saved to the datastore, in such a way that if the condition fails, it blocks the “transaction”

And, what about saving before the data even loaded?

(Of course this framework still offers methods to determine when a player’s data loaded)

This makes it virtually impossible for data loss/duping to occur. The only vulnerabilities I foresee are vulnerabilities caused by errors, induced by exploiters (purposefully making the datastore error to achieve something…), in cases where two different datastores are being updated.
For example, an exploiter could have filled one of the keys of his own datastore to exceed the 4mb limit (perhaps the settings or keybinds field), causing his datastore to error constantly, and so when trading with another player, his item count never decreases
(I have implemented a simple per key maximum length check that prevents this exploit)

If someone knows of other vulnerabilities I overlooked, please let me know

Section 2, The framework:

As this module was built for a game I script for, I contains some extras, some unconventional features

This is the overview of the system:
image

BindToClose - workspace:BindToClose() wrapper module, written by me. Used by the Datastore system. Feel free to use it in your own game

DatastoreModule - This is the entrance to the datastore system. From your script (server or client, I will come back to this later), require this module, and then you can access PlayerDatastore or other, like this:

local ForbiddenDatastore = require(game.ReplicatedStorage.DatastoreModule).ForbiddenDatastore
Code
local Datastores = script.Datastores

local DatastoreModule = {}

DatastoreModule.PlayerDatastore = require(Datastores.PlayerDatastore)
DatastoreModule.ForbiddenDatastore = require(Datastores.ForbiddenDatastore)

return DatastoreModule

Runner - This is literally just a script that requires DatastoreModule, it has the RunContext property set to Server, thus why it can run from inside ReplicatedStorage.
It is important that the datastore system is running server side even if not used on the server, because the client needs to receive the data from the server (again, I will come back to this later)

Datastores (PlayerDatastore & ForbiddenDatastore) - These are the datastore modules, which expose methods for external scripts to modify data. PlayerDatastore is a more typical datastore module with Settings, Keybinds, Currency, Inventory, …, for each player, while ForbiddenDatastore is a more cut down version, where every player share the same key, used to do the above tests.
This folder is where you can add new datastrores to your game (such as a leaderboard datastore), and these modules are the ones that are meant to be modified (not that the other ones can’t or shouldn’t be modified)

The Version module script under them is for converting older versions of the data to newer versions. It is currently pretty much empty as there is only 1 version of the data initially


The following modules are used by the Datastores:

DatastoreMain - This is a module that abstracts away the calls to :UpdateAsync() and :GetAsync(). It handles the logic to merge data between the datastore and UpdateObj (UpdateObj is explained a couple lines down) and has automatic retries with exponential backoff. It also exposes methods to get a CacheObj and UpdateObj, and exposes types of the UpdateObj, CacheObj, and the Signal/Event from the custom Signal module

Signal - Custom signal library, written by me. I tried to give the connect functions proper type checking, but the new type system would only give me error types…
Feel free to use in your own work

CacheObj - This is an “object” for chaching the datastore’s data directly on the server and client. It also contains signals for when data changed. This allows for the Datastore system to store the data directly, so calls like :GetCurrenty() or :UpdateCurrency() get or update the cached data rather than the datastore data itself, making them non-yielding functions

/!\ - CacheObj performs deepEquals checks when data is updated, and only fires the data changed events if the deepEquals returned false. This is a performance and complexity consideration I’m not too fond of, so you might want to consider removing it if having the signals fire even if the data stayed the same isn’t a concern
(note that every key is updated when the datastore is saved, and so every update signal would get fired, also triggering replication of the data to the client)

UpdateObj - This is an “object” for scheduling update functions to run against the datastore data when it is saved (or loaded, if the player started playing before the data loaded). You can add UpdateFunctions, and OverwriteUpdateFunctions (it is useful to overwrite data for fields like Keybinds or Settings, where updating the data doesn’t make sense, and data loss is basically inconsequential). It also has an iterator function used by DatastoreMain to apply every update function on the old data

Section 3, PlayerDatastore:

This section will go further into details on how the PlayerDatastore module works

So, through this thread you might have noticed that the DatastoreModule is inside of ReplicatedStorage rather than ServerScriptService, and that the client can require and use it. This is a special thing I’ve done a couple times where I write modules in such a way that they work on both the client and server, even when they shouldn’t…
In the case of PlayerDatastore, the client is allowed to use every :Get() methods, and the :Update() methods for Settings, Keybinds and two others. Mostly everything else is forbidden.
This was achieved by having whitelisted functions for functions usable directly on the client, or functions that, when called by the client, send a RemoteEvent/RemoteFunction to the server, runs the function on the server, and returns the result if it is a RemoteFunction. For the latter, sanity checks must be performed directly in the function. Since the data comes from the client, it has to be validated
It is setup as a whitelist table, rather than blacklist, for the very simple fact that you will inevitably forget to add new functions to the list

Code snippet
-- Functions that can be called from the client
-- This is setup as a whitelist for 1 very simple reason, you WILL forget to add a function to this list
local ClientFunctions = {
	"HasPlayerLoaded",
	"WaitUntilPlayerLoaded",
	"GetRawData",
	"GetValueChangedSignal",
	"GetValueChangedSignalForKey",
	"GetPlaybux",
	"GetWins",
	"GetCurrentGear",
	"GetGears",
	"GetCurrentTitle",
	"GetTitles",
	"GetKeybinds",
}

-- Functions that the client can call, but go through a RemoteFunction that runs the function on the server, and returns the results
local BridgeFunctions = {
	"UpdateCurrentTitle",
	"UpdateCurrentGear",
	"UpdateKeybinds",
	"UpdateSettings",
}

-- // Server side setup
if RunService:IsServer() then

	-- [...]
	-- Here is unrelated code that handles players joining/leaving

	RemoteFunction.OnServerInvoke = function(Player : Player, Index, ...) 
		if not table.find(BridgeFunctions, Index) then error("Player "..Player.UserId.." is trying to access the restricte method "..Index) end -- TODO -- Report user as exploiter
		return PlayerDatastore[Index](PlayerDatastore, Player.UserId, ...)
	end
end

--[[
	Function responsible for overwriting functions inaccessible from the client,
	or overwriting functions that bridge over to the server to call the server sided version of the function.

	Note that this is not some poor attempt at preventing exploiters from calling functions they shouldn't, that's not the goal.
	If a function is overwritten, it's to avoid criptic error messages when that function inevitably errors, and
	give the developer a clear error message as to why it failed
]]
local function OverwriteFunctions(Table : {any})

	for i, v in pairs(Table) do
		if type(v) == "table" then 
			OverwriteFunctions(v)

		elseif type(v) == "function" then

			-- // Core of the function, where the overwriting takes place

			if not table.find(ClientFunctions, i) then
				-- Overwriting non allowed functions with an error function
				-- RemoteFunctionFunctions are overwritten again right below
				PlayerDatastore[i] = function(self, UserId, ...)
					error("Method '"..i.."' cannot be accessed from the client") 
				end
			end

			if table.find(BridgeFunctions, i) then
				-- Overwriting RemoteFunctionFunctions to make them call the server instead
				PlayerDatastore[i] = function(self, UserId, ...)
					return RemoteFunction:InvokeServer(i, ...)
				end
			end

		else 
			-- ¯\_(ツ)_/¯
		end
	end
end

-- // Client side setup
if RunService:IsClient() then
	OverwriteFunctions(PlayerDatastore)

	local UserId = Players.LocalPlayer.UserId

	task.spawn(function()
		-- First event is always the full cache
		-- The roblox engine will return data even if it is connected after data is received, rather than discarding the data
		local InitCache = CacheReplication.OnClientEvent:Wait()
		CacheObjs[UserId]:SetRawCache(InitCache)

		PlayerLoaded[UserId] = true

		Players.PlayerRemoving:Connect(function() 
			PlayerLoaded[UserId] = nil
		end)
		
		CacheReplication.OnClientEvent:Connect(function(UpdatedCache)
			for Key, Value in pairs(UpdatedCache) do
				CacheObjs[UserId]:SetValue(Key, Value)
			end
		end)
	end)
end

When the key updated signal is fired, the system schedules a RemoteEvent to be fired (within the same frame, but not immediately), to replicate the changes to the client. The client is listening to that RemoteEvent, and updates its own CacheObj (a lot of the code is shared between the server and client).
This is how :Get() functions on the client are non-yielding, it’s similar to roblox’s automatic replication.
You might also expect the client to perform an initial request to the server to get the initial data, but this isn’t the case. Since the server can just assume the client will want the initial data, I made the client use RemoteEvent:Wait(), and ensured the server always fires the remote event when the player joins. This works because, if a RemoteEvent receives something, but nothing is connected to it, it will keep it in a queue. There might be a limit to the size of the queue though…

You might have also noticed that, in the videos, there’s a load and a save button. However, saving and loading is automatic for PlayerDatastore.
(The reason why the videos have a load and save button was because I made loading and saving manual for the ForbiddenDatastore, for testing purposes)
The :SaveAsync() and :LoadAsync() functions are used internally, along with QueueSaving():
:LoadAsync() is used when the player first joins
:SaveAsync() is used when the player leaves the game (but could also be used inside of the update currency function, if you want to guarantee quick updates to the datastore)
:QueueSaving() is used by some of the update functions to schedule a call to :SaveAsync(), using a priority system. The higher the priority, the sooner the save. Multiple calls to :QueueSaving() stack up, and lead to a sooner save

Here is an example of how :Get() and :Update() methods are structured:

Code

CacheObjs is a table of CacheObj, so CacheObj[UserId] is a CacheObj
Same goes for UpdateObjs

-- // Wins

PlayerDatastore.Wins = {}

--[[
	Change a user's amount of wins
]]
function PlayerDatastore.Wins:UpdateWins(UserId : number, WinsDiff : number)	

	local UpdateFunction = function(Value) 
		return Value + WinsDiff
	end
	local DefaultValue = 0

	CacheObjs[UserId]:UpdateValue("Wins", UpdateFunction, DefaultValue)
	UpdateObjs[UserId]:AddUpdateFunction("Wins", UpdateFunction, DefaultValue)

	-- Schedule saving in the future. 5 is the priority, higher number -> shorter wait before saving
	-- If other functions call :QueueSaving(), the priority accumulates, and it'll cause an earlier save
	PlayerDatastore:QueueSaving(UserId, 5)
end

--[[
	Get a user's amount of wins
]]
function PlayerDatastore.Wins:GetWins(UserId : number) : number
	return CacheObjs[UserId]:GetValue("Wins", 0)
end


Here, :OverwriteUpdateFunction() is used. ActionName acts as an id, successive calls with the same id will overwrite the previous function. Functions that were added with a different id will not get overwritten.
Here, ActionName denotes the action associated with the keybind

-- // Keybinds

PlayerDatastore.Keybinds = {}

--[[
	Update a user's keybinds
]]
function PlayerDatastore.Keybinds:UpdateKeybinds(UserId : number, ActionName : string, KeycodeNames : {string})

	do -- Sanity check
		if type(ActionName) ~= "string" then error("[UpdateKeybinds "..UserId.."]: ActionName is not a string") end

		local TableLenght = 0

		for _, Value in ipairs(KeycodeNames) do
			if type(Value) ~= "string" then error("[UpdateKeybinds "..UserId.."]: Keycode is not a string") end
			TableLenght += 1
		end

		if #KeycodeNames ~= TableLenght then error("[UpdateKeybinds "..UserId.."]: KeycodeNames is not an array") end
	end

	local UpdateFunction = function(KeybindsTable) 
		KeybindsTable[ActionName] = KeycodeNames
		return KeybindsTable
	end
	local DefaultValue = {}

	CacheObjs[UserId]:UpdateValue("Keybinds", UpdateFunction, DefaultValue)
	UpdateObjs[UserId]:OverwriteUpdateFunction("Keybinds", ActionName, UpdateFunction, DefaultValue)
end

--[[
	Get a user's keybinds
]]
function PlayerDatastore.Keybinds:GetKeybinds(UserId : number) : {[string] : {string}}
	return CacheObjs[UserId]:GetValue("Keybinds", {})
end

This will be it for this section. I could add a lot more to it, but in my very limited free time, this thread has already taken me a significant amount of time to write, and I expect you to take a look at the code yourself. If you have any questions, ask them as a reply or a dm, and I might add the answers and more explanations to the thread

My code also contains unconventional metatable shenanigans, explained here, a very cursed iterator function, a couple of “immediately invoked function expression”, and probably some other questionable stuff…

Section 4, Files:

License
The MIT License (MIT)

Copyright © 2025 https://www.roblox.com/users/79690730/profile

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), 
to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, 
and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Thought it could be a good idea to protect myself from liabilities if my datastore framework fails and causes issues and financial losses to your games
This datastore system has not been tested in production, so there is a (very much) non zero risk of things going wrong

> Uncopylocked testplace
You can use this test place to look at the code and how it works. Every script are either in ReplicatedStorage, ServerScriptService or StarterPlayerScripts

> Creator Store

> rbxm file (19.9 KB)

Reminder that you are expected to modify this framework, and make it your own. I do not plan on updating this for that reason and my lack of time


I will be answering questions here, or in dms, but do not expect quick replies. I am currently a little busy,

My other modules have also been suffering a bit lately, there are a handful of them I plan on updating,
This datastore system was mostly completed so the most time consuming part was setting up everything for this post

I hope the thead isn’t too long. It sure took a while to write…

1 Like