Datastore questions and code review

Hey there !

I made a basic datastore system using a scripts and modules and i would like to know if it is good or not, if not what is wrong and how can i improve it ?

I’m looking to do something as safe as possible to avoid data loss.


Questions

◈ Datastore is a table, and each “object” of this table is a different version of our datastore saves, from the older to the newest/lastest, does the old versions also count to the 4MB limit ?

◈ If then, should i remove/delete the older versions and only keep the lastest one each time the player is leaving and data are saved to prevent exceeding the limit ?

◈ Is it a good practice to have a different datastore for each type (all currencies, inventory items, equiped items, settings ect…) and use one SetAsync and GetAsync for each (which do around 5 or 6 Async call each time a player is joinning/leaving the game) or should i use only one datastore and save a table that include a dictionnary for each type even if it can ecceed the 4MB limit ?

◈ Also, all of these differents datastore have a certain amount of values, like the inventory, there is a value for each items, which can be a BoolValue or a IntValue depending if we permanently unlocked the items of just show how many of them we have, is it fine to save them into a dictionnay even if there is hundred of values ?


Code Review

Scripts placement
Capture

Main script
It is only handling the classic functions when players join and leave the server, and when the server is closing, then calling the correct modules functions.

--[[ Roblox Services ]]--
local PlayerService = game:GetService("Players")
local RunService = game:GetService("RunService")

--[[ Modules Requirement ]]--
local ModuleBackend = require(script:WaitForChild("Backend"))
local ModuleDataTable = require(script:WaitForChild("DataTable"))

-----------------------------------------------------------------------------

--[[ Main Functions ]]--
PlayerService.PlayerAdded:Connect(function(Player)
	ModuleBackend:SetupData(Player, ModuleDataTable) --Create data folders and values
	
	if not RunService:IsStudio() then
		ModuleBackend:LoadData(Player) --Load player data if not in studio
	end
end)

PlayerService.PlayerRemoving:Connect(function(Player)
	--Save player data if not in studio and data was loaded, to avoid losing progress as it would save default values
	if not RunService:IsStudio() and Player:GetAttribute("DataLoaded") == true then
		ModuleBackend:SaveData(Player)
	end
end)

game:BindToClose(function()
	if not RunService:IsStudio() then
		local MainThread = coroutine.running()
		local ThreadsRunning = 0
		
		--Start a new thread to save each player data faster
		local StartSaveThread = coroutine.wrap(function(Player)
			ModuleBackend:SaveData(Player)
			ThreadsRunning -= 1
			
			if ThreadsRunning == 0 then
				coroutine.resume(MainThread)
			end
		end)
		
		--Save all current players data if not in studio and data was loaded
		for _, Player in pairs(PlayerService:GetPlayers()) do
			if Player:GetAttribute("DataLoaded") == true then
				ThreadsRunning += 1
				StartSaveThread(Player)
			end
		end

		if ThreadsRunning > 0 then
			coroutine.yield()
		end
	end
end)

DataTable Module
This module contain all table array and dictionaries corresponding to their related datastore and use case.
It is used to create folders and values when players are joining the game.

local DataTable = {}

--[[ Data Attributes ]]--
DataTable.Attributes = {
	["DataLoaded"] = false
}

--[[ Data Folders ]]--
DataTable.Folders = {"Currency", "Equiped", "Inventory", "Settings"}

-----------------------------------------------------------------------------

--[[ Main Data ]]--
DataTable.Currency = {
	["Coins"] = {"IntValue", 0};
	["Gems"] = {"IntValue", 0};
	["Level"] = {"IntValue", 1};
	["Exp"] = {"IntValue", 0}
}

DataTable.Equiped = {
	["Pet1"] = {"StringValue", "None"};
	["Pet2"] = {"StringValue", "None"};
	["Pet3"] = {"StringValue", "None"};
	["Pet4"] = {"StringValue", "None"}
}

DataTable.Inventory = {
	["Cat"] = {"IntValue", 0};
	["Dog"] = {"IntValue", 0};
	["Bunny"] = {"IntValue", 0};
	["Mouse"] = {"IntValue", 0}
	-- Hundreds of other values
}

DataTable.Settings = {
	["Music"] = {"BoolValue", true};
	["Shadows"] = {"BoolValue", true};
	["SeeOtherPets"] = {"BoolValue", true};
}

return DataTable

Backend Module
This module contain all the core functions of the datastore.
It create all folders and values, load and save players data.

local DatastoreService = game:GetService("DataStoreService")
local Backend = {}

function Backend:SetupData(Player, ModuleDataTable)
	local DataFolder = Instance.new("Folder") --Create the "Datastore" folder in the player
	DataFolder.Parent = Player
	DataFolder.Name = "Datastore"
	
	for Table, _ in pairs(ModuleDataTable)do
		if Table == tostring("Folders") then --Create all other folders into the main "Datastore" folder
			for _, Index in pairs(ModuleDataTable[Table]) do
				local NewFolder = Instance.new("Folder")
				NewFolder.Parent = DataFolder
				NewFolder.Name = Index
			end
		elseif Table == tostring("Attributes") then --Create all attributes into the player object, used for unsaved informations
			for Index, Value in pairs(ModuleDataTable[Table]) do
				Player:SetAttribute(Index, Value)
			end
		else
			for Index, Value in pairs(ModuleDataTable[Table]) do --Create all values into the folders we just created
				local NewValue = Instance.new(Value[1])
				NewValue.Parent = DataFolder:FindFirstChild(Table)
				NewValue.Name = Index
				NewValue.Value = Value[2]
			end
		end
	end
end

function Backend:LoadData(Player)
	local DataFolder = Player:FindFirstChild("Datastore")
	local UserId = Player.UserId
	
	--Load data for each datastore - Folders are the name reference for the datastore name and data dictionaries
	for _, Folder in pairs(DataFolder:GetChildren()) do
		local Datastore = DatastoreService:GetDataStore(tostring(Folder.Name))
		local CurrentData = nil
		
		repeat
			local Success, ErrorMessage  = pcall(function()
				CurrentData = Datastore:GetAsync(UserId)
			end)
			
			if not Success then
				task.wait(15)
			end
		until
		Success == true
		
		--Loop through the data table, and change our current value into ou data folders
		--If player is new, do nothing as default values are already created
		if CurrentData and Folder then
			for Index, Value in pairs(CurrentData) do
				Folder:FindFirstChild(Index).Value = Value
			end
		end
		
		--Player left the game, stop the data loading
		if not Player or not Folder then
			break
		end
	end
	
	--If the player is still there and didn't left the game,
	--set DataLoaded to true so other scripts in the game take it as reference to load things such as Gui ect..
	if Player then
		Player:SetAttribute("DataLoaded", true)
	end
end

function Backend:SaveData(Player)
	local DataFolder = Player:FindFirstChild("Datastore")
	local UserId = Player.UserId
	local DataTable = {} --All dictionaries and their values in this table
	
	--Create dictionaries in the DataTable, so all values are safe and stored if player object is removed
	for _, Folder in pairs(DataFolder:GetChildren()) do
		DataTable[Folder.Name] = {}	
		
		for _, Value in pairs(Folder:GetChildren()) do
			DataTable[Folder.Name][Value.Name] = Value.Value
		end
	end
	
	--Save dictionaries in the DataTable one by one
	for Table, _ in pairs(DataTable) do
		local Datastore = DatastoreService:GetDataStore(tostring(Table))
		
		repeat
			local Success, ErrorMessage  = pcall(function()
				Datastore:SetAsync(UserId, DataTable[Table])
			end)
			
			if not Success then
				task.wait(1)
			end
		until
		Success == true
	end
end

return Backend
2 Likes

To answer your questions:

  • Replacing keys in a datastore with new values does not increase memory consumption as you are simply changing the values in an already existing table.

  • Use tables to store player data. Every experience has a 4MB DataStore size, not every DataStore. You can easily reference values saved in the player data table by doing plrTable[valueName].

  • Having 2 DataStores to save data if you think your table is too big wouldn’t hurt, but it is recommended to not go above 2 DataStores.

Thank you answering my questions.
After doing a lot of research about datastore in general, i’ve came into a conclusion.


The 4MB limit in only for the key value we’re storing when using SetAsync and UpdateAsync.

Datastore:SetAsync(UserId, DataTable) --(Key, Value(4MB limit))

Then, after doing some calculs, 4MB is a way enough to store thousands of values into one table of multiple dictionnaries, and save the table into one datastore.

So, i should use only one datastore to save all data related to players instead of multiple datastore, one for each player data type, then datastores should be used and organized depending of their use cases, like a datastore for leaderboards, an other for the players ect… which is reducing a lot the use of SetAsync, GetAsync, UpdateAsync and other datastore functions.

About security, the use of SetAsync isn’t safe to save a table of multiple dictionaries like i do, and can result into data inconsistency if two servers attempt to set the same key at the same time unless i implement a session locking into it, so i mostly should use UpdateAsync instead which allow me to implement this session locking easier.


I’m still giving you the solution as you’re the only one that responded to the post. ^^

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.