First time working with DataStoreService! Please tell me how I can improve my code!

zlobny_bylbylator · September 30, 2024, 7:29pm

Hello everyone!
As said in title, this is my first time working with data stores
I would want you to check out my code and suggest me something to improve it.
The main point of how script is supposed to work is to save players data of points, survivals, kills but also some “roles” (which are not fully implemented yet) and player’s purchases in shop that I will surely add in future!

local DataStoreService = game:GetService("DataStoreService")
local PlayersData = DataStoreService:GetDataStore("PlayersData_TEST")

local Players = game:GetService("Players")

local function CreateLeaderstatsFolder(player: Player)
	
	local leaderstats = Instance.new("Folder")
	leaderstats.Name = "leaderstats" -- The name of the folder must be leaderstats oherwise it will not work
	leaderstats.Parent = player
	
	local points = Instance.new("IntValue")
	points.Name = "Points"
	points.Value = 100
	points.Parent = leaderstats
	
	local survivals = Instance.new("IntValue")
	survivals.Name = "Survivals"
	survivals.Value = 0
	survivals.Parent = leaderstats
	
	local kills = Instance.new("IntValue")
	kills.Name = "Kills"
	kills.Value = 0
	kills.Parent = leaderstats
	
end

local function LoadStats(player: Player, data: DataStore)
	local playerLds = player:FindFirstChild("leaderstats")
	playerLds.Points.Value = data.points
	playerLds.Survivals.Value = data.survivals
	playerLds.Kills.Value = data.kills
end

local function deepCopy(original)
	local copy = {}
	for k, v in pairs(original) do
		if type(v) == "table" then
			v = deepCopy(v)
		end
		copy[k] = v
	end
	return copy
end


local dataTemplate = {
	points = 100,
	survivals = 0,
	kills = 0,

	isBanned = false,
	isAdmin = false,
	isOG = false,
	isContentCreator = false
}

Players.PlayerAdded:Connect(function(player)
	
	CreateLeaderstatsFolder(player)
	
	local success, errorMsg = pcall(function()
		local data = PlayersData:GetAsync(player.UserId)
		
		if data == nil then
			data = deepCopy(dataTemplate)
			PlayersData:SetAsync(player.UserId, dataTemplate)
		end
		
		LoadStats(player, data)
	end)
	
	if success then warn(player.Name.."'s data is successfully loaded!") else
		player:Kick("Failed to load player data!")
		error(player.Name.."'s DataLoadError: "..errorMsg)
	end
	
end)

Players.PlayerRemoving:Connect(function(player)
	
	local ldstats = player:FindFirstChild("leaderstats")
	
	local dataTable = {
		points = ldstats.Points.Value,
		survivals =  ldstats.Survivals.Value,
		kills =  ldstats.Kills.Value,

		isBanned = false,
		isAdmin = false,
		isOG = false,
		isContentCreator = false
	}
	
	local success, errorMsg = pcall(function()
		PlayersData:SetAsync(player.UserId, dataTable)
	end)
	
	if success then warn(player.Name.."'s data is successfully saved!") else error(player.Name.."'s DataSaveError: "..errorMsg) end
	
end)

12345koip · September 30, 2024, 7:36pm

It looks a lot better than other beginner data store code I’ve seen!

Here’s a few tips on how you could make it more secure and prevent data loss further:

Use retry logic when saving data
Add an autosave
Add things like session locking and data versioning so that servers don’t save outdated data (links into my next point)
Utilise UpdateAsync, it’s features and it’s safety features. This can significantly help you prevent data loss.
Keep a track of data budgets and handle your data store requests accordingly
Add a function in game:BindToClose() to yield so that data requests have time to process when the server shuts down
Add some kind of failsafe and check it on Players:PlayerRemoving() to make sure that the player’s data isn’t lost if their data fails to load
Add your own queue for data requests as to not overload the data store’s queue and prevent requests from “falling off of the end”

Adding these should help you make your system a lot more secure. Let me know if you have questions about any of these points!

zlobny_bylbylator · September 30, 2024, 7:40pm

Sorry, I didn’t really get a point of what you said
Coul’d you tell it in another words, since I am not so advanced yet. I’ve got only 1 month experience in roblox studio.
Thanks!

12345koip · September 30, 2024, 8:02pm

Sure, sorry about that!

Retry Logic

This is basically where you try multiple times to save data. It means if it fails once, it’ll try again in the hopes of saving it.

--initialising values
local success, result
local attempt  = 0

repeat
    success, result = pcall(--[[save data]])
    attempt += 1 --increment attempt
until
    success or attempt == 3 --try a maximum of 3 times before giving up

Autosave

Basically does what it says on the tin. Saves player’s data at regular intervals. This means that if for whatever reason data isn’t saved when they exit the game, they have a more up-to-date version of data in the data store.

Session locking

This is basically where you add a piece of data to the entry in the data store. It essentially acts as a lock, preventing another server from overwriting the player’s data. This means if a request is particularly slow on another server and the player has more up-to-date data on the server they are currently playing on, the other server won’t update it.

Typically, you would add:

The current server’s game.JobId (A unique identifier for the server)
A timestamp indicating when the session was locked

This lock would be removed when the player leaves, indicating that the next server can “claim” the lock and the player isn’t currently in the game session. On the chance that data fails to save (meaning the lock isn’t removed), add a timeout for the lock that the next server checks when updating the store.

Don’t bother using session locking for get requests, only when changing the data.

UpdateAsync

UpdateAsync reads data before it writes, which basically allows you to compare data from the previous entry. This is extremely important because it makes things like session locking and data versioning work!

local success, result = pcall(DataStore.UpdateAsync, DataStore, saveKey, function(old)
    --"old" will be the previous entry in the store.
    --this means you can compare entries as necessary.

    return data --you must then return the new entry you want to be saved to the store.
end)

Data Budgeting

Each server has a data request budget for request types. You have the request types get, set, update, increment, remove, etc. When the server runs out of a budget for that request type, the requests are now subject to a thing called throttling. This basically means requests will take much longer to go through and are just added to a queue to be later processed, which significantly slows down the data saving process. It’s also one of the reasons session locking and data versioning are so important - so outdated data doesn’t overwrite fresher data.

If you manage your server’s request budget, you can prevent requests from ever reaching this queue. This is great because it means your player’s data is more likely to save more quickly.

To get a request type’s budget:

local ds = game:GetService("DataStoreService")

--example for UpdateAsync
local budget = ds:GetRequestBudgetForRequestType(Enum.DataStoreRequestType.UpdateAsync)

game:BindToClose() yielding

When the game server closes down, just before it closes, it’ll run any functions bound with game:BindToClose(). And, if your game server closes without data requests being processed, those data requests are lost, which means that player’s data doesn’t get saved. The game will wait a maximum of 30 seconds before closing, which means we can force it to wait so that our data requests can finish processing before it closes:

local runService = game:GetService("RunService")

game:BindToClose(function()
    if runService:IsStudio() then --wait 5 seconds in studio because there's only one request that needs to go through - yours!
        task.wait(5)
    else --wait the full 30 seconds in Roblox servers - we don't know how many requests there are
        task.wait(30)
    end
end)

Failsafe

Again, does what it says on the tin. You know that first return result of pcall, indicating whether or not the function executed successfully? Well, you can use this on your data loading pcall to check whether or not the player’s data loaded. If it didn’t, well, PlayerRemoving would fire, and your function would save their blank data, which is well, not good. You somehow need to indicate if loading failed to the PlayerRemoving function, whether it be an attribute or an instance value. You can check this in PlayerRemoving and avoid saving data if it failed to load.

Queueing saving requests

You know that queue I mentioned earlier in data request budgeting? Well, it’s also an issue if you spam the store with requests, which can happen when multiple players leave at the same time… especially if the server just shut down. The warning states:

Data store request was added to queue. If queue fills, further requests will be dropped.

Obviously, we don’t want requests to be dropped. Ideally, they should all go through. So, if we queue them through our own table and then pass them individually to the data store without filling that queue, we can effectively avoid requests “falling off” the end of the queue.

Hope this helps. Again, please let me know if you have any questions!

zlobny_bylbylator · September 30, 2024, 8:43pm

After a bit of editing according to your advices here’s what I’ve got:

local DataStoreService = game:GetService("DataStoreService")
local Players = game:GetService("Players")
local RunService = game:GetService("RunService")

local PlayersData = DataStoreService:GetDataStore("PlayersData_TEST")

local function CreateLeaderstatsFolder(player: Player)
	
	local leaderstats = Instance.new("Folder")
	leaderstats.Name = "leaderstats" -- The name of the folder must be leaderstats oherwise it will not work
	leaderstats.Parent = player
	
	local points = Instance.new("IntValue")
	points.Name = "Points"
	points.Value = 100
	points.Parent = leaderstats
	
	local survivals = Instance.new("IntValue")
	survivals.Name = "Survivals"
	survivals.Value = 0
	survivals.Parent = leaderstats
	
	local kills = Instance.new("IntValue")
	kills.Name = "Kills"
	kills.Value = 0
	kills.Parent = leaderstats
	
end

local function LoadStats(player: Player, data: DataStore)
	local playerLds = player:FindFirstChild("leaderstats")
	playerLds.Points.Value = data.points
	playerLds.Survivals.Value = data.survivals
	playerLds.Kills.Value = data.kills
end

local function deepCopy(original)
	local copy = {}
	for k, v in pairs(original) do
		if type(v) == "table" then
			v = deepCopy(v)
		end
		copy[k] = v
	end
	return copy
end


local dataTemplate = {
	points = 100,
	survivals = 0,
	kills = 0,

	isBanned = false,
	isAdmin = false,
	isOG = false,
	isContentCreator = false
}

Players.PlayerAdded:Connect(function(player) -- load player data
	
	CreateLeaderstatsFolder(player)
	
	local success, errorMsg
	local attempt = 0

	repeat
		success, errorMsg = pcall(function()
			local data = PlayersData:GetAsync(player.UserId)

			if data == nil then
				data = deepCopy(dataTemplate)
				PlayersData:SetAsync(player.UserId, dataTemplate)
			end

			LoadStats(player, data)
		end)
		attempt += 1 --increment attempt
	until
	success or attempt == 3 --try a maximum of 3 times before giving up
	
	--[[local success, errorMsg = pcall(function()
		local data = PlayersData:GetAsync(player.UserId)
		
		if data == nil then
			data = deepCopy(dataTemplate)
			PlayersData:SetAsync(player.UserId, dataTemplate)
		end
		
		LoadStats(player, data)
	end)]]
	
	if success then warn(player.Name.."'s data is successfully loaded!") else
		local loadFailure = Instance.new("BoolValue")
		loadFailure.Name = "LoadFailure"
		loadFailure.Parent = Players
		player:Kick("Failed to load player data!")
		error(player.Name.."'s DataLoadError: "..errorMsg)
	end
	
end)

Players.PlayerRemoving:Connect(function(player)
	
	local ldstats = player:FindFirstChild("leaderstats")
	
	local dataTable = {
		points = ldstats.Points.Value,
		survivals =  ldstats.Survivals.Value,
		kills =  ldstats.Kills.Value,

		isBanned = false,
		isAdmin = false,
		isOG = false,
		isContentCreator = false
	}
	
	local success, errorMsg
	local attempt  = 0

	repeat
		success, errorMsg = pcall(function()
			if not player:FindFirstChild("LoadFailure") then
				PlayersData:SetAsync(player.UserId, dataTable)
			else
				return false
			end
		end)
		attempt += 1 --increment attempt
	until
	success or attempt == 3 --try a maximum of 3 times before giving up
	
	--[[local success, errorMsg = pcall(function()
		if not player:FindFirstChild("LoadFailure") then
			PlayersData:SetAsync(player.UserId, dataTable)
		else
			return false
		end
	end)]]
	
	if success then warn(player.Name.."'s data is successfully saved!") else error(player.Name.."'s DataSaveError: "..errorMsg) end
	
end)

game:BindToClose(function()
	if RunService:IsStudio() then --wait 5 seconds in studio because there's only one request that needs to go through - yours!
		task.wait(5)
	else --wait the full 30 seconds in Roblox servers - we don't know how many requests there are
		task.wait(30)
	end
end)

Seems to work just fine! Though I didn’t really understand how I can use data budgetting, queing and UpdateAsync…

12345koip · September 30, 2024, 9:56pm

Looks better. Don’t bother with retry logic for GetAsync though. I suggest you add an autosave, and all you can really do to preserve data requests is to adjust the wait time between autosaves depending on the remaining budget.

As for queueing, have a table to contain all of the requests for the store. Then, use a repeat-until loop until the data request is at the front of the queue.

table.insert(queue, data)

repeat
    task.wait(1)
until
    table.find(queue, data) == 1

table.remove(queue 1)

For UpdateAsync, the main use for it in data systems is to allow things like session locking and
versioning to work. It also provides a couple of security features, but you’ll need to utilise it properly to fully see the improved effect.

Here’s the documentation: GlobalDataStore | Documentation - Roblox Creator Hub

(Don’t confuse data versioning with key versioning. Key versioning, mentioned in the documentation, allows you to retrieve older data versions where as data versioning is incrementing a value within data, that you created yourself, each save so servers can make sure they don’t overwrite with outdated data.)

Sorry this reply isn’t that detailed, I’m going to sleep now so I can help more in the morning.