Datastore best practices

Update

This post references outdated code. Please check out the code here:

References

Note
I am well aware of the basic functions of datastore. This post is for creating my own datastore module using a system I’ve thought of. What I am asking is what are the best practices.

Here is how my current system works

  1. JOIN - create a table for them
  2. EDIT - edit their created table
  3. SAVE - save their data when the leave

Current code

-- Get the data
local DatastoreService = game:GetService("DataStoreService")
local PlayerData = DatastoreService:GetDataStore("PlayerData")

-- Create a server storage 
local serverPlayersStorage = {}

-- add players
game.Players.PlayerAdded:Connect(function(plr)
	local data = PlayerData:GetAsync(plr.UserId)
	if not data then
		data = {
			["plr"] = plr.UserId,
			["coins"] = 0
		}
	end
	
	table.insert(serverPlayersStorage, data)
	
end)

-- find data
local function findData(plr)
	for i,data in pairs(serverPlayersStorage) do
		if data.plr == plr.UserId then
			return data
		end
	end
end

-- edit data
local function editData(plr, valueName, value)
	findData(plr)[valueName] = value
end

-- Save data 
game.Players.PlayerRemoving:Connect(function(plr)
	PlayerData:UpdateAsync(plr.UserId, findData(plr))
end)

Anyways so what are the best practices? What am I missing, and more specifically how would I implement these missing things. Code examples help.

Thank you for your time!

Maybe json encoding or table packing it when the player leaves to save space and fit more in?

If I were you, I would make it so instead of having multiple modules (I am assuming you will make it a module script, if not you should). You should add a function to add another data table like for example it could be a function called “Add Data”.

EX Function:

module.addData function(name)
    serverPlayersStorage[tostring(name)] = {}
end

The module works by determining the data based on a folder located inside the module script. The folder will have attributes that determine both the values and their default values that the module will use. I am not adding a add function to the module script because I want the data to be consistent among all players.

sounds great to me, it is your system

okay so I’ve added caching and Json encoding/decoding and retries:


-- Get the data
local DatastoreService = game:GetService("DataStoreService")
local PlayerData = DatastoreService:GetDataStore("PlayerData")
local HttpsService = game:GetService("HttpService")

local serverPlayersData = {}
local cache = {}
local maxRetries = 5
local retryDelay = .5



local function findData(plr)
	for i,data in pairs(serverPlayersData) do
		if data.plr == plr.UserId then
			return data, i
		end
	end
end

local function findCache(plr)
	for i,cache in pairs(cache) do
		if cache.plr == plr.UserId then
			return cache
		end
	end
end

-- add players
game.Players.PlayerAdded:Connect(function(plr)

	local data = findCache(plr)
	local shouldDecode = true

	if not data then

		local tries = 0
		local success = false
		
		repeat 
		success = pcall(function()
			tries = tries + 1
			task.wait(retryDelay)
			data = PlayerData:GetAsync(plr.UserId)
		end)
		until success or tries >= maxRetries

		if not data then
			shouldDecode = false
			data = {
				["plr"] = plr.UserId,
				["coins"] = 0
			}
		end
	end
	
	
	if shouldDecode then
		data = HttpsService:JSONDecode(data)
	end
	
	table.insert(serverPlayersData, data)
	
end)


-- edit data
local function editData(plr, valueName, value)
	findData(plr)[valueName] = value
end

-- Save data 

local function saveData(plr)
	local tries = 0
	local success = false
	local data,index = findData(plr)
	local encodedData = HttpsService:JSONEncode(data)

	repeat 
		tries = tries + 1
		success = pcall(function()
			task.wait(retryDelay)
			PlayerData:UpdateAsync(plr.UserId, encodedData)

		end)
	until success or tries >= maxRetries

	if findCache(plr) == nil then
		table.insert(cache, encodedData)
	end
	table.remove(serverPlayersData, index)
end

game.Players.PlayerRemoving:Connect(function(plr)

	if #game.Players:GetPlayers() == 0 then
		game:BindToClose(function()
			saveData(plr)
		end)
	else
		saveData(plr)
	end

end)

A lot of top developers use ProfileService. It handles all of the possible edge cases for you, and once its setup, its as easy to use as anything else. I use it for every project now. Definitely check it out.

I am pretty sure my code is the same as profile service but easier for myself to understand

I’ve honestly have never had an issue with any sort of data saving speed, data loss, etc. I just do something like this for getting the values:

local DATA = Data:GetAsync(player.UserId.."-Leaderstats")
if DATA == nil then
	cash.Value = 0
else
	cash.Value = DATA[1]
end

And for saving I just do:

local cash = player.leaderstats.Money.Value

local DataTable = {cash}

Data:SetAsync(player.UserId.."-Leaderstats",DataTable)

I’ve never ran into an issue, I have no clue why people do all these fancy stuff, all does the same thing for me.

The big differences are that ProfileService has session locking (data will not be written until other servers have successfully saved the data), data reconciliation (where if you add new data to the data template, existing players will have that added to their data as well), service failure handling (if the datastores are down, it will attempt to manage it for you), etc.

There are a lot of great features most people wouldnt bother to add. It makes it super super unlikely to lose any data, which your players will thank you for.

:arrow_down: 1. Session locking wastes resources because the only way I can think of this is like using messagingService. Roblox instantly saves your data and I check if the data is outdated before saving anyways so if they notice that their data was lost and leave the game then rejoin it restores their data with mys ystem.

:heavy_minus_sign:2. Data reconciliation: I already have this in my code where it checks if it will override existing data using Tick()

:heavy_minus_sign:3. My script already manages this.

1 Like

If you’re looking for an alternative to ProfileService, you could also try out Suphi’s DataStore module:

This datastore implementation helps handle some of the spin-locking nature from ProfileService’s session locking, which is something you may have issues with. There’s more details about the comparison of these modules in this video:

Profile service’s session locking is at no additional resource cost, and profileservice generally has low datastore overhead.

Your script does not session lock nor does it have the useful utilities that profile service offers

The best practice is to use a DataStore module like ProfileService or Suphi’s. ProfileService is battle-tested and has been used by a number of very popular games. It also addresses all of my concerns below.

Your script doesn’t have autosaving, so if the server crashes, you’ll lose all of your data.

Your script also attempts to fetch the player’s data 5 times, but if that fails, it will just set your data to the default template. When the player leaves the game, if the UpdateAsync call works, they’ve just lost their data.

Session locking is another big issue your script doesn’t address. This is prone to servers fighting over your player data. Which opens you up to dupe vulnerabilities.

There’s also no way for another script to call your edit/save functions since you don’t use a ModuleScript or expose any API to do that.

Thank you for the feedback. Can you tell me how profile service did this and what “useful utilities” it has?

  • My script manages server crashes through its bindable event.
  • My script verifies the data before saving it with updateasync
  • How do I implement session locking? I can only think of using messaging service, which costs resources

when you use profileservice paired with replicaservice, it makes everything easier to handle

To clarify, I’m not interested in using 3rd party datastote modules on a post I’m trying to promote my own 3rd party datastore module.

How would i do session locking without uisng a service like messagingservice

(Which is part of the ROBLOX api, not third party like it may sound)

2 major issues with your module:

  1. The function connected to PlayerRemoving function won’t work properly.
game.Players.PlayerRemoving:Connect(function(plr)

	if #game.Players:GetPlayers() == 0 then
		game:BindToClose(function()
			saveData(plr)
		end)
	else
		saveData(plr)
	end

end)

According to documentation:
Fires when a player is about to leave the game

Hence, GetPlayers() will not return an empty table.

Moreover, creating a connection to BindToClose under such weird condition is incorrect.
BindToClose will trigger right after the sever is requested to be shut down.
It means, that when you decide to shutdown some server, the BindToClose will not trigger because it’s not yet initialized.
Move it out of PlayerRemoving and loop through the players saving their data.
Then, add a condition in PlayerRemoving to prevent double save.

  1. You’re not checking whether the data has been loaded before saving them.
    This is the most frequent cause of data loses so you should definitely make sure you’re handling it properly.

Generally, there’s more to improve, for example indexing directly the result of findData(). What if the data was not found?
Then, your script errors.

If you’re not experienced with creating data store systems, it’s recommended to use a battle-tested module which handles the difficulties for you.
I personally never used one but I had to learn from many mistakes. :slightly_smiling_face:

To adress your concerns:

  • if it still errors after 5 attempts, i will kick them and tell the player the game is experiencing issues

  • to adress crashes, i will add a module.save function for the developers. The developers should call this function during major events like beating a level or buying currency.

——

  • for session locking, i will add a lock boolean. Im still thinking about how i would do this without costing resources.