Constant random data loss please help

Hello, I have this script that should automatically creates leaderstats and adds +1 time played to the leaderstat every minute and also handles saving points that can be earned by completing tasks in-game and saves whenever the player leaves however there seems to be data loss since some players after rejoining have there playtime and points reset to 0 and there’s no errors or warnings players just seem to randomly have their data reset and I seem to be the only one who is not affected since I have not lost any data but everyone who joined lost their data I am completely lost

local datastores = game:GetService("DataStoreService")
local datastore = datastores:GetDataStore("PlayerData")
local players = game:GetService("Players")

local increaseTimeEvery = 60 -- increase play time after this many seconds

-- Deserialize player data and apply it to leaderstats
local function deserializeData(player, data)
	local leaderstats = player.leaderstats
	for statName, statValue in pairs(data) do
		local stat = leaderstats:FindFirstChild(statName)
		if stat then
			stat.Value = statValue
		end
	end
end

-- Serialize player data from leaderstats for saving
local function serializeData(player)
	local data = {}
	local leaderstats = player.leaderstats
	for _, stat in ipairs(leaderstats:GetChildren()) do
		data[stat.Name] = stat.Value
	end
	return data
end

-- Update player data every minute to track time played
local function trackTimePlayed(player)
	while task.wait(increaseTimeEvery) do -- Wait for 60 seconds
		if player and player.Parent then
			local timePlayed = player.leaderstats:FindFirstChild("TimePlayed")
			if timePlayed then
				timePlayed.Value += 1 -- Increment time played by 1 minute
			end
		else
			break -- Stop if the player leaves
		end
	end
end

-- Handle player joining the game
local function onPlayerAdded(player)
	-- Create leaderstats folder
	local leaderstats = Instance.new("Folder")
	leaderstats.Name = "leaderstats"
	leaderstats.Parent = player

	-- Add time played stat
	local timePlayedStat = Instance.new("IntValue")
	timePlayedStat.Name = "TimePlayed"
	timePlayedStat.Parent = leaderstats

	-- Add points stat
	local pointsStat = Instance.new("IntValue")
	pointsStat.Name = "Points"
	pointsStat.Parent = leaderstats

	-- Load data from DataStore
	local success, result = pcall(function()
		return datastore:GetAsync(player.UserId)
	end)

	if success and result then
		deserializeData(player, result)
	else
		if not success then
			warn("Data error: "..result)
		end
	end

	-- Start tracking time played
	task.spawn(function()
		trackTimePlayed(player)
	end)
end

-- Handle player leaving the game and save their data
local function onPlayerRemoving(player)
	local data = serializeData(player)
	local success, result = pcall(function()
		return datastore:SetAsync(player.UserId, data)
	end)

	if not success then
		warn("Data error: "..result)
	else
		print("Saved data for", player.Name)
	end
end

-- Save data for all players during server shutdown
local function onServerShutdown()
	for _, player in ipairs(players:GetPlayers()) do
		local data = serializeData(player)
		local success, result = pcall(function()
			return datastore:SetAsync(player.UserId, data)
		end)

		if not success then
			warn("Data error: "..result)
		else
			print("Saved data for", player.Name, "on shutdown")
		end
	end
end

-- Connect functions to player events
players.PlayerAdded:Connect(onPlayerAdded)
players.PlayerRemoving:Connect(onPlayerRemoving)
game:BindToClose(onServerShutdown)

Don’t use SetAsync, use UpdateAsync to limit data loss, also you should store data in replicated storage, not under player, and last thing is to don’t fire OnPlayerRemoving when server is shutting down

The main issue is a common one that will result in data loss until you build logic in to protect returning players.

At the moment if a returning player with existing data enters your game and the pcall is unsuccessful then you treat them as a new player.

When they exit the game you overwrite their existing data with just the data they accumulated in that one session (data loss). Set asyc or update asyc doesn’t really matter if you’re saving things like booleans or tables of data so it’s best to correct the root cause reason for this flaw in logic.

Some steps you can take to reduce the chance of accidentally treating a player with historic data as a new player and overwriting their data. Network calls to pull data from the datastore do fail.

Suggestions:

  • set up a control value like # of visits that you save with the player in the datastore
  • your initial pull of player data when a player enters your game if its unsuccessful wait and try it again before treating them as if they have no data. This right away may reduce your current data loss by 50%
  • when your player is leaving the game before saving their new data to the datastore if their number of visits is 1 (new player) try to pull their existing data again. Compare # of visits you just pulled with what you will be saving for the leaving player. If it passes the test then save the data to the datastore. If not don’t.

Basically for you to accidently treating a returning player as a new player becomes very very unlikely.

For a fail to occur all the below would have to happen.

  • player added pcall data pull would need to fail
  • player added second attempt pcall data pull would need to fail
  • player removing pcall data pull would need to fail.

Only after all that would you have a data loss to an returning player.

Note
I wouldn’t run a loop on server to track time played. If you need to display it visually just let the player track it locally and run the loop there. For data security just record start time on server side when player entered and calculate the time played when they leave on server.
Hope this helps.

From looking at your code, I noticed a few other things that haven’t been mentioned yet.

  1. You are saving on BindToClose, but giving the requests no time to process. SetAsync, along with other data store operations, only schedules the request; you need to wait at the end of the BindToClose function to give requests time to process.
  2. On that note, don’t save on BindToClose. Only yield. Since Players.PlayerRemoving will fire for every player that got kicked, you’re just adding extra requests.
  3. Try to manage the data store request budget - this is basically how many requests you can make before throttling happens, which is basically where requests take much longer to process.
  4. Advanced features like session locking andd data versioning cam help protect player data. They boht stop other servers from overwriting data which could be outdated (this is where UpdateAsync really becomes helpful).
  5. Retry saving up to, say, 5 times if previous attempts fail.

If you need more info on these, just ask.

thanks you for the tips I have updated the code should something like this work?

I moved the logic that creates the leaderstats to a separate script

here’s the new script that loads/saves

local DataStoreService = game:GetService("DataStoreService")
local datastore = DataStoreService:GetDataStore("PlayerData")
local players = game:GetService("Players")
local maxRetries = 5 -- Max retries for saving data

print("Data Save and Load Script Active")

-- Deserialize player data and apply it to the leaderstats
local function deserializeData(player, data)
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for statName, statValue in pairs(data) do
			local stat = leaderstats:FindFirstChild(statName)
			if stat then
				stat.Value = statValue
			end
		end
	end
end

-- Serialize player data from leaderstats for saving
local function serializeData(player)
	local data = {}
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for _, stat in ipairs(leaderstats:GetChildren()) do
			data[stat.Name] = stat.Value
		end
	end
	return data
end

-- Load data for the player when they join
local function onPlayerAdded(player)
	local success, result = pcall(function()
		return datastore:GetAsync(player.UserId)
	end)

	if success and result then
		deserializeData(player, result)
	elseif not success then
		warn("Data error loading for " .. player.Name .. ": " .. result)
	end
end

-- Retry saving if failure
local function saveDataWithRetries(player, data, retries)
	retries = retries or 0

	local success, result = pcall(function()
		return datastore:UpdateAsync(player.UserId, function(oldData)
			-- If oldData exists, merge with new data to avoid overwriting
			if oldData then
				for key, value in pairs(data) do
					oldData[key] = value
				end
				return oldData
			else
				return data
			end
		end)
	end)

	if not success then
		if retries < maxRetries then
			warn("Retrying data save for " .. player.Name .. " (Attempt " .. (retries + 1) .. ")")
			saveDataWithRetries(player, data, retries + 1)
		else
			warn("Failed to save data for " .. player.Name .. " after " .. maxRetries .. " attempts: " .. result)
		end
	else
		print("Saved data for", player.Name)
	end
end

-- save data of when they leave
local function onPlayerRemoving(player)
	local data = serializeData(player)
	saveDataWithRetries(player, data)
end

-- wait a bit
local function onServerShutdown()
	task.wait(2)
end

-- Connect functions 
players.PlayerAdded:Connect(onPlayerAdded)
players.PlayerRemoving:Connect(onPlayerRemoving)
game:BindToClose(onServerShutdown)

and here’s the script that creates the leaderstats incase that’s needed

local players = game:GetService("Players")
local increaseTimeEvery = 60 -- increase play time after this many seconds

print("Leaderstats Creation and Time Tracking Script Active")

-- Create leaderstats and initialize stats for player
local function setupLeaderstats(player)
	-- Create leaderstats folder
	local leaderstats = Instance.new("Folder")
	leaderstats.Name = "leaderstats"
	leaderstats.Parent = player

	-- Add time played stat
	local timePlayedStat = Instance.new("IntValue")
	timePlayedStat.Name = "TimePlayed"
	timePlayedStat.Parent = leaderstats

	-- Add points stat
	local pointsStat = Instance.new("IntValue")
	pointsStat.Name = "Points"
	pointsStat.Parent = leaderstats
end

-- Update player data every minute to track time played
local function trackTimePlayed(player)
	while task.wait(increaseTimeEvery) do
		if player and player.Parent then
			local timePlayed = player.leaderstats:FindFirstChild("TimePlayed")
			if timePlayed then
				timePlayed.Value += 1 -- Increment time played by 1 minute
			end
		else
			break -- Stop if the player leaves
		end
	end
end

-- Handle player joining the game
local function onPlayerAdded(player)
	setupLeaderstats(player)

	-- Start tracking time played
	task.spawn(function()
		trackTimePlayed(player)
	end)
end

-- Connect the function to the PlayerAdded event
players.PlayerAdded:Connect(onPlayerAdded)

thanks in advance!

Hey, Slate!

I’ve analyzed your code and found out you have a few parts where you could improve.

Race Condition

A race condition can arise in software when a computer program has multiple code paths that are executing at the same time. If the multiple code paths take a different amount of time than expected, they can finish in a different order than expected, which can cause software bugs due to unanticipated behavior.

Source: Wikipedia

In this code, when the Player joins and leaves without their data loading in, you are saving a completely empty state. This causes the player to have 1) Corrupted data or 2) Total data loss.


(a flowchart I made, but of course datastores take longer than a few milliseconds)

Add a state to determine if the data has loaded in.
Only save player data if it has already loaded in.

p.s. seems you’ve already implemented a retry function for the errors on saving player data
But you should also do that for when you are loading data in.

In this function you rightfully check both conditions of the datastore call with if success and result then. However if success is nil you leave the condition unhandled, i.e. you then allow the player to continue in the game with zero data and still setup auto-saving their data and saving it in OnPlayerRemoved. You have to handle the fail conditions, i.e. setup a temporary leaderstat or outright kick the player, then you don’t save empty data on exit.

That looks better, but it could still use a lot of work.

  1. your use of UpdateAsync isn’t really that helpful without session locking and data versioning which also help stop data loss. When you call the save function, data is queued to be stored, it isn’t actually stored when UpdateAsync returns, hence Async (asynchronous saving). This means other servers, or even the current server, may end up overwriting outdated data. These fratures can prevent that.
  2. even if you kick the player on data saving fail, you need some kind of way to check if saving worked. I’d suggest an attribute.
  3. combine leaderstats and data store if possible, you could use a script-module structure.
  4. Wait a bit longer on BindToClose. I’d suggest waiting 5 seconds in Studio and 30 seconds in-game (30 seconds is the limit).
  5. Your retry logic could be simplified:
local success, result
local attempt = 0

repeat
    success, result = pcall(DataStore.UpdateAsync, DataStore, key, function(old)
        --...
    end)
    attempt += 1
until
    success or attempt == 3
  1. Keep a track of the data request budget, which indicates how many requests are left before throttling.

Again, if you need any more info on these, just ask.

How would i go by implementing session locking and data versioning

so, for data versioning, you just want to have an integer in a field of the data table that you update each save. Then, you check that on each save and return if it’s less than the current one. Here’s a basic example:

local data = --[[get save data, add a "Version" field. Set it to 0 if the player is new to the game]]
data.Version += 1 --increment the version

local success, result = pcall(DataStore.UpdateAsync, DataStore, key, function(old)
    --returning nil cancels the save when using UpdateAsync
    if (old and old.DataVersion <= data.DataVersion) then return nil end

    return data --save new data
end)




Session locking is a bit more difficult. Here’s a basic rundown:

  • When the player joins, the data store is updated with the current server’s ID and time of locking. This is the session lock itself.
  • When the player leaves, the session lock is removed along with data saving.
  • Should data saving fail, try and remove the session lock in a separate request. If that fails, have a timeout which the lock becomes invalid after.
  • While the lock is active, other servers can’t overwrite data.
local function onPlayerAdded(player: Player)
    --creating session lock data
    local data = {
        JobId = game.JobId, --the current server's ID
        WriteTime = os.time() --time when it was written to
    }

    --...
    
    --since UpdateAsync returns the data, we can use that to retrieve data along with locking so less requests are used
    local success, result = pcall(DataStore.UpdateAsync, DataStore, key, function(old)
        --if the player has no data, another server can't be updating it because they are new
        if (not old) then return nil end

        --retrieve values
        local server, writeTime = old.SessionLock.JobId, old.SessionLock.WriteTime

        if (server and writeTime and server ~= game.JobId) then
            --another server's session lock exists. Check to see if it's expired.
            local difference = os.time() - writeTime
            if (difference >= 300) then --5 minutes as an example
                print("Session lock has expired. Siezing control.")
                old.SessionLock = data
            else --valid session lock in place
                print("The session is locked. Data cannot be overwritten.")
                player:SetAttribute("DataLoadFailed", true) --indicate failed data loading, however that may be
                return nil
            end
        else --no session lock in place
            old.SessionLock = data
        end
        
        return old
    end)

    if (success) then
        --do what you will with the data, stored in result
    else
        warn("Failed to load data either due to data store failure or session locking failure")
    end
end

You can then check this session lock whenever data is saved and if it’s not the same server’s ID don’t save data.

thanks for the help so far, should something like this work?

edit forgot to use preformatted text

local DataStoreService = game:GetService("DataStoreService")
local datastore = DataStoreService:GetDataStore("PlayerData")
local players = game:GetService("Players")
local maxRetries = 5 -- Max retries for saving data
local version = 1 -- Current data version

print("Data Save and Load Script Active")

-- Deserialize player data and apply it to the leaderstats
local function deserializeData(player, data)
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for statName, statValue in pairs(data) do
			local stat = leaderstats:FindFirstChild(statName)
			if stat then
				stat.Value = statValue
			end
		end
	end
end

-- Serialize player data from leaderstats for saving
local function serializeData(player)
	local data = {
		Version = version, -- Track data version
		SessionLock = {} -- Session lock data
	}
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for _, stat in ipairs(leaderstats:GetChildren()) do
			data[stat.Name] = stat.Value
		end
	end
	return data
end

-- Load and lock player data when they join
local function onPlayerAdded(player)
	local sessionData = {
		JobId = game.JobId,
		WriteTime = os.time()
	}

	local success, result = pcall(function()
		return datastore:UpdateAsync(player.UserId, function(oldData)
			-- If no previous data, initialize
			if not oldData then
				oldData = { Version = version, SessionLock = sessionData }
				return oldData
			end

			local server, writeTime = oldData.SessionLock.JobId, oldData.SessionLock.WriteTime

			-- Check if the session is already locked by another server
			if server and server ~= game.JobId and (os.time() - writeTime < 300) then
				player:SetAttribute("DataLoadFailed", true)
				return nil -- Cancel loading due to active session lock
			end

			-- Update session lock for current server
			oldData.SessionLock = sessionData
			return oldData
		end)
	end)

	if success and result then
		deserializeData(player, result)
	else
		warn("Data load error for " .. player.Name .. ": " .. (result or "Unknown error"))
		player:SetAttribute("DataLoadFailed", true)
	end
end

-- Retry saving with version and session lock checks
local function saveDataWithRetries(player, data, retries)
	retries = retries or 0
	data.Version = version

	local success, result = pcall(function()
		return datastore:UpdateAsync(player.UserId, function(oldData)
			-- Ensure oldData exists and has a Version field, defaulting to 0 if not
			if oldData and (oldData.Version or 0) >= data.Version then
				return nil -- Cancel save if the stored version is newer or equal
			end

			-- Check session lock
			if oldData and oldData.SessionLock and oldData.SessionLock.JobId ~= game.JobId then
				warn("Session lock mismatch for " .. player.Name .. ", save canceled")
				return nil
			end

			return data
		end)
	end)

	if not success and retries < maxRetries then
		warn("Retrying data save for " .. player.Name .. " (Attempt " .. (retries + 1) .. ")")
		saveDataWithRetries(player, data, retries + 1)
	elseif not success then
		warn("Failed to save data for " .. player.Name .. " after " .. maxRetries .. " attempts: " .. result)
	else
		print("Saved data for", player.Name)
	end
end

-- Handle player removal
local function onPlayerRemoving(player)
	local data = serializeData(player)
	saveDataWithRetries(player, data)
end

-- Server shutdown
local function onServerShutdown()
	task.wait(20)
end

-- Connect events
players.PlayerAdded:Connect(onPlayerAdded)
players.PlayerRemoving:Connect(onPlayerRemoving)
game:BindToClose(onServerShutdown)

From looking over it, it looks like it should work fine. Test it and let me know if you encounter any issues.

you are not going to enjoy waiting 20 seconds for Studio to become useable again why not just do this:

local function onServerShutdown()
    if game:GetService("RunService"):IsStudio() then
        task.wait(5)
    else
        task.wait(30)
    end
end

oh thanks lol that’s why studio was freezing up i guess however there is an issue and that the previous data before this script (data that was saved with the original script that didn’t get lost) is getting reset to 0 i want it to keep that old data since if players rejoin after I publish update with the new data script their data will be lost

You can compare with the old data format and update it to a newer format before you check the session lock. For example, if they have no session lock data, just add the unlocked version with an older write time before then checking the session lock.

I tested and it successfully displayed the old data that was created from the original script however I just wanna ask if this is a good way of doing it?

local DataStoreService = game:GetService("DataStoreService")
local datastore = DataStoreService:GetDataStore("PlayerData")
local players = game:GetService("Players")
local maxRetries = 5 -- Max retries for saving data
local version = 1 -- Current data version

print("Data Save and Load Script Active")

-- Deserialize player data and apply it to the leaderstats
local function deserializeData(player, data)
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for statName, statValue in pairs(data) do
			local stat = leaderstats:FindFirstChild(statName)
			if stat then
				stat.Value = statValue
			else
				-- Set default values for missing stats
				if statName == "Points" then
					data[statName] = data[statName] or 0 -- Default Points
				elseif statName == "PlayTime" then
					data[statName] = data[statName] or 0 -- Default PlayTime
				end
			end
		end
	end
end

-- Serialize player data from leaderstats for saving
local function serializeData(player)
	local data = {
		Version = version, -- Track data version
		SessionLock = {} -- Session lock data
	}
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
		for _, stat in ipairs(leaderstats:GetChildren()) do
			data[stat.Name] = stat.Value
		end
	end
	return data
end

-- Check and upgrade data format if outdated
local function upgradeDataFormat(oldData)
	-- Set default version if missing
	oldData.Version = oldData.Version or 0

	-- If missing SessionLock, add default to mark as unlocked
	if not oldData.SessionLock then
		oldData.SessionLock = {
			JobId = nil,
			WriteTime = 0 -- Far-past time to indicate no active lock
		}
	end

	-- Update Points and PlayTime for older versions if missing
	oldData.Points = oldData.Points or 0
	oldData.PlayTime = oldData.PlayTime or 0

	-- Update to current version
	oldData.Version = version

	return oldData
end

-- Load and lock player data when they join
local function onPlayerAdded(player)
	local sessionData = {
		JobId = game.JobId,
		WriteTime = os.time()
	}

	local success, result = pcall(function()
		return datastore:UpdateAsync(player.UserId, function(oldData)
			-- Initialize new data if none exists
			if not oldData then
				oldData = { Version = version, SessionLock = sessionData }
				return oldData
			end

			-- Upgrade old data format if needed
			oldData = upgradeDataFormat(oldData)

			-- Session lock check after upgrade
			local server, writeTime = oldData.SessionLock.JobId, oldData.SessionLock.WriteTime
			if server and server ~= game.JobId and (os.time() - writeTime < 300) then
				player:SetAttribute("DataLoadFailed", true)
				return nil -- Cancel loading due to active session lock
			end

			-- Update session lock for current server
			oldData.SessionLock = sessionData
			return oldData
		end)
	end)

	if success and result then
		deserializeData(player, result)
	else
		warn("Data load error for " .. player.Name .. ": " .. (result or "Unknown error"))
		player:SetAttribute("DataLoadFailed", true)
	end
end

-- Retry saving with version and session lock checks
local function saveDataWithRetries(player, data, retries)
	retries = retries or 0
	data.Version = version

	local success, result = pcall(function()
		return datastore:UpdateAsync(player.UserId, function(oldData)
			-- Ensure oldData exists and has a Version field, defaulting to 0 if not
			if oldData and (oldData.Version or 0) >= data.Version then
				return nil -- Cancel save if the stored version is newer or equal
			end

			-- Check session lock
			if oldData and oldData.SessionLock and oldData.SessionLock.JobId ~= game.JobId then
				warn("Session lock mismatch for " .. player.Name .. ", save canceled")
				return nil
			end

			return data
		end)
	end)

	if not success and retries < maxRetries then
		warn("Retrying data save for " .. player.Name .. " (Attempt " .. (retries + 1) .. ")")
		saveDataWithRetries(player, data, retries + 1)
	elseif not success then
		warn("Failed to save data for " .. player.Name .. " after " .. maxRetries .. " attempts: " .. result)
	else
		print("Saved data for", player.Name)
	end
end

-- Handle player removal
local function onPlayerRemoving(player)
	local data = serializeData(player)
	saveDataWithRetries(player, data)
end

-- Server shutdown
local function onServerShutdown()
	if game:GetService("RunService"):IsStudio() then
		task.wait(5)
	else
		task.wait(30)
	end
end
-- Connect events
players.PlayerAdded:Connect(onPlayerAdded)
players.PlayerRemoving:Connect(onPlayerRemoving)
game:BindToClose(onServerShutdown)

That looks alright, but it may be better to create a function to handle it all as I see you have about 3 different points in your code where you do it.

I’ll also include a way to handle nested tables in case you add them in the future - table.clone doesn’t clone nested tables so modifying them would modify the default.

local missingValues = {
    ["Points"] = 0,
    ["ExampleTable"] = {{}}
    --add other values as field: default
}

local function deepCopy<iterable>(target: iterable): iterable
	--new table
	local clone = {}
	
	--clone each value
	for key, value in next, target, nil do
		--if it's a table, clone it
		if type(value) == "table" then
			clone[key] = deepCopy(value)
		else
			clone[key] = value
		end
	end
	
	return clone
end


--then, when you need to:
for key, value in next, missingValues, nil do
    data[key] = value
end
1 Like

thanks for all your help getting these scripts to work!!

1 Like

For your sake, I hope you never need more stats. Hardcoded checks on named objects are a source of OOP headaches. There should never be hardcoded reference/index/value checks on any container, EVER! If they have the same constructor, then treat them just the same. Use looped functional logic to check, edit, compare and infinitely scale them, with zero code refactoring.

And:

local function deserializeData(player, data)
	local leaderstats = player:FindFirstChild("leaderstats")
	if leaderstats then
	...
	end

No fail condition, so it remains unhandled if there are no leaderstats. What does that mean? You tell me. Is that gonna hurt you someday?

Once again:

local function onPlayerAdded(player)
...

	-- Load data from DataStore
	local success, result = pcall(function()
		return datastore:GetAsync(player.UserId)
	end)

	if success and result then
		deserializeData(player, result)
	else
		if not success then
			warn("Data error: "..result) <-- this changes nothing, no return, no failure mechanism
		end
	end

	-- Start tracking time played? because who cares if it failed above???
	task.spawn(function()
		trackTimePlayed(player)
	end)
end

There is way more than this, but my fingers are tired!