I'm so fed up with data loss, please help!

My data saving code:

local playerData
local success, errorMessage = pcall(function()
	playerData = MainDataStore:GetAsync(player.UserId)
end)

if playerData then
	local success, errorMessage = pcall(function()
		MainDataStore:UpdateAsync(player.UserId,function(oldValue)
			local previousData = oldValue or {DataId = 0}
			if playerData.DataId == previousData.DataId then
				data.DataId += 1
				return data
			else
				return nil
			end
		end)
	end)
else
	MainDataStore:SetAsync(player.UserId,data)
end

I’ve only had one data loss report so far with adding UpdateAsync to my data store. A big game like Bedwars has no data loss whatsoever, so I know it’s possible to make a perfect data store. Please, any help is appreciated, my game has a 1k player average!

I really recommend you switch to DataStore2, it does all the work for you. It might be difficult transitioning to DS2 but it’s worth it.

1 Like

Before I can give specific input, I need to know a few things:

  1. How often is this code called/executed?
  2. What is the client (script/part of code in your project) of this code?

I did see some immediate issues:

  1. This is an incorrect use of UpdateAsync. What you are doing here is conducting a pcall against GetAsync to retrieve the player’s data. If this fails, the update never occurs. However, UpdateAsync does in fact have an internal “get.” In UpdateAsync, the current value is passed as an argument of your update function, and if your function returns something, the value is updated. Check the docs here.
  2. You are conducting absolutely no backups. This also makes me think you have no disaster recovery plan. If you have 1000 concurrent users, your game is no longer a Roblox game; it is an enterprise service that users expect to function at a high degree of uptime and reliability. Not having backups / a disaster recovery plan is the easiest way to lose data if things go wrong. And data loss is one of the easiest ways to lose player trust.
  3. You have absolutely no instrumentation on this code. If something goes wrong, nothing is printed, making it impossible to investigate this issue by checking your game’s logs. You should, at minimum, print error messages if they occur so that you can check your game’s logs and get fine information about the error. What would be even better is tracking errors using an analytics service like Google Analytics or PlayFab so you have keep track of the amount of times errors are occurring across your entire game.

Building atop @Fusionet, many developers have noticed the shortcomings of DataStore and have built libraries that compensate for some of its issues. In addition to DataStore2, ProfileService is also a great alternative.

2 Likes

can you send some example code for what you think an UpdateAsync function should look like?

We can go through the example on the documentation:



local DataStoreService = game:GetService("DataStoreService")
 
local nicknameStore = DataStoreService:GetDataStore("Nicknames")
 
-- The is the update function. The first argument 'currentName' passes the current value of the key being updated. KeyInfo contains metadata.
local function makeNameUpper(currentName, keyInfo)
	local nameUpper = string.upper(currentName) -- Here, we save a local variable that is the uppercase version of the current value of the key being updated.
	local userIDs = keyInfo:GetUserIds() -- Metadata stuff, feel free to ignore...
	local metadata = keyInfo:GetMetadata() -- Metadata stuff, feel free to ignore...
	return nameUpper, userIDs, metadata -- We return several things. The last two are optional, but the first thing returned is the value that should replace the current value assigned to this key in the database.
end
 
-- This calls UpdateAsync on the key "User_1234" against the update function makeNameUpper
local success, updatedName, keyInfo = pcall(function()
	return nicknameStore:UpdateAsync("User_1234", makeNameUpper)
end)
if success then
	print(updatedName)
	print(keyInfo.CreatedTime)
	print(keyInfo:GetMetadata())
end

Something this example does not do is instrument errors. We can do that by changing the end lines like so:

if success then
	print(updatedName)
	print(keyInfo.CreatedTime)
	print(keyInfo:GetMetadata())
else
    warn("An error occured when trying to update the datastore on key "..updatedName)
    warn(updatedName) -- The second returned element from a pcall will be the error msg/object.
end
1 Like

this is my full data saving script btw, and idk how I’d be able to implement the amount of data I’m trying to save.

local data = {}

data.DataId = 0

data.leaderstats = {}
data.Blocks = {}
data.BoughtBlockAmounts = {}
data.Armor = {}
data.SelectedArmor = ""
data.Weapons = {}
data.SelectedWeapon = ""
data.GamepassWeapons = {}
data.Items = {}
data.SelectedItems = {}
data.Votes = {}

data.Round = Values.Round.Value
data.JobId = game.JobId

data.leaderstats.Gunpowder = player.leaderstats.Gunpowder.Value
data.leaderstats.Wins = player.leaderstats.Wins.Value

data.Banned = player.Banned.Value

local Settings = player.Settings
data.Settings = {}
data.Settings.Flashes = Settings.Flashes.Value
data.Settings.Particles = Settings.Particles.Value
data.Settings.Sounds = Settings.Sounds.Value

for i, v in pairs(player.leaderstats:GetChildren()) do
	data.leaderstats[v.Name] = v.Value
end

for i, v in pairs(player.BlockAmounts:GetChildren()) do
	data.Blocks[v.Name] = v.Value
end

for i, v in pairs(player.BoughtBlockAmounts:GetChildren()) do
	data.BoughtBlockAmounts[v.Name] = v.Value
end

for i, v in pairs(player.Armor:GetChildren()) do
	data.Armor[v.Name] = v.Amount.Value
end

for i, v in pairs(player.SelectedArmor:GetChildren()) do
	data.SelectedArmor = v.Name
end

for i, v in pairs(player.Weapons:GetChildren()) do
	if not Gamepasses:FindFirstChild(v.Name) then
		table.insert(data.Weapons,v.Name)
	else
		table.insert(data.GamepassWeapons,v.Name)
	end
end

for i, v in pairs(player.SelectedWeapons:GetChildren()) do
	data.SelectedWeapon = v.Name
end

for i, v in pairs(player.Items:GetChildren()) do
	data.Items[v.Name] = v.Amount.Value
end

for i, v in pairs(player.SelectedItems:GetChildren()) do
	if not Gamepasses:FindFirstChild(v.Name) then
		data.SelectedItems[v.Name] = v.Amount.Value
	end
end

for i, v in pairs(player.VoteKicks:GetChildren()) do
	table.insert(data.Votes,v.Name)
end

local playerData
local success, errorMessage = pcall(function()
	playerData = MainDataStore:GetAsync(player.UserId)
end)

if playerData then
	local success, errorMessage = pcall(function()
		MainDataStore:UpdateAsync(player.UserId,function(oldValue)
			local previousData = oldValue or {DataId = 0}
			if playerData.DataId == previousData.DataId then
				data.DataId += 1
				return data
			else
				return nil
			end
		end)
	end)
else
	MainDataStore:SetAsync(player.UserId,data)
end

if not success then
	warn("Error saving data: "..errorMessage)
end

I’m not going to fix this code for you, but I will give pointers:

  1. You are saving a huge amount of data into a single key. This not only puts you at risk of hitting the single-key data size limit, but it is also leaves you at risk of large-scale data loss. If you save all your users’ data into a single key, then if the datastore save operation on that key fails, then they lose all their data since the last time the data saved.
  2. Remove the redundant GetAsync call as discussed.
  3. Instrument errors as discussed. When data loss occurs, join the server where it happened and read the error off of the dev console. That will give you 100% surefire insight into the underlying cause of the function failing.
1 Like

after I remove I remove the GetAsync, it errors, since it won’t be able to save right. Now what do I do :yawning_face:

	local success, errorMessage = pcall(function()
		MainDataStore:UpdateAsync(player.UserId,function(oldValue)
			local previousData = oldValue or {DataId = 0}
			if playerData.DataId == previousData.DataId then
				data.DataId += 1
				return data
			else
				return nil
			end
		end)
	end)

“playerData” becomes nil

Again, I will be getting something from BaseAdmin.

So, I have had absolutely no data loss whatsoever with this model. Other users didn’t complain either. It doesn’t save data when the player leaves, it saves data when the player joins. It saves data when it changes.

So, this is what I think you should be looking for. Save the data when it changes, not when the player leaves. When you save data when the value changes rapidly, it won’t work. But in BaseAdmin, you aren’t saving the data rapidly, so this isn’t a problem for me.

For bans, where the data is crucial, I honestly thought that datastores will be unreliable (and they actually were early in development), so I added some fixes to it, just by detecting if the data is missing and trying to load it in another time.

If this is your problem, then I actually have no idea how to solve it. I have the same issue and I cannot understand why BaseAdmin’s save system is so much better than my game. I made both of them.

1 Like

I finally got it!

local success, errorMessage = pcall(function()
	MainDataStore:UpdateAsync(player.UserId,function(oldValue)
		local previousData = oldValue or {DataId = 0}
		if player.DataId.Value == previousData.DataId then
			data.DataId += 1
			return data
		else
			if previousData.DataId == nil then
				data.DataId += 1
				return data
			else
				return nil
			end
		end
	end)
end)

What do you think btw? I’m solutioning for now though in case ppl see this