[R$10,000 Bounty] Random Data Loss

Oh nice, so the check isn’t even needed there then, can simply be… (am I correct?)

	local success, err = pcall(function()
	
	-- print("Setting all the data now", player.Name .. "(" .. playerId .. ")")

	print("Updating a player's stats now")
	dataStore:UpdateAsync(player.UserId, function(old)
		return Copy
	end)
  
end)
if not success then
	spawn(function()
		error(err)
	end)
	wait(1)
	Attempts = Attempts or 1
	return Attempts < 10 and Module.updateStatsToDataStore(player, Attempts + 1)

Yeah. I supplied an update of the pcall in my post in an edit, forgot to include that. I don’t know if that solution actually fixes anything for you but at the very least I can fix what it’s doing. One thing: changed the second return variable of the pcall to ret because UpdateAsync does return the new data and pcall is a callback, so it can return more than just an error message.

3 Likes

A while back I was working with a DataStore and I had problems with the data resetting. It turns out it was because I was checking for data before the data had time to actually load. My solution was to make the module fire an event when the data was ready and the the script could wait for that event before trying to read/set data. (Using event:Wait() )
Is there a chance that you are trying to edit data right when a player joins and you didn’t give the data enough time to load? This could cause laggy clients to lose data.

Do the players only lose data when the join back into a game or can they lose data mid-game? Knowing this this could help find the problem.

We have wait statements to confirm the data has loaded.

I’ve currently moved to UpdateAsync(), so far no reports of data loss but will be finding out shortly.

2 Likes

I have something new to report, apparently all the players who have suffered from data loss “crashed.”

Meaning they didn’t voluntarily leave the game.

I’ll reply to you gain 44, update async seems to have lowered, but it appears when a player crashes it reverts to their initial data rather than their new one.

1 Like

Is that progress then? Have you been able to resolve a couple of data loss issues aside from players crashing? In that case, there might be a few flow changes to be doing.

In my experience, it’s best to decouple players and data. While I will use a player in order to specify who’s data belongs to who, crashes (server or client) and shut downs tend to not fair well. That’s why when it comes to BindToClose functions, I assume that the server doesn’t have players. I typically do not experience data loss this way but it could be a different story for you.

What I do is assign player data to a key in a dictionary by their UserId. PlayerAdded would add their UserId as a key to the dictionary with a value of their data and PlayerRemoving would save and remove the key. Now the difference is that for auto save and BindToClose, I have no need for player objects so if they are unexpectedly removed, I can handle that case. I just iterate through the dictionary with pairs and accordingly update DataStores with the key-value pair in the current iteration.

No player object necessary, no problem. The server will exist for up to 30 seconds after all players are gone so making the assumption that no one is there after the fact may help. The server would still be able to apply data saves without any player object required.

Essentially: have your data module rely less on the player object and more on their information. In PlayerAdded/Removing, send the UserId to the data module. For BindToClose and auto saving, iterate through the dictionary (will have to expose the raw data tables from DataManager) and call the save function across each entry in there.

5 Likes

Someone sent me this video replicating it, seems like a total loss.

2 Likes

Wow. What an interesting but frustrating problem.

The last suggestion I have before I’ve completely exhausted the ways I can help (and will just try writing something in Studio or the like): use CollectionService as a backup effect. A crashing player seems to not trigger PlayerRemoving or your save feature, but surely CollectionService should.

Players are destroyed when they leave the server so I’m working by the assumption that the Player instance of a crashed player still gets destroyed but just that PlayerRemoving doesn’t fire. I would then attach a tag to each Player from AddTag so the server knows for later. Just a casual line in any relevant PlayerAdded function:

CollectionService:AddTag(Player, "Player")

Now, you may need to do a bit of flow editing so you don’t trip a race condition or end up double saving (PlayerRemoving fires before the Player instance is destroyed), but from CollectionService, use GetInstanceRemovedSignal against your aforementioned player tag and connect a saving function to it. This way, when the Player instance is destroyed, it’ll tell the module to save data.

Destroyed will automatically clear tags off an instance which will trigger the remove signal and the event fires with the removed instance, so:

CollectionService:GetInstanceRemovedSignal("Player"):Connect(function (Player)
    -- Data saving code on the Player
end)

You still might want to look into workflow changes where the Player is only useful for supplying the UserId but otherwise you don’t involve the Player object in the data module. I specified some of the benefits some ways up so if you ever need a reference, that’s there.

1 Like

I find the reset weird though, stats should still save every minute, not sure why a crash back tracks it completely.

1 Like

Yeah wait if the stats are resetting, but there’s auto save enabled, it can’t possibly mean that leaving resets it.

Could it be set to nil when a player leaves or something of the sort?

Does saving at all ever work in any circumstance or does data loss occur 100% of the time? If it’s the former, then there’s a deeply rooted implementation problem to resolve. If it’s the latter, then the saving code has some kind of issue in which that whole bit would need to be rewritten.

You can check for if the entry goes nil using prints. I wouldn’t know, that seems purely like an implementation thing.

Got a secondary video.

Data saving works 99.9% of the time, there’s 8,000 concurrent players online and it’s very very very rare resets.

1 Like

Okay things I’ve noticed.

In the first section he has all his stuff, level 4 with 180 XP.

In the second version, he’s level 1 with 100 XP.

function characterAdded(player, character)

repeat wait(0.1) until dataManager.playerDataLoaded(player)

local playerData = dataManager.playerData(player)

character.Humanoid.Died:Connect(function()
	petEvent:FireAllClients({UpdateType = "RemoveAllPets", Petplayer = player})
end)

local leaderstats = Instance.new("Folder")
leaderstats.Name = "leaderstats"

local gold = Instance.new('IntValue')
gold.Name = 'Gold'
gold.Value = playerData.Stats.Money
gold.Parent = leaderstats

local levels = Instance.new('StringValue')
levels.Name = 'Level'
levels.Value = playerData.Stats.Level
levels.Parent = leaderstats

local timePlayed = Instance.new('StringValue')
timePlayed.Name = 'Time'
local currentTime = playerData.Stats.TimePlayed
timePlayed.Value = ('%02i:%02i:%02i'):format(currentTime / 60 ^ 2, currentTime / 60 % 60, currentTime % 60)
timePlayed.Parent = leaderstats

leaderstats.Parent = player

His leaderboards aren’t loaded on the top right, meaning that where the data is loaded failed.

local successData, currentStatsData = pcall(function()
	return dataStore:GetAsync(player.UserId)
end)
if successData then
	if currentStatsData == nil then
		--Module.updateStatsToDataStore(player)
		
		Module[player.UserId]["Loaded"] = true
		Module[player.UserId]["Spawned"] = false	
	else
		--Module[player.UserId] = currentStatsData

– local json = game:GetService(“HttpService”):JSONEncode(currentStatsData)
– print(“Here”)
– print(json)

		local editedVersion = deepCopy(player, Module[player.UserId], currentStatsData)
		local editedVersion2 = nextDeepCopy(player, editedVersion, currentStatsData)
		
		Module[player.UserId] = tableMerge(editedVersion, editedVersion2)
		--Module[player.UserId] = currentStatsData
		
		Module[player.UserId].Stats.MaxInventoryPets = 25
		
		Module[player.UserId].Loaded = true
		Module[player.UserId].Spawned = false	
		
		local playerData = Module.playerData(player)
		
		-- print("Finished fetching data for " .. player.Name, "("..player.UserId..")")
	end
else

Loaded wasn’t set to true apparently.

When he rejoins for a third time, the player data is just completely reset as the XP bar shows it correctly.

1 Like

Update number 3, went through game analytics which apparently finds erros.

ServerStorage.Managers.DataManager:140: bad argument #1 (table expected, got nil)
Stack Begin
Script 'ServerStorage.Managers.DataManager', Line 140 - function deepCopy
Script 'ServerStorage.Managers.DataManager', Line 239 - function getDataFromDataStore
Script 'ServerStorage.Managers.DataManager', Line 126 - function insertPlayer
Script 'ServerScriptService.Events.DataHandler', Line 15
Stack End

ServerScriptService.Events.PlayerJoin:267: attempt to index nil with 'Pets'
Stack Begin
Script 'ServerScriptService.Events.PlayerJoin', Line 267 - function characterAdded
Script 'ServerScriptService.Events.PlayerJoin', Line 23
Stack End

ServerStorage.Managers.DataManager:231: attempt to index nil with 'Loaded'
Stack Begin
Script 'ServerStorage.Managers.DataManager', Line 231 - function getDataFromDataStore
Script 'ServerStorage.Managers.DataManager', Line 126 - function insertPlayer
Script 'ServerScriptService.Events.DataHandler', Line 15
Stack End

According to the data, it’s still going up.

2 Likes