Data loss starting two days ago


#1

I’ve noticed severe data loss affecting all kind of data (player data as well as internal data) starting two days ago. Player data is not overwritten or else affected if the loading process throws an error. In order for the data to be actually lost, getting must return nil, or setting must write nil. The issue does not happen permanently and it hasn’t happened while I was present yet.

I made custom data store functions to prevent throttling and similar. I haven’t made any changes to either my game nor the module within a few days, so this starting all of the sudden is likely related to some sort of change to the engine. I don’t know if this is some sort of Roblox issue or not.

Below are my functions. “lib” refers to the table holding the functions.

Init
local success = pcall(function()
	lib.DataService = game:GetService("DataStoreService")
	lib.DataStore = lib.DataService:GetGlobalDataStore()
	lib.ListDataStore = lib.DataService:GetOrderedDataStore("FlubsLibServerList")
end)
lib.DataReady, lib.DataInit = success, true
Budget
function lib:GetDataBudget(typ)
	return lib.DataService:GetRequestBudgetForRequestType(Enum.DataStoreRequestType[typ .. "Async"])
end
Get
function lib:GetAsync(key, store, size, min, max)
	repeat wait() until lib.DataInit if not lib.DataReady then return end
	if key == nil then return end
	local store, result = store or lib.DataStore
	for i=1,5 do
		local success = pcall(function()
			repeat wait() until lib:GetDataBudget("GetSorted") > 0 and lib:GetDataBudget("Get") > 0
			result = size and store:GetSortedAsync(key, size, min, max) or store:GetAsync(key)
		end)
		if success then return result end wait(10)
	end
	error("Failed to get async.")
end
Set
local removekey = "FlubsLibDataRemoveKey"
local setters = {}
function lib:SetAsync(key, value, store)
	repeat wait() until lib.DataInit if not lib.DataReady then return end
	if not (key and value) then return end
	local store = store or lib.DataStore
	setters[store] = setters[store] or {}
	setters[store][key] = setters[store][key] or {Last = 0}
	setters[store][key].Value = value
	repeat wait() until setters[store][key].Last + 7 < tick()
	if setters[store][key].Busy then return end
	setters[store][key].Busy = true
	for i=1,5 do
		local success, message = pcall(function()
			repeat wait() until lib:GetDataBudget("SetIncrement") > 0
			setters[store][key].Last = tick()
			local value = setters[store][key].Value
			if value == removekey then
				store:RemoveAsync(key)
			else
				store:SetAsync(key,value)
			end
		end)
		if success then setters[store][key].Busy = false return end wait(2)
	end
	setters[store][key].Busy = false error("Failed to set async.")
end

Developers need reliable DataStores. What we have now is not acceptable
Data Stores Issue Feedback
#2

There have been multiple posts lately regarding this same issue. I don’t think anybody has come up with a definitive answer but you may want to try switching over to datastore2 (How to use DataStore2 - Data Store caching and data loss prevention)

Also, AlvinBlox has a tutorial on this if you are having any issues: https://m.youtube.com/watch?feature=youtu.be&v=hBfMfB0BwGA


#3

If you want to take matters into your own hands, an alternative is to use HttpService and have your own server running with its own database. I’m not sure how often people do this, but it’s good for peace of mind.


#4

Can you update your code to prin the error of the pcall and I am not sure why you wait for both GetSorted and Get.

	for i=1,5 do
               -- use the result
		local success, res = pcall(function()
                        -- not sure about needing to wait for both ????
			repeat wait() until lib:GetDataBudget("GetSorted") > 0 and lib:GetDataBudget("Get") > 0
			result = size and store:GetSortedAsync(key, size, min, max) or store:GetAsync(key)
		end)
		if success then return result end
                warn("data store call failed attempt " .. res)
                wait(2)
	end

#5

Code fanciness is not what I’m on about. Preventing data loss has priority.
As I said, if data failed to load and the function threw an error, which it would do after 5 failed attempts, the player’s data would not be overwritten or modified. Thus errors on the pcall are irrelevant to this problem.
The function is returning nil for some reason I cannot explain. Since I didn’t change any of my data saving code in a few weeks before this started happening, I’m almost certain this is something on Roblox’ end.


#6

There are a number of ways your code will just return nil. You need to add in prints/output which are useful. This has nothing to do with “Code fanciness” it is simply including the relivant information upon an error / incorrect internal state.

All of this information will help you get to the core of the issue.


#7

It happened to four of my users today.

Based on my code their data was wiped because the DataStore call returned nil (without throwing an error in pcall) when fetching their player data. I have entries of player activity in my analytics, but their profiles have been wiped on Roblox. This is completely new for my game and has never been seen before.


#8

Looks like I’m having similar issues as loleris.
I haven’t touched any of my datastore code in weeks and I never had any issues with it. I just added some debugging and whilst it approved that all data has been loaded properly, someone just lost their player data.
The datastore is thus returning nil without throwing an error when there actually is data present.


#9

Shouldn’t this be ROBLOXCRITICAL? There’s not many valid uses for it but this is definitely one.

Unless the engineers are already aware of it.


#10

I didn’t hear any feedback from engineers so far.


#11

@Flubberlutsch I think this should be ROBLOXCRITICAL since it seems to be affecting multiple games.


#12

All DS2 does better is allow you to revert to previous saves. As far as I know, there is no way to detect if data has been lost if this bug happens. What’s happening is the roblox data store servers are making it look like the player has never had any save data. It returns the exact same thing as if they joined the game for the first time without any errors.

EDIT: Unless you have something like an external proxy server that you compare against


#13

I have a player feedback system. People are reporting data loss in small bursts.

Latest reports:
(01/07/2019) 4:20 AM PST - 5:40 AM PST : 6 people reported losing their data
(01/06/2019) 2:30 AM PST : 1 person reported losing their data
(01/05/2019) 7:30 PM PST - 8:10 PM PST : 2 people reported losing their data
(01/03/2019) 7:30 PM PST - 9:00 PM PST : 3 people reported losing their data
(01/02/2019) 11:40 PM PST : 1 person reported losing their data

There are no complaints about item loss beyond Jan 2’nd


#14

Wasn’t sure if this problem was something on my end when I first posted. I don’t have the time to create a detailed report rn, feel free to do so.


#15

This isn’t the first time this sort of issue has happened, and having seen it before, there’s not much to do if you didn’t already have code to help restore what’s lost :frowning: (i.e. having a log for player data saves to restore old data from)

Definitely needs to be ROBLOXCRITICAL as stated by EncodedLua - that will notify an engineer (from what I’ve been told), speeding up the fix.


#16

This is happening to Vehicle Simulator too (multiple reports of people losing data w/o changes to our datastore code). I will edit this post w/our datastore code when I’m home in a bit.

I believe it started occurring on Christmas last year.


#17

Vehicle Simulator players have been complaining about data loss for a few days now as well.

ROBLOX CRITICAL


#18

Hi everyone,

Thanks for sharing your concerns. In order to help with these issues, we need error codes/messages to diagnose what is happening. If you guys can reply to this post with said information I can work with engineers to diagnose what is going on.


#19

No errors are happening, that’s the issue.


#20

Getasync/setasync (the two we use for data) are wrapped in pcall. If the call errors, we don’t save the data w/auto save/when leaving. The calls are returning/setting nil without erroring (same behavior as someone playing the game for the first time).