Data loss starting two days ago

I didn’t hear any feedback from engineers so far.

1 Like

@Flubberlutsch I think this should be ROBLOXCRITICAL since it seems to be affecting multiple games.

1 Like

All DS2 does better is allow you to revert to previous saves. As far as I know, there is no way to detect if data has been lost if this bug happens. What’s happening is the roblox data store servers are making it look like the player has never had any save data. It returns the exact same thing as if they joined the game for the first time without any errors.

EDIT: Unless you have something like an external proxy server that you compare against

3 Likes

I have a player feedback system. People are reporting data loss in small bursts.

Latest reports:
(01/07/2019) 4:20 AM PST - 5:40 AM PST : 6 people reported losing their data
(01/06/2019) 2:30 AM PST : 1 person reported losing their data
(01/05/2019) 7:30 PM PST - 8:10 PM PST : 2 people reported losing their data
(01/03/2019) 7:30 PM PST - 9:00 PM PST : 3 people reported losing their data
(01/02/2019) 11:40 PM PST : 1 person reported losing their data

There are no complaints about item loss beyond Jan 2’nd

1 Like

Wasn’t sure if this problem was something on my end when I first posted. I don’t have the time to create a detailed report rn, feel free to do so.

This isn’t the first time this sort of issue has happened, and having seen it before, there’s not much to do if you didn’t already have code to help restore what’s lost :frowning: (i.e. having a log for player data saves to restore old data from)

Definitely needs to be ROBLOXCRITICAL as stated by EncodedLua - that will notify an engineer (from what I’ve been told), speeding up the fix.

This is happening to Vehicle Simulator too (multiple reports of people losing data w/o changes to our datastore code). I will edit this post w/our datastore code when I’m home in a bit.

I believe it started occurring on Christmas last year.

1 Like

Vehicle Simulator players have been complaining about data loss for a few days now as well.

ROBLOX CRITICAL

3 Likes

Hi everyone,

Thanks for sharing your concerns. In order to help with these issues, we need error codes/messages to diagnose what is happening. If you guys can reply to this post with said information I can work with engineers to diagnose what is going on.

3 Likes

No errors are happening, that’s the issue.

1 Like

Getasync/setasync (the two we use for data) are wrapped in pcall. If the call errors, we don’t save the data w/auto save/when leaving. The calls are returning/setting nil without erroring (same behavior as someone playing the game for the first time).

How often are you pinging get/set? Are you 100% sure they aren’t being throttled?

Should be within limitations - we autosave everyone’s data every 10 min (w/X second delay, I think 5-10 seconds) between calls - if it errors it waits longer before moving to the next person). Get only calls on joining the game.

Data saves when leaving the game + auto save. Get fires when joining (and gives 10 attempts w/delay inbetween if last call errored). Get also checks the budget for getasync and if it’s 0, it waits until it’s not 0/the 10 attempt queue is exhausted.

Edit: Roughsphereblox is a life saver - ends up if the call to getasync is successful or fails/throttles, the result is cached for 4s. I knew it was cached for successful requests, but didn’t realize it applied to throttled/errored requests too. Thank you!!!

For the past week, I’ve been having over 10 people coming to me due to their data completely being wiped from in-game, no clue what’s going on with it, but hope it gets fixed soon.

Can confirm this, have had a lot of people reporting data loss lately when I never had any reports about it before.

There’s no new errors in my logs and players who lost their data can play the game just fine, which shouldn’t work if there was an error loading it. They also seem to get my tutorial popups again which only happens after joining without data, so it’s as if their data has been wiped completely internally.

1 Like

Traitor Town has it too. I’ve had to close the game because of this.

Heads up, after talking to the amazing @RoughSphereBlox (Seriously, you’re the real MVP here), this was the issue with Vehicle Simulator’s data code (But likely applies to other games too that try repeated attempts to load data):

If you make a call to getasync, it caches for 4s (Per the wiki documentation). If the request throttles/errors, and another request is made within that 4s period, getasync will return nil w/o erroring (Because nil was what was cached).

Due to holiday traffic, this bug never reared its head for us (Because we only had two calls to getasync when a player joined the game).

Our game (When a player joins) would try 10x to load the data (In succession w/a delay of 1s if data failed to load). If the data failed to load the 1st time (Due to throttling/errors), it’d wait 1s and try again. Being that it was within that 4s cooldown period, it would return nil w/o throwing an error (Because technically it was a cached result).

RoughSphereBlox/the engineers recommend using an exponential delay with trying to call getasync (If it errors/throttles).

Example:
1st attempt fails, wait 6s.
2nd attempt fails, wait 12s.
3rd attempt fails, wait 18s.
etc.

tl;dr: Calling getasync within 4s of calling getasync (If it successfully runs/errors/throttles) returns the cached value, even if it’s nil because of an error/throttle.

Edit: This is speculation at this point, need a roblox engineer to verify if this is the case or not.

13 Likes

The holidays always seem to be when the bugs start appearing en masse. Thanks for posting this solution so others can apply this as well.

Was there any reason specified as to why an exponential delay was recommended? I’m kind of iffy on doing this. I completely support whatever it takes to ensure players receive their data, but keeping them waiting is another thing. Both waiting a long time and losing data are undesirable and can hamper the patience of players (especially within a young demographic), resulting in loss of players and consequently revenue.

RoughSphereBlox recommended it after conversing internally w/staff. Likely need the exponential delay if there’s issues loading data and you want to retry loading in succession.

1 Like

Currently my data system attempts to re-load a player’s data if GetAsync() fails. I’ll have to disable the load retries until caching is removed.

1 Like