Will Datastores ever successfully return malformed data?

Help and Feedback Scripting Support
#1

Lately I’ve been rethinking how I approach data saving/loading to implement some better practices and reduce likelihood of data loss and corruption, since I’ve been having issues in that area in the past few projects I’ve released.

At the moment, I’m focusing on handling failed loads. My current strategy is to pcall the GetAsync call, and if it errors, notify the player and retry every 10 seconds or so until the call is successful. Whether or not the call actually errors is how I determine if the data is acceptable or not, but that means that the player’s data could still be wiped if the Datastore happens to successfully return nil or some other incorrect response. I’ve considered also checking if the data is nil, and retrying if that’s the case, but that wouldn’t work for new players joining the game for the first time with no previous data saved.

My question boils down to this: Do I need to be concerned about Datastores returning malformed/missing data without throwing an error? Or will it always error if it can’t get the data exactly right? If it does return malformed/missing data, how can I avoid an overwrite?

1 Like
#2

I think, so, data should save, after rejoining, etc. If its, not working, show me the code.

#3

I’m not asking for help with any specific code, I’m asking if I should be concerned about certain aspects of how I write it. My purely theoretical code would be dependent on Datastore calls either being completely correct or erroring out, never anything else. However, there have been some cases where that hasn’t been the case, and I’m wondering if (and how) I should plan for it to happen again:

#12

Just a disclaimer: I’m no Roblox engineer nor a Web Developer but I’ll do my best with what knowledge I have to answer these questions.

Q: Do I need to be concerned about Datastores returning malformed/missing data without throwing an error?

I would be concerned but, if this happens often, then I would contact Roblox immediately-there’s not really a way you can fix that on your end if it’s Roblox who is having the issues. Some good third party services that help you trace errors like these can be found on the internet. Personally, I recommend Sentry.

Q: Will it always error if it can’t get the data exactly right?

I would say yes. To my understanding, data storage is sensitive and has to be just right (aka JSON encoded). However, if you’re saving it the current way you are, you should be fine.

Q: If it does return malformed/missing data, how can I avoid an overwrite?

My suggestion is to offload data to a separate emergency server. Either that or you will have to respond in your own way by utilizing the error watching method above.

Hopefully I helped at least some bit. This is a really big issue and I have to say these issues are purely theoretical. Roblox does the best they can to keep their data store service up, I’m sure; However, the ability to deal with this will vary among developers and no answer to it is simple.

2 Likes
#14

Thanks, but I think this is more applicable to managing save errors, which is a whole other ballpark. Right now I’m focused more on the loading aspect. I’m wondering if there’s anything I can do to identify the difference between a player who had no data in the first place, and a player whose data hasn’t been loaded correctly, so I wouldn’t have to be concerned about any potential bad calls.

2 Likes
#15

In that case I have a couple comments.

for one, I would NOT recommend using a second data store to store “new player” values if the service is experiencing issues.

For a fix, I’d recommend using the Badge Service since many games already use it to identify new players and it gives them a fun award that makes them feel good about themselves.

EDIT: In which case, you’d be able to utilize nil values in a badge check to determine if a player’s data failed to load or if it is a new player.

3 Likes
#17

I do like this idea, but it raises a couple potential issues of its own:

  • BadgeService has its own potential vulnerabilities (to what extent I have no idea)
  • If a player joins the game and gets the badge during a Datastore outage, next time they join it’ll trigger a false positive (can’t get any data, but player has played before).

I suppose in that second scenario, it would be up to the saving mechanism to manage the issue–maybe by not awarding the badge until the first save is successful?

1 Like
#18

Unfortunately, the Badge Service will have it’s fair share of issues but as does everything now and then. Heck, that’s why we’re discussing this in the first place. I can’t give you a permanent fix for the system but maybe speak to a Roblox engineer on the topic to come up with some creative solutions to it.

To explain my idea around utilizing the badge service, it would follow this grid:

Player joins > service checks data (if nil then check badge > if nil then player is new else player is old) > if data exists then the player is old

Players.PlayerAdded:Connect(function(plr)
    if playerHasData(plr) then
        if playerHasBadge(plr) then
            -- player is old
        else
            -- player is new
        end
    else
        -- player isn't new
    end
end)
1 Like