[R$10,000 Bounty] Random Data Loss

There’s obviously an issue somewhere in that check, and it doesn’t hurt at all to have redundancy when it comes to something as sensitive as player data. You should always be doing your checks as close to the data being set as possible.

If you’d like help migrating to DataStore2, I’d be glad to assist.

2 Likes

I skimmed around the game a bit, and noticed that the leaderboards have some seemingly high values (top person has a quadrillion cash according to your game)
Can you ensure that this data loss is random across all users who have reported it? High number values may cause a data entry to act oddly.

Also, as @AspectW mentioned, you may want to check the size of the data.
You can do print( string.len(game:GetService("HttpService"):JSONEncode(data) ) to test it.

1 Like

This provokes an idea, tostring() all numbers and convert back/save as a JSON table string instead of the array?

Yeah unfortunately I’m a minecraft developer, thought I had to do data saving from scratch didn’t think there was a good working public one.

Still weirds me out nonetheless, I’ll keep the bounty open.

Would you wanna discuss on Discord?

I had that issue once in a game that went above the quintillions, and I used that method to fix it. I don’t believe it’s the best though, not sure

In callbacks passed to DataModel::BindToClose you are only given 30 seconds for all of them to complete. When those 30 seconds are up the server shuts down regardless of them being finished or not. I am not saying 100% of the issue is because of this but maybe this tutorial I wrote a few months ago might help. How to properly save player data in data stores upon server close

The error tends to happen with low currency numbers.

I don’t see quadrillion being a major issue as of right now.

One thing I have confirmed is that there were other players on a server, so it doesn’t have to do with bind to close, will look at what you suggested nontheless, but it’s not the cause of the error.

The datastore service is a webservice and has a chance of failing, as a result, it’s important to check whether the service is up, as if it fails in the middle of a game session for a player, even though it checks for the service being up when the player joins, it doesn’t check whether it changes in the middle of a session. As a result, one can use :GetAsync(plr.UserId) as a sort of webservice status checker when pcalled ```lua
local success, _ = pcall(function()
if PlayerManager[plr][“DataSaving”] then
return PlrData:GetAsync(plr.UserId)
end)
if not success then
PlayerManager[plr][“DataSaving”] = false
–Warn the player via gui
end
if PlayerManager[plr][“DataSaving”] then
–Run Code for saving

Could you follow up with your discord? I’d love to talk more on this, Possibly give my DataManager to review over and see some common errors.

Yep of course, it’s Bladian#6411

If it only happens in one of the games anything that’s the same between games can be eliminated. The issue must be coming from the scripts that require the module and use the function updateStatsToDatastore or the default data in insertPlayer.

I don’t know how much of the code in this module is the same between games but I would double check any lines that are different. Good luck!

1 Like

@BladianMC I private messaged you a possible solution, did you check it out yet?

Possibly because you used SetAsync without verifying previous data’s existance and logical sense, you would need to UpdateAsync to update the value.

I know that it’s for the bounty, but please post solutions in here so others can use it in the future. Even if it’s not the exact solution, it may help others.

2 Likes

It’s ridicoulously easy to set up and is far more powerful than the default datastore service.

I would suggest using DataStore2. It’s very useful and has more features than a normal datastore. Also, on your leaderboard, there is someone with a very high number of money. High numbers may cause some data saving scripts to behave oddly, so would check into that. My friend uses a data store script in his game:

  1. – Set up table to return to any script that requires this module script
  2. local PlayerStatManager = {}
  • local DataStoreService = game:GetService(“DataStoreService”)
  1. local playerData = DataStoreService:GetDataStore(“PlayerData”)
  • – Table to hold player information for the current session
  1. local sessionData = {}
  • local AUTOSAVE_INTERVAL = 60
  • – Function that other scripts can call to change a player’s stats
  1. function PlayerStatManager:ChangeStat(player, statName, value)
  2. local playerUserId = “Player_” … player.UserId
  3. assert(typeof(sessionData[playerUserId][statName]) == typeof(value), “ChangeStat error: types do not match”)
  4. if typeof(sessionData[playerUserId][statName]) == “number” then
  5. sessionData[playerUserId][statName] = sessionData[playerUserId][statName] + value
  6. else
  7. sessionData[playerUserId][statName] = value
  8. end
  9. end
  • – Function to add player to the “sessionData” table
  1. local function setupPlayerData(player)
  2. local playerUserId = “Player_” … player.UserId
  3. local success, data = pcall(function()
  4. return playerData:GetAsync(playerUserId)
  5. end)
  6. if success then
  7. if data then
  8. – Data exists for this player
  9. sessionData[playerUserId] = data
  10. else
  11. – Data store is working, but no current data for this player
  12. sessionData[playerUserId] = {Money=0, Experience=0}
  13. end
  14. else
  15. warn(“Cannot access data store for player!”)
  16. end
  17. end
  • – Function to save player’s data
  1. local function savePlayerData(playerUserId)
  2. if sessionData[playerUserId] then
  3. local success, err = pcall(function()
  4. playerData:SetAsync(playerUserId, sessionData[playerUserId])
  5. end)
  6. if not success then
  7. warn(“Cannot save data for player!”)
  8. end
  9. end
  10. end
  • – Function to save player data on exit
  1. local function saveOnExit(player)
  2. local playerUserId = “Player_” … player.UserId
  3. savePlayerData(playerUserId)
  4. end
  • – Function to periodically save player data
  1. local function autoSave()
  2. while wait(AUTOSAVE_INTERVAL) do
  3. for playerUserId, data in pairs(sessionData) do
  4. savePlayerData(playerUserId)
  5. end
  6. end
  7. end
  • – Start running “autoSave()” function in the background
  1. spawn(autoSave)
  • – Connect “setupPlayerData()” function to “PlayerAdded” event
  1. game.Players.PlayerAdded:Connect(setupPlayerData)
  • – Connect “saveOnExit()” function to “PlayerRemoving” event
  1. game.Players.PlayerRemoving:Connect(saveOnExit)
  • return PlayerStatManager

I would read some articles online to also find out what’s wrong. This issue may just be the game, because of the high number values. I hope you figure this out! I would also check out this article: Documentation - Roblox Creator Hub

-Inkqo

1 Like

Hey, could you accept me on Discord? I’ve made several custom DataStore solutions before so I might be able to help more in-depth on there.

It’d be appreciated if people stopped moving off to Discord or private messages attempting to suggest resolutions to this problem. The forums are still a public resource so in posting your solutions here, you can help others experiencing similar troubles. It’s not in good faith to hide solutions.


On the topic of actually reviewing this module, I’d like to try going top down. There isn’t much information to work off of, especially no debugging information especially if it’s supposedly been debugged to the maximum, so that already provides me with an initial disadvantage. Do you even know where the problem is isolated to and when it happens? I am almost confident that it is anything but random. Your code is very deterministic.

The defaultStats table takes up 0.4% of your allowed value size per DataStore key. While this is less than 1%, there’s still the active concern of knowing how far you go until you start pushing to save wherein loss starts occurring. It’s not exactly clear if you hit any maximums or anything.

You have quite a bit of wiggle room with limitations but for the sake of making it clear what you’re doing and divorcing unrelated stats, I highly recommend using the scope part to break off into other DataStores and refining that default portion, unless you’re confident that your data will never surpass limits and you don’t receive errors in that regard, then that’s fine.

For the if statement after the player’s data is defaulted, hopefully you’ve also made sure to review this as well as if everything here is intended. If you want to be more specific about how this if statement is being evaluated, put brackets around operand pairs. If not, then it’ll just follow default evaluation. You might not be getting the results you want there.

Up until there, that portion of the code is for you to debug. I don’t know whether your saving process are fine and these is all moot recommendations where your only problem is supposedly data being randomly lost, or if there’s further problems in your code as well up to how you fetch and apply data. It’s worth checking if data even saves at all and whether or not it pulls properly.

One case I do recommend is that you switch to UpdateAsync from SetAsync and perform a literal transformation against saved data. UpdateAsync is guaranteed to be called multiple times internally until data saves (UpdateAsync queues it), while SetAsync is a one-and-done force data to apply and if it doesn’t, then it doesn’t.

For arrays, please use ipairs, not in next. That’s an old practice because the old VM for whatever ran that faster than ipairs. ipairs is idiomatic and the proper way to iterate through arrays. You have multiple instances where you use in next over ipairs, I’d switch.


All these things aside, they seem more like nitpicks than anything relevant towards solving the problem. That’s because there’s no information to go off of and you don’t have any provide, so actually trying to help you solve this beyond telling you to rewrite it and revise as you go is virtually the only thing that can be done.

Please try debugging further so you can have a better grasp at the problem and better information to provide. We only know as much as you do here without horribly butchering your code to even begin to find out where the problem is to make changes and see if that mitigates or removes the problem altogether.

11 Likes

@BladianMC the purpose of the forum is for people to help regardless of “bounties”, I suggest editing the title.


Anyways, what I said in my reply was my proposed solution.

GetAsync returns nil for inexistent data, so my suggestion was for him to use that as a check possibly, and then use UpdateAsync to update data that already exists.

For a better list of improvements check out @446576656C6F706572 's post above.

2 Likes

Mostly when I suggest going on discord / DMs it’s to prevent spamming on here, some problems require more in-depth conversation, and once solved I always reply with the solution for others to see and to end the thread. Not sure about others, but that’s how I do it. I personally think it’s better to discuss certain things away from the public until progress has been made, instead of flooding a thread with replies.


On the topic of the problem, it might be that you’re going over your DataStore budget resulting in the data not being saved. Your script shouldn’t go over the budget but there’s a good chance you make other DataStore requests in other scripts. Eg for things like leaderboards and codes. As you’re running a simulator, and those are almost always a feature. (Had this issue before.)

Therefore, the problem might be that you’re going over the budget, and whenever a player leaves it can’t and won’t save the data. If a player is new it might result in their data being lost completely due to it never being saved, if a player has played before their data would be “reset” to an older version as the changes are never saved. The best way to get around this is by either optimizing any DataStore requests and minimizing storing as much as possible, or creating a queue system that stores failed save attempts and tries again every x seconds until successful. It’s best to try both options at once.

Another thing to keep in mind is to try to not make duplicate save requests, when the data hasn’t changed you shouldn’t attempt to save it, this will reduce the number of DataStore requests you make as well as reducing possible problems resulting from that.

I might be wrong, but having made simulators before I’ve ran into a few similar issues so it might be worth looking into.


TLDR;
The issue might be related to going over your DataStore budget due to other scripts on top of this one, try to optimize any scripts that work with the DataStore system and queue failed save requests retrying them every x seconds until success. Whilst doing that, prevent save requests when the data hasn’t changed to preserve your DataStore budget and reducing any potential errors resulting from going over it.