Datastores randomly failing to save data, and I don't know why

Per @TomPlays63’s request, I will go ahead and explain what I’ve done with my game to save and load player data reliably. It’s important to mention that there are a few primary ways player data can be lost.

  • The player’s data fails to load but the player is treated as a new player, therefore all of their data will be overwritten upon the first save.
  • The player’s data is overwritten after the player joins, before it finishes loading.
  • The player’s data fails to save but the script does not keep attempting to save their data.
  • The game closes without saving each player’s data.
  • The player leaves without their data being saved.
  • The player’s data is being saved too often, and therefore fails to save.
  • Their is a logic or script error in the saving code that is preventing the code from executing.

There are solutions to all of these problems. Some of these have already been addressed in this post. For example, some have mentioned that your pcall function is endlessly attempting to save data. You should be setting a maximum limit of attempts on each save, I use 3. In the rare case that a save fails 3 times, do not fret; use intermittent Auto Saves to protect player data (probably once every minute), as well as PlayerRemoving and BindToClose events.

Personally, I don’t use auto saves anymore because I trigger a save anytime any data changes (e.g. the player picks up some resources, or spends some, or buildings something, or kills a mob, or kills a player, etc.). This might sound like I am saving too often, however I simply added a 40 second timer to each save type that couples any further requests within those 40 seconds so that there is a maximum of one save per 40 seconds. However, there is nothing wrong with auto saves; I just found this an elegant solution since it will not waste any resources saving data when nothing has changed.

You must save on PlayerRemoving and BindToClose as the last line of data loss prevention. If you aren’t using this, you are at a major risk for losing player data. I am pretty sure that CharacterRemoving is not triggered when a Player leaves, but in any case you shouldn’t be saving every time a character dies or you will hit throttling issues for any player who is resetting or running straight into combat.

Imagine a player rapidly leaves the game and then joins another server. If their data doesn’t save immediately, then it will be overwritten. There are additional safeguards for protecting data, like using UpdateAsync so that data is updated according to the supplied function rather than being completely overwritten by SetAsync with no prevention. BindToClose is the only way to ensure that the server does not close before your save code finishes executing. The moment the BindToClose function is done running, the game will finalize closing; but keep in mind that if you do not yield the function then it will not wait at all. This is a problem if you are saving player data in a coroutine or task.spawn function. You must add some kind of yield until player data completes saving.

On the topic of overwriting data, if a player’s data fails to load when they join then they should immediately be prevented from data being saved during the session. You can add a way for them to try reloading data, or kick them from the place, or allow them to play without data saving. Also if a player’s data has not finished loading, then saving should not be possible until it has. Additionally, anything being saved must be in a folder or location that will not be removed when the player leaves; I use several folders created on runtime which get moved into ServerStorage after a player leaves, then all of the information is saved and only then is their folder(s) deleted.

Another concern that I will mention, is data store limits on characters. Most people don’t need to worry about this but if you are saving a lot of data, or very long strings, then you should be mindful that each DataStore can hold up to 4million characters. I save a lot of buildings per player, which needs to save their position, level, workers assigned at them, resources stored in them, etc. and all of this must be as compact as possible since the player can build well over a thousand units. A lot of things can be condensed really far, for example I save the X and Z position in object space of the player’s island, and I recalculate the Y position on runtime since it is always the same based on the ground beneath the unit. I save the rotation of the building as a single digit integer which when multiplied by 90 represents the 90 degree turn of the building. These are just some examples of how you can save only the necessary information.

One problem I see with your code is that you are saving EVERY player’s data under one data store. If your game ever exceeds a few hundred thousand players, you are going to hit limits for data. I save each player’s data under their own DataStore which is referenced by the name of the data store and their user id, I use multiple data stores per player so that I don’t have to save ALL of their data every time one stat changes and so that I don’t have to worry about limits for the build system being further hindered by every other system in the game. So if they build something new, I update their building save. If they earn some resources, I update their stats. If they buy something with robux, I update their purchases DataStore. I do the same thing for tools, and for quests.

You should ALWAYS save using the player’s id, not their name. As other’s mentioned, if they ever change their Roblox username then they will lose their data in your game.

PS: Don’t ever use RemoveAsync unless your intention is to actually delete the player’s data (e.g. they wipe their file). SetAsync already overwrites the data by default, and UpdateAsync does an even better job by allowing you to compare the old data and update it with the new data which if used correctly makes data extremely reliable to save.

I hope I addressed everything here, I was trying to rush this because I knew it would take me days to get around to this if I didn’t do it tonight haha. Let me know if this was useful information and if there is anything else you would like to know or if you have any questions about what I’ve explained.

Edit: Btw when you are using pcalls, you must print the error message; it will not print to the output automatically. I see that you DID add a print for this, but you put it after an infinitely repeating loop of attempts (which means even if it did fail, it isn’t telling you what happened until it succeeds; at which point there is nothing to report that happened).

2 Likes

Would recommend just using a module like ProfileService, or datastore2, suphi’s datastore module or a combination of those. Takes a lot of the headache away from datastores so you can focus on actually important features

I personally use ProfileService. It’s very simple to implement, and I’ve had 0 issues with over 100m visits since I began using it - would recommend

If you want you of course can write your own datastores, but datastore2 much easier in use. You was wonder why datastores isn’t working and you doesn’t see any errors. The reason why you don’t see any logs, is because of repeat-until loop. It repeats until succes is false. And then you do check if succes then (code) else do warn. But this warn will never shown.

1 Like