Datastores randomly failing to save data, and I don't know why

why are you saving the data each time the player dies?

To update the datastore when the player dies and drops all of their armor, for example. In a separate script, when the .Died function fires, it takes all of the player’s inventory items (such as armor) and puts them into a loot bag on the ground. So, when the character is then removed after they die, the datastore is updated to reflect they no longer have those items.

2 Likes

Try adding a task.wait(5) at the bottom of the BindToClose function.

2 Likes

Would this delay the game actually closing for a bit to perhaps allow additional time to process datastore requests, if that is the issue?

3 Likes

Isn’t actually true, BindToClose runs when the game is about to shutdown, source — documentation. So it seems to be clear.

2 Likes

I didn’t understand why’s here is a repeat-unit? It’s useless because you have log right after that, and log will never work because you set loop to make attempts to save data until it will not be saved. So for the first one repeat-until, it’s very uncommon loop (Better use while-do), for the second one it’s useless because of reason provided before. But actually to make not reinvent the wheel again, you should use Datastore2, it’s much more simplier and faster in-use.

2 Likes

Are you referring to:

	local success, err
	local ArmorData

	repeat
		success, err = pcall(function()
			ArmorData = myDatastore:GetAsync(Player.Name)
		end)
	until success

If so, then mind you this is my first real excursion into datastores and this repeat until success loop is used on the documentation.

Also, I had someone tell me that the Datastore2 you linked was obsolete because of updates ROBLOX made to Datastores sometime in 2019 or 2020 or something like that, and it was better to use default systems now. Is that not true?

2 Likes

I think this might be the issue. Just try removing this statement and see if it saves - SetAsync() overwrites the data anyway.

Also, I see that you are using a repeat loop to save data until success. This not only sends an absolute TON of requests if data saving fails, but can run infinitely if data stores are down, or just generally run into a continuous error. Add an attempt limit to this.

2 Likes

Unrelated to Saving Failures (or potentially not)

I noticed a few of caveats with your current system.

Rate Limiting (semi-important with new Datastore limits):
You could encounter Rate-Limiting with all the request queues you are sending, which can potentially worsen with bigger servers.

Saving with Player.Name
You should be saving with Player.UserId, due to many reasons. Security, and ROBLOX name-changing system (ya-know, that costs 1000 robux).

You are also using :RemoveAsync() for no reason? There is no improvement from doing this, you are just adding more requests.

2 Likes

Edit:
Additionally with Rate-Limiting, you are Protected Calling your DataStore requests, which is great! But it doesn’t seem you are printing/warning in console to see if there was dropped Queues or errors.

I was wrong, sorry.

2 Likes

I can’t remember why I decided to clear the data before setting it again, but I think I had issues (potentially related) that caused the old data to fail to erase, for example on player death. But I suppose this could be fixed by firing myDatastore:RemoveAsync(Player.Name) on Humanoid.Died, potentially, if I end up needing it at all, which currently doesn’t make sense why I even have it in the first place thinking about it.

I was just going off of examples used in the documentation, as linked previously. What you said makes sense, but what would be a better way to do it? OR should I just drop the repeat until loop in general? What if there is an issue with rate limiting and it drops the request instead of adding it to a queue (as I assume a repeat until loop is best suited for a situation such as this)?

2 Likes

You are right, and I did previously run into a ratelimit issue with a different datastore system (unrelated to this, but same game). But, what would be a better way to handle this? Also, see my post above this - same question, basically.

You’re right. The only reason I wasn’t using Player.UserId was for testing and easily being able to access players datastores via their usernames. An easy thing to fix nonetheless.

2 Likes

That’s incorrect, remove the pcall (if no data then it will return nil and not throw an error), the repeat also needs to be removed

1 Like

True, saving data with the players username is basically erasing the players data when they change their username and keeping useless pieces of data that will never be used

2 Likes

But does the pcall / loop not serve a purpose? Just genuinely confused why people are suggesting this.

2 Likes

I am the co-owner of a game called Castaway. I personally wrote all of the code in the game. We save A LOT of data, including up to 2000+ placed buildings and decorations per player. I have never lost player data in the few years the game has been available to the public.

If you are still having problems with reliably saving player’s data, I would be happy to share with you exactly how my save system reliably saves every player’s data and would love to answer any questions you have.

4 Likes

add a task.wait(number) to your repeat statement, you may be burning out calling the datastore and hanging

2 Likes

Per @TomPlays63’s request, I will go ahead and explain what I’ve done with my game to save and load player data reliably. It’s important to mention that there are a few primary ways player data can be lost.

  • The player’s data fails to load but the player is treated as a new player, therefore all of their data will be overwritten upon the first save.
  • The player’s data is overwritten after the player joins, before it finishes loading.
  • The player’s data fails to save but the script does not keep attempting to save their data.
  • The game closes without saving each player’s data.
  • The player leaves without their data being saved.
  • The player’s data is being saved too often, and therefore fails to save.
  • Their is a logic or script error in the saving code that is preventing the code from executing.

There are solutions to all of these problems. Some of these have already been addressed in this post. For example, some have mentioned that your pcall function is endlessly attempting to save data. You should be setting a maximum limit of attempts on each save, I use 3. In the rare case that a save fails 3 times, do not fret; use intermittent Auto Saves to protect player data (probably once every minute), as well as PlayerRemoving and BindToClose events.

Personally, I don’t use auto saves anymore because I trigger a save anytime any data changes (e.g. the player picks up some resources, or spends some, or buildings something, or kills a mob, or kills a player, etc.). This might sound like I am saving too often, however I simply added a 40 second timer to each save type that couples any further requests within those 40 seconds so that there is a maximum of one save per 40 seconds. However, there is nothing wrong with auto saves; I just found this an elegant solution since it will not waste any resources saving data when nothing has changed.

You must save on PlayerRemoving and BindToClose as the last line of data loss prevention. If you aren’t using this, you are at a major risk for losing player data. I am pretty sure that CharacterRemoving is not triggered when a Player leaves, but in any case you shouldn’t be saving every time a character dies or you will hit throttling issues for any player who is resetting or running straight into combat.

Imagine a player rapidly leaves the game and then joins another server. If their data doesn’t save immediately, then it will be overwritten. There are additional safeguards for protecting data, like using UpdateAsync so that data is updated according to the supplied function rather than being completely overwritten by SetAsync with no prevention. BindToClose is the only way to ensure that the server does not close before your save code finishes executing. The moment the BindToClose function is done running, the game will finalize closing; but keep in mind that if you do not yield the function then it will not wait at all. This is a problem if you are saving player data in a coroutine or task.spawn function. You must add some kind of yield until player data completes saving.

On the topic of overwriting data, if a player’s data fails to load when they join then they should immediately be prevented from data being saved during the session. You can add a way for them to try reloading data, or kick them from the place, or allow them to play without data saving. Also if a player’s data has not finished loading, then saving should not be possible until it has. Additionally, anything being saved must be in a folder or location that will not be removed when the player leaves; I use several folders created on runtime which get moved into ServerStorage after a player leaves, then all of the information is saved and only then is their folder(s) deleted.

Another concern that I will mention, is data store limits on characters. Most people don’t need to worry about this but if you are saving a lot of data, or very long strings, then you should be mindful that each DataStore can hold up to 4million characters. I save a lot of buildings per player, which needs to save their position, level, workers assigned at them, resources stored in them, etc. and all of this must be as compact as possible since the player can build well over a thousand units. A lot of things can be condensed really far, for example I save the X and Z position in object space of the player’s island, and I recalculate the Y position on runtime since it is always the same based on the ground beneath the unit. I save the rotation of the building as a single digit integer which when multiplied by 90 represents the 90 degree turn of the building. These are just some examples of how you can save only the necessary information.

One problem I see with your code is that you are saving EVERY player’s data under one data store. If your game ever exceeds a few hundred thousand players, you are going to hit limits for data. I save each player’s data under their own DataStore which is referenced by the name of the data store and their user id, I use multiple data stores per player so that I don’t have to save ALL of their data every time one stat changes and so that I don’t have to worry about limits for the build system being further hindered by every other system in the game. So if they build something new, I update their building save. If they earn some resources, I update their stats. If they buy something with robux, I update their purchases DataStore. I do the same thing for tools, and for quests.

You should ALWAYS save using the player’s id, not their name. As other’s mentioned, if they ever change their Roblox username then they will lose their data in your game.

PS: Don’t ever use RemoveAsync unless your intention is to actually delete the player’s data (e.g. they wipe their file). SetAsync already overwrites the data by default, and UpdateAsync does an even better job by allowing you to compare the old data and update it with the new data which if used correctly makes data extremely reliable to save.

I hope I addressed everything here, I was trying to rush this because I knew it would take me days to get around to this if I didn’t do it tonight haha. Let me know if this was useful information and if there is anything else you would like to know or if you have any questions about what I’ve explained.

Edit: Btw when you are using pcalls, you must print the error message; it will not print to the output automatically. I see that you DID add a print for this, but you put it after an infinitely repeating loop of attempts (which means even if it did fail, it isn’t telling you what happened until it succeeds; at which point there is nothing to report that happened).

1 Like

Would recommend just using a module like ProfileService, or datastore2, suphi’s datastore module or a combination of those. Takes a lot of the headache away from datastores so you can focus on actually important features

I personally use ProfileService. It’s very simple to implement, and I’ve had 0 issues with over 100m visits since I began using it - would recommend

If you want you of course can write your own datastores, but datastore2 much easier in use. You was wonder why datastores isn’t working and you doesn’t see any errors. The reason why you don’t see any logs, is because of repeat-until loop. It repeats until succes is false. And then you do check if succes then (code) else do warn. But this warn will never shown.

1 Like