DataPersistence FAILING

Developers, read 0 and if you’re having the same problem then please post so. I would use DataStore, but there are some things I don’t like about it such as:

  • you can’t have two different value types in a list
  • you can’t iterate through it unless it’s an OrderedDataStore
  • you can’t findFirstChild one (DataStore:findFirstChild(“Dinizterz”)), I know you could just iterate through it but 1. Can only iterate if it’s an Ordered 2. Unnecessary work
  • DataStores can’t store Objects. (The only Object I ever save is a Value Object, I’m not saving maps or anything!)
  1. What issue are you having?
    I’ve had players message me about their level (experience points) being reset to 0. Not only did it turn out to be true, but EVERYTHING that’s supposed to be saved along with their level is lost (reset) too. All the stuff that’s saved are only value objects (BoolValue, StringValue, NumberValue, etc.), all parented under one BoolValue, that BoolValue is saved through DataPersistence, and apparently sometimes when someone loads Data, it doesn’t load!

  2. Does the bug happen 100% of the time?
    No. I’m deeming it as a rare but deadly glitch.
    It’s happened 6 times now. (Your progress being lost is horrible! It should be a major problem even if it happens once!)

  3. If so, are there steps that reproduce the bug?

  4. Where does the bug happen (www, gametest, etc)
    In-Game, not game test

  5. Is it level-specific?
    Probably not since it’s happened in my games, Zombiecon and Assassins. Not sure about other games.

  6. Would a screenshot or video help describe it to someone?

  7. For graphics bugs, it is sometimes helpful to know your system specs, especially graphics card.

  8. When did the bug start happening? If we can tie it to a specific release that helps us figure out what we broke.
    I’m not sure, this has happened 6 times over the course of about 4 months, starting February, but don’t take that for fact!

  9. Anything else that you would want to know about the bug if it were your job to find and fix it.
    What the process of saving and loading during DataPersistence is.
    Do you use github?
    No

do you save as they leave the place?

that’s bad bad bad

[quote] do you save as they leave the place?

that’s bad bad bad [/quote]

That’s actually not bad. I believe the problem he’s having is when players get disconnected(they lose connection to the game for some reason) their stats get reset. I had this problem in Multi Pong, but I switched over to datastore and it fixed everything.

Even then, if the player gets disconnected, why should the data be lost? Unless it works like yanking out a flash drive while copying stuff to it, then I’d understand. But I’d also be confused. Players in my game have disconnected several times but not all those players have had their data lost. Like I said I would use DataStore if it wasn’t a limited, complicated process. Say I’m saving Cash, Points, Level, and Title value for each person. First of all, I’d have to create a new data store for one person since those are different types of values (Number and String), that just seems like a fat waste right there, maybe it’s not actually, but a data store being able to be infinite and having only 4 variable entries is a waste of a DataStore’s ability. It’s like using a Master Ball on a Caterpie. I’d have a DataStore named Dinizterz, if I wanted to get the Title of Dinizterz I’d do “Dinizterz.Level”. Seems simple enough? It would get harder the more creative you get, which is a minus. What if I wanted to add to Dinizterz’ datastore? I may run into an error where I’m assuming something that could be added on is there when it’s not Yet. Also DataStore can’t be accessed by LocalScript, I think. And also, every time I’d access the DataStore, I’d be accessing the DataStore! I’m making a rather large connection every time I request something simple. DataStore is something for leaderboards, not for saving a player’s data.

And also, I have an Autosave feature that saves through DataPersistence every 30 seconds AND the option of saving on leave. People that have lost their data had Autosave on, so the glitch doesn’t happen due to leave, I understand that leaving is riskier and I’m straying from it, but it’s not the only cause.

I’m having a similar problem; players pay for things in my game and then data persistence fails to load. The game subsequently saves when their stats never loaded - thereby resetting them.

See also:

So that’s why stats are reset, Ozzy. Never thought of it that way. Still, it’s a huge problem I find it.

ds = game:GetService(“DataStoreService”):GetDataStore()
util = assert(LoadLibrary(“RbxUtility”))

stats = {Cash, Points, Level, Title} – mock up variables
ds:SetAsync(tostring(player.userId)…“:Stats”, util.EncodeJSON(stats)) – remember to define “player”

Loading:

tableofdata = util.DecodeJSON(ds:GetAsync(tostring(player.userId)…“:Stats”))

This happens to my game a lot too. People just randomly have their stuff erased. :confused:

This happened to me in one of my favorite games recently! I 100%ed the game a while ago… Joined the other day, and my data has been ENTIRELY erased.

I had that happen a lot to my friends game last November, then I converted the script to :Changed () and it solved the issue.

What do you mean converted it to :Changed(), fromlego?

“ds:SetAsync(tostring(player.userId)…”:Stats", util.EncodeJSON(stats))"

You don’t need to do this. Get/Set/UpdateAsync automatically JSON Encode / Decode the value that you’re storing with them. You can just directly use them like so and it will work correctly:

ds:SetAsync('PlayerList$80119', {
Name = 'Stravant';
Id = 80119;
ListOfStuff = { ... };
})

“- you can’t have two different value types in a list”

I’m not sure what you mean by that. You can store whatever data you want in the DataStore, the only restriction is that you can’t have numeric indices and non-numeric keys in a table that you’re storing. (That is, the thing that you store has to be either an array or a map, but it cannot be both at the same time) And if you really need to store a table like that you can always flatten it like so before storing it, and expand it afterwards when reading it:

function flatten(tb)
local flat = {}
for k, v in pairs(tb) do
table.insert(flat, {k, v})
end
return flat
end
function expand(flat)
local tb = {}
for _, dat in pairs(flat) do
tb[dat[1]] = dat[2]
end
return tb
end

“- you can’t iterate through it unless it’s an OrderedDataStore”

This is a hard problem to solve. The issue is that most of the data stores will be too big to dump in a single request. If your database has 5000+ entries in it that’s too many to simply return to you and let you iterate over them, as your code would hang the server while you were trying to digest all of that data, as well as hurting the performance of the database by having it spit out all of that data. And there’s also the issue that you can’t dump the whole DB in a consistent state easily. It may take up to a few minutes to dump out the whole data store if you request that: What should your servers do during that time? Not be allowed to touch the data store? Because if they are allowed, you won’t be able to get out one consistent “snapshot” of the data, you’ll get half of the edits but not the other half that happened during the time that you were reading it.

“- you can’t findFirstChild one (DataStore:findFirstChild(“Dinizterz”)), I know you could just iterate through it but 1. Can only iterate if it’s an Ordered 2. Unnecessary work”

If you want both iteration and searching then you should use a combination of both an OrderedDataStore and a normal DataStore. For instane, you can have your normal data store, and an ordered data store:

Normal: userId -> {
score = ...;
name = ...;
id = ...;
}
Ordered: userId -> score

And when you update the score, update both the ordered data store, and the normal data store entries. That way you can get the scores by name, and also still show the top leaderboard entries by order.

“- DataStores can’t store Objects. (The only Object I ever save is a Value Object, I’m not saving maps or anything!)”"

You shouldn’t store objects. You should serialize out the data from your objects into a table, and store that in the data store. That way you can also reorganize the structure of your game objects in a game without “messing up” the data that’s currently in your data store, because you just have to change the serialization code that figures out how to turn your roblox objects into tables. EG:

function storeObjects(player)
DataStore:UpdateAsync('PlayerList$'..player.userId, function(oldValue)
oldValue = oldValue or {
score = 0;
kills = 0;
}
oldValue.score = oldValue.score + player.Round.Score.Value
oldValue.kills = oldValue.kills + player.Round.Kills.Value
return oldValue
end)
end)

And as for the original post, losing data: I’m almost certain that it’s a problem in your code. We’ve been using the data store for our internal games team game for months now and never seen data randomly get lost. Here’s some things that you can try looking into:

  • Make sure that your value-size that you’re storing isn’t too big. If you’re trying to store 100KB worth of stuff in a single key the data store will get corrupted and will no longer be able to read or write that key, it will always fail to be read, irreversibly. So you should be very careful if you’re storing unbounded lists (lists that may grow arbitrarily long) in the data store within a single key that they do not become too large. If you stay under 10KB worth of key then you should be safe.

  • Make sure that you’re only using SetAsync where you can afford for data to get lost. You should always be using UpdateAsync if the integrity of your data is very important. If multiple servers try to SetAsync on the same key at the same time then data can and will be lost. On the other hand UpdateAsync guarantees that nothing can go wrong with the update.

  • Make sure that you’re very careful with implementing “default” values. If you have a default value for some DataStored value you should be making very sure that there is definitely no way it can accidentally get stored back to a player’s data when they already have a non-default value there, or when they do something like leave the game before their initial loading of data has completed then they’ll lose data. (For instance, if you save on leave, and populate their data on loaded. The load can take an arbitrarily long amount of time until the data store responds with it’s value, if they leave before the loading values get back, and you save the default values that are still there for them at that time, then they’ve lost their data)

  • Make sure that your keys are by ID and not by name. Remember that people can change their names, and you don’t want people to lose data when they do so.

Some good notes on safely using DataStores. Some stuff I never thought about, like ensuring data has loaded before saving anything.

We recommend to using DataStores instead of DataPersistence.

If all you want to save is data per-player, it’s pretty easy to save it as lua table instead of Value Objects. When you get lua table back, you can iterate through it quickly, find first key, etc, etc.

If you store data per-player, this is the preferred way to go (as opposed to iterate through whole DataStore).

[quote]

  • Make sure that you’re very careful with implementing “default” values. If you have a default value for some DataStored value you should be making very sure that there is definitely no way it can accidentally get stored back to a player’s data when they already have a non-default value there, or when they do something like leave the game before their initial loading of data has completed then they’ll lose data. (For instance, if you save on leave, and populate their data on loaded. The load can take an arbitrarily long amount of time until the data store responds with it’s value, if they leave before the loading values get back, and you save the default values that are still there for them at that time, then they’ve lost their data). [/quote]

Great point! I’d never though of that before!

This bug is basically the story of my inbox. (Well, before exploits came back)

[quote]

I’m certain the datastore doesn’t lose data like DataPersistence may be doing, my problem is with DataPersistence not DataStore. I can understand why DataStore is the better option but I’m asking why is DataPersistence not? If it’s faulty, why the heck is it still running?! I find DataPersistence much easier so I’m frustrated to see it go, never had problems with it until it started acting up. I can switch to DataStore, I’m just seeing it as nonsense when DataPersistence is easier and all of sudden stops working.

Of course to all 3, and yeah I figured that’s how data’s being lost, I think a simple line of code would fix that.

Agreed, gotcha. Thanks for your input.

I still wonder why DataPersistence is losing data. I think the answer has been found, maybe all the cases where data has been lost were situations where they left before they loaded causing their data to be replaced when the server saves their unloaded data. The thing is, this may not be the case. In my game, when a player leaves, their data is saved like Player:SaveInstance(“Settings”, Player.Settings), the object named “Settings” is what’s being saved, and is only present in the player if the player has loaded it. So it can’t be the “it saves before it loads” case, unfortunately… I thought this problem was solved

1 Like

Funny enough, I just made this. It could help you out, if there are any bugs let me know (if you do decide to use it).

1 Like