Hi, I’m currently learning data stores (coz I never really focused on them too much), my data store is based entirely on a data table. It’s mostly done but there’s something that’s bothering me
Event.Event:Connect(function(Player)
local Data = {}
local DataFolder = Data_Parent:FindFirstChild(Player.UserId.." data")
for _,v in pairs(DataFolder:GetChildren()) do
Data[v.Name] = v.Value
end
for _,v in pairs(Data) do
warn(v.. " VALUE")
warn(_.." INDEX")
end
DataDS:SetAsync(Player.UserId.." data",Data)
end)
this part of the script saves EVERYTHING, what if I only want to update a single value in the datastore? Do I have to use UpdateAsync and if yes how do I do it? Should I even bother in changing it or should I just make it save all of the values at once?
The UpdateAsync Method is used for ensuring atomic operations, however, since that is data is only for one player, and no player is in multiple game instances there is little-to-no need to atomic Operations.
By using UpdateAsync, it performs two operations, a Get and a Set operation, so it makes no performance increases by using an Update operation.
Personally, I would recommend using ProfileService, which is used by a very large number of games, you can see more about it here: ProfileService
It Provides utilities such as Session Locking, that prevents mutliple Keys being open at the same time.
Creating your own Data System is definitely a good idea, It helps you understand how systems such as ProfileService works, and I would highly recommend making something similar (though less complex) as this will be very beneficial for anyone. I personally (though this was before ProfileService existed) created my own system, before looking at trying to use module such as DataStore2. And that was very beneficial since I now now the quirks of Datastores and why ProfileServices does things in that way.
Even if you don’t use your own data system in a Released Game, it is a very good idea to make one, just to help with your own understanding.
I’ve written my own data store layer and simply put, you should just design your system to write the whole table every time.
To solve the problem of data becoming over-large and / or hitting limits, you’re data model should be encapsulated into different components that use different data stores i.e. use as many stores as you need to make sure any one table is not that big and hence writing it all every time is no problem.
You don’t have to but it is a good idea - it prevents making mistakes that could crop up when updating the data store from multiple servers at once. Or just handling random failures. It also creates versions for you automatically within the data store which could be very handy if you ever ran into a situation where you needed to access for old data for any reason (major bug with player-purchased items, just plain old auditing, etc.)
Using it is pretty straightforward on a basic level:
local setSuccess, errorMessage = pcall(function()
store:UpdateAsync(key, function(curVal: any, keyInfo: DataStoreKeyInfo)
return val
end)
end)
if not setSuccess then
-- A ROBUST SOLUTOIN WOULD NEED TO RETRY
warn('CachedDs - persistToDS ERROR '..errorMessage)
end
As mentioned above, my solution writes the whole table. But I also make sure my tables aren’t large and I don’t write them more than once / ~10 seconds (using an in-memory cache).
Can you expand on why this is a good Idea? Datastores have a limit in size, and as long as you are not getting close to this limit, why would you want to split it up? All it really does is increase the number of requests that have to be sent, filling up your DataStore quota. We are in the times that the speed of getting data from the database over the network is negligible.
SetAsync will still create new versions so that should not be a reason to use UpdateAsync (though it is still useful)
This is inherently unnecessary since the UpdateAsync does two operations, a Get and a Set, and by just returning the value from further up in the script. This is bad since you are performing an unnecessary Get request which will fill up your DataStore quota.
Sure… expanding:
My data store layer is setup to cache data changes locally and only update on a set interval that is based on the number of stores the server is updating. So hitting quotas isn’t an issue - not hitting quotas is handled by a different policy layer.
Second, keeping data store entries small-ish:
makes managing the data (migrations, reading for debugging, etc.) much easier.
means they caches take less memory
allows for treating data stores like tables in a relational database so rationalizing about the data architecture is easier and enables better encapsulation
Third, networks may be fast, but they are still a performance consideration. Along with the other benefits listed above, why not?
Do you have a reference for this assertion? It was my understanding this was not true. And in testing, I’ve not seen versions created when using just set.
A couple assertions here… let’s take them one at a time.
This is inherently unnecessary since the UpdateAsync does two operations, a Get and a Set, and by just returning the value from further up in the script.
I’m not following this - yes, UpdateAsync does a get and a set but what is “inherently unnecessary” because of that? I’m not really following what you’re saying.
This is bad since you are performing an unnecessary Get request which will fill up your DataStore quota.
No, it isn’t “bad” because there is no other way to make sure a value is updated correctly given multiple asynchronous updates across multiple servers - see above. And the quota issue, again, should be handled by a different policy layer
I’m confused - I thought you just said it was bad?
Maybe… as I understand it, using update is the only way to get automatic versioning. If I’m right, then that can be a good reason. Regardless, the function was a template to show how to frame up the call - any number of policies could be applied depending on what the dev is trying to accomplish and what the constraints of the update for a particular store are.
