I’m really paranoid that UpdateAsync will behave differently to what everyone has been saying about UpdateAsync preventing data loss and whatnot if the DataStore service itself is down?
Does anyone have experience with using UpdateAsync during such outages? I know that UpdateAsync handles both getting and setting data internally but will it throw nil and override data even if I throw it inside a pcall? Is there any risk for such a thing to occur at all?
I’d like to save my data like so and I need you to tell me if this is will lead to data loss if these services are down (this is a very primitive example of saving data with UpdateAsync and I know it’s missing things like BindToClose and other functions bound to core game functionality where UpdateAsync will be later be utilized):
local PS = game:GetService("Players")
local DS = game:GetService("DataStoreService")
local ORD = DS:GetOrderedDataStore("MainGameDataStore")
local function parallelAsync(key)
local playerData = 0
local success, err = pcall(function()
playerData = ORD:UpdateAsync(key, function(oldData)
if not oldData then
-- Handle first time the player joined, play some fancy GUI or tutorial.
end
local newValue = oldData or 0
newValue += 1
return newValue
end)
end)
if success then
return playerData
else
-- DataStore service is down.
warn(err)
end
end
local function loadData(player)
local key = string.format("Player_%d", player.UserId)
local userData = parallelAsync(key)
if userData then
-- Do whatever
print(userData)
else
-- Handle outage here... notify players... prompt GUI... whatever.
end
end
local function onPlayerAdded(player)
loadData(player)
end
for _, player in ipairs(PS:GetPlayers()) do
task.spawn(onPlayerAdded, player)
end
PS.PlayerAdded:Connect(onPlayerAdded)
This is unfortunately still affecting my development workflow and I urgently need an answer to this. Does anyone out there know anything regarding this?
UpdateAsync doesn’t guarrantee data loss prevention. It doesn’t solve all your problems. The only thing UpdateAsync is good for is for considering the previous saved value before saving a new value.
This isn’t what I’m asking, I know all that. I’m asking to know how it will behave when the datastore service goes down and whether wrapping it in pcall will save the data from being overridden in those situations. I need an answer from someone who’s got hands on experience. Please read the thread before commenting, thank you.
If the DataStoreService is down, then, well, it simply won’t work, it needs to fetch the data from the service first, and then save new data over it, if the service is unavailable, it won’t work and data will be lost
I suggest you implement a system to counter this
Assuming you already have autosaving setup, implement a failcounter into it, if autosaving fails too often, it’s safe to assume the service is down, you can then let players know that the service is down, the point of this is not to prevent data loss, but minimize it, however, if, you’re willing to get into HttpService, you can setup your own backup datasaving system, and have your game switch to this to save data, I suggest you lookup php and MySQL if you’re willing to do this.
It should be noted, however, that Roblox is aware of how important the DataStoreService is, outtages are extremely uncommon, and Roblox will put the site into maintainence mode when such an issue does occur, preventing anyone from accessing it, so overall DataStoreService is pretty reliable
UpdateAsync has no extra benefit over SetAsync when the datastore server goes down if anything its worse as it has to do two datastore calls instead of one
the only benefit of using UpdateAsync is when two servers are trying to access the same key at the same time by using UpdateAsync it ensures the second server that calls UpdateAsync is using the latest data
in the example you gave above your using UpdateAsync to increment the value by 1 and that’s fine
now your using UpdateAsync for no reason because your not taking the old data into consideration so you would be better of using SetAsync like this and not waste a get request
ORD:SetAsync(key, newPlayerData)
so what can you do when the datastore goes down
short answer nothing when the datastore goes down there is nothing you can do other then wait for the datastore to come back up
this is why its good to save the player progress every lets say 30 seconds then if the datastore does go down then they will lose at most 30 seconds of gameplay
Using SetAsync and GetAsync would ofcourse solve all my issues but I do not want to use them EVER because of race conditions and them not being able to session lock.
Using them would ofcourse be super easy because I could always immediately validate the service not working by wrapping both in pcalls and notify the player that the services are down.
UpdateAsync on the other hand is something I will always stick to because it will re-poll the latest data from across all experiences ensuring no data loss in that regard whereas if you were to use SetAsync and GetAsync there is a chance of returning older data and something people abuse a lot in trading.
However, I am mainly worried that the datastore service becomes unstable in a situation where:
UpdateAsync polls for the latest data and returns nil since the service goes down but within that timespan inside the UpdateAsync thread the service for setting data starts to work again. Would this still raise the pcall flag? No one still has explained how UpdateAsync behaves in this situation:
I’ve looked into this multiple times throughout the years but I’ve found that doing this is not realistic. You’d need a scalable K8 cluster to loadbalance all the API requests as the game gets more popular which is something I’m not willing to spend my money on. Currently a single Django server with a REST API can handle about 20 or 30 requests per second at least on the cloud servers I tried on Linode which is far too low for a Roblox server.
The question has originally been answered. Roblox DataStore methods will not work when the service is down since they are Http Requests after all. You can also clear your paranoia by using ProfileService, which utilizes UpdateAsync properly and is a very good DataStore resource.
ProfileService is not meant to be used with Leaderboards though which is something I definitely want to use since they do not use OrderedDataStores. Loleris himself says not to use it for that purpose.
Yes but you got to understand that UpdateAsync does 2 requests in 1. It gets the data and then it proceeds to save it. Since it behaves this way I need to know if the pcall flag will immediately be called if the service is down in the get request and returned out of the thread or will it continue to call set too and only then warn me about it?
I guess I will have to put faith into Roblox that the service is smart enough to not still call the callback function when getting the data returns a service error. Not really a way to validate this either but I’ll solve this thread. Thanks for everyone participating.
The first time you run it, it will print out “CALLED”. When you run it with no internet, nothing gets printed, so it is safe to assume it does not call the function again.
Also, how often is it that a Roblox server goes down due to internet loss? And how often does that happen during an UpdateAsync?
I considered testing it like that but I thought Roblox prevented you from using Studio without an internet connection so I didn’t even bother testing it.
If it truly does not print anything it means it would work just the way I’d want it to.
You never mentioned the use of Leaderboards. Furthermore, leaderboards (OrderedDataStores / MemoryStores) should be based on a primary DataStore, not alone on itself.
If a UpdateAsync call was happening while there was an outage (which is likely impossible, since the code inside UpdateAsync CANNOT YIELD), then the Roblox Server wouldn’t receive any response and wouldn’t update anything in the key.
What I meant is:
-Get() process is successful because Roblox servers are up
-Roblox servers are down
-Update() process will not be able to communicate because Roblox servers are down. This would mean that no response would be received from a Roblox Server.
Did not think it would be relevant to the issue. I’ve vaguely heard of that before, can you link me to the sources that explain the best practices when it comes to:
Well, if you have your player datastore, for example, and there’s a key named “Points”, you should base your ordered datastore on that. Never rely on OrderedDataStores for actual data saving, as their only purpose is ordering data for it to be shown.
There’s also no actual reason to be using UpdateAsync on OrderedDataStores if you already utilize UpdateAsync methods on the primary DataStore, unless you want to bypass GetAsync() 4 second cache
I tried to extract data from a regular datastore with an ordereddatastore once before but it didn’t seem to work. They act as their own separate datastores to me, I’d have to save the same data into both. It’s just really frustrating since Roblox does not yet provide proper querying tools but it’s luckily further down the Roblox roadmap…
How would I use both together then? If you’re willing to, could you provide a thread or even a snippet of code? I’d appreciate it a lot.
