We will soon be enabling an automatic-retry system to datastore requests.
Things to note:
When the backend throttles a datastore request and returns a retry-response, developers handle this case by attempting the same request again. With this feature, the RobloxClient will handle this case for you.
The property DataStoreService.AutomaticRetry has been added to enable or disable the retry behavior. If enabled, requests are marked to use the retry code path when called. Disabling the property will not affect requests that are currently being processed.
This property will be enabled automatically
While enabled there are a few caveats to take note of:
In order to prevent stale data from being written to the datastore, we have added guards to how SetAsync and IncrementAsync interact with one another. A new SetAsync request on a key will cancel all queued SetAsync and IncrementAsync requests on that same key. Queued requests are requests that have been called but have not been sent to our backend. This can happen when your requests require a retry or you exceed the throttle limit. It will also block you from starting more IncrementAsync requets on that key. Subsequent SetAsync requests can still go through, and when that happens, it will invalidate any SetAsync requests that are currently queued.
If the same request has been rejected by the backend five times, the request will be dropped.
-To disable this feature, you can set the AutomaticRetry property by entering the following in the Studio command line and republishing your game.
Is AutomaticRetry smart enough to only retry when there’s an issue connecting to the DataStore’s server, or does it also retry when the request fails due to a Lua error, because the request threshold has been reached, etc?
Edit: Also, do automatic retries use up additional requests from the request limit? And I assume requests will still yield until all retries have completed?
I would defiantly love this to go live. I actually have a large part of Ultimate Boxing’s back end depending on my own implementation, except I have a more requests to retry for gets because I use only 1 per player when they enter.
Will we have an event or callback to handle when this happens? If data doesn’t get saved, I want to know about it and write specific behaviors for it.
Just to get this straight: This feature only revolves around SetAsync and how it affects other queued calls, while it’s not going to change how UpdateAsync works if your game is not using SetAsync at all?
It’s still a little lost. The way I understand it is that this feature basically makes SetAsync like UpdateAsync without the read?
Its all Datastore requests.
If it fails itll retry up to 5 times before failing.
This involves GetAsync and SetAsync etc etc
if SetAsync and UpdateAsync requests are queued meaning they haven’t been sent to ROBLOX’s backend and you make another request to the same key then your previous request is removed and the new one attempts to go through.
The request will retry when the DataStore servers run into issues such as receiving way too many requests to update the same key (which really shouldn’t be happening in the first place). So, in otherwords, expect retries to occur when the DataStore servers throttle requests.
Lua errors, on the other hand are a little bit different. There are safety measures to queue up some requests that go over the DataStore request limit, but this is just a safety buffer and definitely is not something developers should depend on. Going over the request limit greatly increases the chance for instant failures for requests. But following the request limit guidelines will resolve nearly all of those issues.
The retry feature won’t use up the request limit on the client, but the DataStore servers will still be counting each requests.
Currently my DataStore interface takes a naive approach to handling limits and errors. If a request fails, it retries the next pass. If it fails 10 times, it discards the request as a Lua error. If three separate requests fail consecutively, it assumes a rate limit or temporary (just a hitch) / maybe longer-term (for a few hours) DataStore outage.
As a developer, I would like to explicitly handle limits as you suggested, but with current DataStore design that’s not possible:
There is no documentation on the errors DataStores produce
Error messages can change for any reason at any time, so I can’t hard-code those into the script
I can’t tell how many requests I have remaining
Requests remaining:
I could manually hard-code DataStore limits into my script, but we’re not sure how DataStores work behind the scenes, so we don’t know if we add +1 request every 60/#requestsPerMinute seconds, or alltogether each minute, and even if there’s some clock we have to sync with to keep our request limit tracker accurate. To begin with, hard-coding limits was never a good idea:
We would need to have an API method to get the #requests remaining, as I don’t think the existing API would be backwards-compatible with returning the limits (everything except SetAsync already returns something), unless we maybe added it as a second returned value.
Ability to plan for errors:
The types of errors we get should be small in amount – we should keep the full error, but be able to distinguish from a simplified error set e.g. “RateLimitReached”, “LuaError”, “RobloxCPPError”, “RequestDropped”, “DataStoreOffline”, and maybe a misc/few others. We can’t plan around them if there are so many (undocumented) errors, and errors whose error messages can change for whatever reason.
Not entirely sure if this is what you’re asking for in the “requests remaining” thing, but you can get the number of DataStore requests remaining for all types:
I look forward to seeing how this can further reduce the errors I encounter with data stores!