Weird behaviour with datastore server limits

Try removing the task.wait(), see if it throttles then. The wait interval could be causing this, I’m guessing the throttling takes place before the requests are sent asynchronously through the network.

I couldn’t find the enum for just SetAsync, but SetIncrementAsync might be more accurate anyway:
image
(from Enum.DataStoreRequestType documentation)

I did originally and the attempts went up to 44013 or something like that. Still didn’t throttle.

I’m pretty sure the throttling is to save any data being used up when roblox realises you are sending too many requests. So the engine should’ve realised that and throttled me at some point.

Try each thread running asynchronously and see if that causes the throttling to happen. Maybe try with a larger data set and something like UpdateAsync() which takes longer to go through.

The data store methods (SetAsync(), UpdateAsync(), etc.) only will add a request to the queue before returning to the main thread. The queue requests are then asynchronously executed one after another in order from the queue, so a larger request may cause the throttling.

So should I add random waits in between?

repeat
local yieldtime = math.random(1, 10)
--Send requests
task.wait(yieldtime)
until
false

No, by running asynchronously I mean run it in coroutines or task thread (coroutine.resume or task.spawn). Although its not truly asynchronous, they replicate asynchronous behaviour and should replicate the effect.

(asynchronously meaning two things running at the same time)

Really? On google it says this " not synchronous; not occurring or existing at the same time or having the same period or phase"

So would you want it like this?

local a = 0
	repeat
		local yieldtime = math.random(1, 10)
		task.spawn(function()
			PlotStore:SetAsync("hi", 123)
		end)
		PlotStore:SetAsync("hi", 123)
		a += 1
		print(a)
	until
	false

Ok this has caused this warning to be sent:

DataStore request was added to queue. If request queue fills, further requests will be dropped. Try sending fewer requests.Key = hi

However, it hasn’t stopped me from continuing to send requests, which is weird because I can fondly remember being stopped once in studio.

Okay second edit now it hasn’t warned me and has given the throttle error after I removed the second setasync method below the task.spawn:

DataStoreService: SetAsyncThrottle: SetAsync request dropped. Request was throttled, but throttled request queue was full. API: SetAsync, Data Store: PlayerData

But why was it that hard just to get it to throttle? Does this mean it would be almost impossible for me to reach these server limits unless I purposely do something like task.spawn??

Synchronous code in programming is code running one after the other. The executor waits for one task to finish before starting the next one. Asynchronous programming allows a task to start whilst the program is still responsive to other code.

These methods I mentioned split each task up into tiny blocks, and executes blocks from each program at the same time, for example:

fetch instruction for thread 1 and decode it
execute instruction for thread 1
fetch instruction for thread 2 and decode it
execute instruction for thread 2
just as a basic idea.

It isn’t actually asynchronous, but it replicates asynchronous behaviour.

also in case you didn’t know async is short for asynchronous





It’s likely that the reason it took so long to throttle is because of the synchronous nature of the code you used before.

Any connections used with :Connect, :Once, etc. are all run in their own seperate thread. This means if two players leave at the same time, the server won’t wait for one player’s data to save before attempting to save the other player’s data (if you save data on player leaving, do if you don’t already :smiley: ).

In a situation like BindToClose, the game’s server shuts down and every player is kicked. This causes a lot of requests to be sent to the store at once, which this code I suggested replicates. That’s why it’s important to ensure the queue doesn’t get filled, especially with crucial and mass data which would be present.

I’d recommend not saving on BindToClose but just yielding so data store requests can finish and data is saved properly. You should also implement your own queue system that is shared by all these threads so that requests are actually sent sequentially and the store isn’t overloaded, resulting in requests being dropped.

Yeah I know how to use datastores properly and what not, I was just testing out the server limits. But does that mean if let’s say I used pcall retry logic incase data hasn’t loaded or saved that I would never hit the limit?

repeat
local success = pcall(function()
datastore:SetAsync(player.UserId, 1000)
until
success

Also I don’t save on BindToClose so yeah.

Also apparently there is a 6s cooldown but I haven’t experienced this at all when doing this

It could be because i’m testing on studio/local server but just want to make sure.

The limit is reached when too many requests are sent (i know it’s obvious and you probably know, but that’s just it).

So, if you sent too many requests in a short period of time (which the retry logic could do), you could hit the limit. That’s why you should wait in-between requests with retry logic.

The example I gave earlier was just of a situation that might cause the queue to get filled.




The data budget is related to the queue, but they are not the same thing. When the data budget runs out, requests following will be throttled and added to the queue, which means they take longer to go through (and is also why you should wait in retry logic). Requests within the budget will take a shorter amount of time to go through, but may still be throttled if too many are sent, which is exactly what the code you used to cause the warning replicates.



huh ive never experienced that either even in my retry logic

So would this be a good idea? Let’s say this was on a player removing event, this would retry over and over until it succeeds but have a 10 second cooldown.

repeat
local success = pcall(function()
datastore:SetAsync(player.UserId, 1000)
task.wait(10) -- Cooldown
until
success

I think a ten second cooldown is a little excessive, I would go for a 2 or 3 second cooldown. Also make sure that it also stops after a certain amount of attemps

local success, result
local attempt = 0

repeat
    success, result = pcall(DataStore.UpdateAsync, DataStore, SaveKey, function(oldData: TypeForData): TypeForData?
        --checks and stuff go here

        return newData
    end)

    --increment attempt and give request time to process
    attempt += 1
    task.wait(2)
until
    success or attempt == 3

But wouldn’t this still lead to data loss if it hasn’t worked after a few attempts?

If the data stores are down for whatever reason and can’t process requests, this will be retrying infinitely, causing more damage than just leaving the data after enough attempts.

So do you mean if there is a backend problem from roblox? Yeah that makes sense.

Also, for loading data would it be a good idea to retry a couple times and if it still fails to teleport the player back into the game?

local attempts = 0
repeat
if attempts == 5 then
TeleportService:Teleport(player, placeid) -- Cant remember exact syntax
end
local success = pcall(function()
datastore:GetAsync(player.UserId, 1000)
attempts += 1
task.wait(3)
until
success

I just kick the player if their data loading fails the first time otherwise it may eat through requests. If you want to teleport them, go ahead.

I was thinking teleporting would be better because players could get annoyed that they have to manually rejoin, also it runs the risk of losing a player when you can keep them in by just teleporting them.

Good idea. It seems we’re drifting a little bit off topic now, so it might be best to leave this here. If you have any more questions or issues, please feel free to DM me, or I’ll likely stumble across your topic about it, should you make one. I’d be happy to help.

If you have any more questions about the server limits, reply to this message and I’ll do my best.

Yeah one more question about the limits, if the experience limit cooldown is 6 seconds, why not wait for 6 seconds before repeating the retry again? And if you haven’t experienced it and neither have I, does this mean that the cooldown simply does not exist?

It’s possible the documentation may be incorrect, but I’m not really sure. I haven’t heard about it until you linked that post.