How do I slow down http requests?

Help and Feedback Scripting Support
, ,
#1

I need to slow down the number of HTTP requests I am requesting to 5/second, though I am unsure how to do this. I am trying to get every game on a favorites page but I get rate limited, so I need to slow down the requests to avoid this.

This is the code that makes the request, I assume I utilize async or task.wait somehow but I have no idea how.

local BaseUrl = "https://www.roproxy.com/users/favorites/list-json?assetTypeId=9&cursor=1&itemsPerPage=20&pageNumber=2&userId=7597220565"
local data = HttpService:JSONDecode(HttpService:GetAsync(BaseUrl))

I created a queue, though I do not know how to use it.

local UrlQueue = HttpService.HttpQueue.new({
	retryAfter = {cooldown = 5},
	maxSimultaneousSendOperations = 5
})
#2

I am assuming you’re using this tutorial, which seems to have the solution you’re looking for. You should probably mention that you’re using auxiliary resources in future posts.

You can do this fairly easily even without the use of a third-party library. The gist is that you’ll track the number of requests made per second. If that number exceeds five, you have two options; you can either:

  • ignore any additional requests until a second has passed
  • queue any additional requests made during that second and try to send them in the next available second

The first option is simple; simply do not make a request if 5 or more have been made in the duration of a second. The second option is a bit tricker, but you can just store those requests in a queue and try to send the oldest requests first once a new second has begun.

It would be beneficial to handle this in one script. The pseudo-code (which I should note, is not written to developer standards and is missing some important things not relevant to the question you’d need to handle) would probably look like this:

local HttpService = game:GetService("HttpService")
local RequestsThisSecond = 0

-- It is more beneficial to use coroutine.create/resume here instead of task.spawn, 
-- but again, this is an implementation detail left for the reader
task.spawn(function()
    -- Alternatively, instead of a permanent loop, you could signal when a 
    -- request has been made and start counting from that moment instead
    while task.wait(1) do
        RequestsThisSecond = 0
    end
end)

-- Handle requests
local function SendRequest(url, ...)
    if RequestsThisSecond > 5 then
        -- You could either store requests in a queue here, 
        -- or toss any additional ones made in that second
        return
    end
    RequestsThisSecond += 1
    local response = HttpService:GetAsync(url)
    local data = HttpService:JSONDecode(response)
    if data.message == "success" then
        return data
    end
end
1 Like