Roblox Http Queue
A library to queue Roblox HTTP Requests for all your different external services.
Current Version: 1.0.0
Disclaimer: This project is currently in a Release Candidate status. This means it should work, but is not completely stable yet. Make sure to report me bugs!
Writing code to make requests is simple, and maybe fun. Writing code that gracefully handles everything that can go wrong in a request… Well, that’s a boring thing to do.
This library is intended to help easing this by, in particular, handling servers that impose rate limits. Writing code to handle that and make sure every request we make is accepted* by the server and is not lost.
This project is powered by evaera’s Promise implementation and Osyris’ t typechecking library.
The library is available at GitHub and you can use it according to the terms of the MIT license.
* For accepted I mean “not rate-limited”. I cannot make guarantees that the service will not refuse to process the request due to, for example, invalid tokens or permissions.
Installation
Just grab the .rbxmx file from the releases page and drop into your project - as simple as that!
roblox-ts
You can use this library on your roblox-ts project via the @rbxts/http-queue package.
Documentation
The library already features built-in API documentation that can be read using ProbableAI’s Documentation Reader plugin. Happy coding!
Usage
Require the module:
local Http = require(game:GetService("ServerScriptService").HttpQueue)
We can start by doing a simple task:
Create a request and send it - this is pretty much the same behavior of using HttpService directly:
local request = Http.HttpRequest.new("https://some.website.com/", "GET", nil, {auth = "im very cool", cool = true})
-- Actual Request URL is https://some.website.com/?auth=im very cool&cool=true
-- The :Send() method returns a Promise that resolves to a response!
request:Send():andThen(function(response)
print(response.Body)
end):catch(function(err)
print("ERROR!", err)
end)
-- Do some work while we wait for the response to arrive
-- If you want to yield the script until the response arrives
local response = request:AwaitSend()
This is cool and all, but we can make this more interesting. Let’s say you want to use Trello in your application. Unfortunately, the rate limiting of Trello is very tight (10 requests per 10 seconds per token for Roblox clients), but you also want to send 1000 requests without dealing with 429’s. How do we do it?
Instead of worrying about it yourself, you can delegate the responsibility of dealing with the rate limits to a queue.
local TrelloQueue = Http.HttpQueue.new({
retryAfter = {cooldown = 10} -- If rate limited, retry in 10 seconds
maxSimultaneousSendOperations = 10 -- Don't send more than 10 requests at a time: This value shouldn't be higher than the rate limit itself
})
-- Let's change the name to a Trello board, 1000 times (don't do this at home!)
for i = 1, 1000 do
local request = Http.HttpRequest.new("https://api.trello.com/1/boards/5d6f8ec6764c2112a27e3d12", "PUT", nil, {
key = "Your developer key",
token = "Your developer token",
name = "Your board's new name (" .. tostring(i) ..")"
}))
TrelloQueue:Push(request):andThen(function(response)
-- This will never print "429 Too Many Requests"
print(response.StatusMessage)
end)
end
-- Do some work while we wait for the response to arrive
-- If you want to yield the script until the response comes in:
local response = TrelloQueue:AwaitPush(request)
When pushed to a queue, requests that are throttled are tried until accepted, and then the promise resolves.
The queue works on a “first come, first serve” basis. This means that requests being pushed first will be dealt with first by the queue. (HOWEVER, this doesn’t mean the responses will arrive in order!)
You can override that behavior by passing a priority parameter to the :Push() or :AwaitPush() methods. There are three options available:
HttpRequestPriority.Normal - the default priority. The request is pushed to the back of the regular queue.
HttpRequestPriority.Prioritary - The request is pushed to the back of the prioritary queue, that is done by the queue runner before the regular queue.
HttpRequestPriority.First - The request is pushed to the front of the prioritary queue.
NOTE: The priority features should be used sparingly.
Example:
TrelloQueue:Push(request, Http.HttpRequestPriority.Prioritary)
Type Guards
This library also comes with type guard functions that allow you to check whether a value is actually what you want:
isHttpRequest(value)
isHttpRequestPriority(value)
isHttpResponse(value)
isHttpQueue(value)
These return true if the value is a and false otherwise. They might be handful if you’re writing your own library.
And that’s it! Happy coding!
Props to @VoidedBIade for literally helping me fix some issues while developing this!
Changelog
Changelog
| Version | Changes |
|---|---|
| 1.0.0-rc.1 | Initial pre-release |
| 1.0.0-rc.2 | Fixed roblox-ts typings; updated to latest Promise implementation |
| 1.0.0-rc.3 | The queue is now also aware of throttling from the HttpService itself. URL’s that do not have a “/” in the end will automatically have one appended (this is important if you also have query options) |
| 1.0.0 | Fixed Url normalization. The “/” at the end will only be appended when an Url path does not exist already. |
| 1.1.0 | Now you can determine the cooldown period of a queue using a callback, which allows you to manually inspect the response. (Useful if the service you’re using has a very specific way of determining the cooldown period) |
