Fixing HttpService: Make PUT, PATCH, DELETE requests, access response headers, proxy to any site, and more

ProxyService

Roblox’s HttpService has always been severely lacking. This open-source project essentially aims to serve as a replacement, providing an Http client that opens the door to actually using REST APIs, reading response headers, reading status codes, accessing roblox.com from in-game, and more.

Example uses are accessing Roblox, Discord, Trello, and Firebase APIs, including crazy stuff like logging into a Roblox account from in Roblox. You can use this for virtually any API.

The way it works is that the game makes a request to a proxy server instead of the server you want to access and the proxy server actually sends the request for you. It returns an HTTP 200 so Roblox does not error and then appends response headers/codes to the response. This is all done in the background with a free, open-source, personal server that you can setup very easily.

Features

This makes Roblox Http requests more complete by adding support for the following features:

  • Make PUT, PATCH, and DELETE requests in addition to GET and POST requests (with that, be able to use REST APIs from Roblox).
  • Read response headers.
  • Read the status code and status message.
  • Modify the User-Agent header (usually not allowed by Roblox).
  • Read response even if the response contains a 400- or 500- status code (this includes body, headers, and the status code and message).
  • Access sites usually unavailable, including roblox.com APIs as well as discord webhooks (and being able to view headers means you will be able to obey discord rate limits, which means this proxy is allowed).

Server Setup Tutorial

  • Create a heroku account here: https://signup.heroku.com. Make sure to verify your email and set a password. If you already have a heroku account, log into it.
  • Click this button

Deploy

  • Type in whatever name you want.
  • Click “Deploy app”. Don’t touch any of the variables unless you know what you’re doing.
  • Click view and copy the URL.
  • Click manage app and go to Settings > Reveal Config Vars and copy the ACCESS_KEY.

That’s it.

(Setting up without heroku is simple: run node server.js with the environment variables specified here)

Client Setup Tutorial

  • Get the handler script from here and put it in a module script in ServerScriptService.
  • In the script you want to use this from, require the ModuleScript. If your module is named “ProxyService”, for example, you would add local ProxyService = require(game:GetService('ServerScriptService').ProxyService) to the top of your script.
  • Add a line to create your proxy client, this will generally look like this: local Proxy = ProxyService:New('PASTE_DOMAIN_HERE', 'PASTE_ACCESS_KEY_HERE') (see below for a more complete example)
  • Use Proxy exactly as you would use HttpService. The only difference is an extra overrideProto argument. You can pass in http if you are using an API that doesn’t support https (the default protocol).

Video Tutorial

(Go here to follow along)
Here’s a video of the above instructions (excluding heroku sign up):

Client API

ProxyService:New(root, accessKey)
  returns Proxy
Proxy:Get(url, nocache, headers, overrideProto)
Proxy:Delete(url, nocache, headers, overrideProto)
Proxy:Post(url, data, contentType, compress, headers, overrideProto)
Proxy:Put(url, data, contentType, compress, headers, overrideProto)
Proxy:Patch(url, data, contentType, compress, headers, overrideProto)

All methods return

{
  "headers": {
    "name": "value"
  },
  "body": "string",
  "status": {
    "code": "number",
    "message": "string"
  }
}

Note that all response headers are lowercase

Root is the root of your heroku application including the http:// or https://.

Simple example script:

local ProxyService = require(script.Parent.ProxyService)
local Proxy = ProxyService:New('https://prxysvr.herokuapp.com', '6ddea1d2a6606f01538e8c92bbf8ba1e9c6aaa46e0a24cb0ce32ef0444130d07')

print(Proxy:Get('https://api.roblox.com/users/2470023').body)
-- Note that the proxied request will always be https unless specified by overrideProto
-- The protocol of the request to the proxy is dependent on the root and not the url

Advanced example script (login to a user and remove their primary group):

(Actually logging in to a Roblox account from in-game to use essential functions is not recommended)

local ProxyService = require(script.Parent.ProxyService)
local Proxy = ProxyService:New('https://prxysvr.herokuapp.com', '6ddea1d2a6606f01538e8c92bbf8ba1e9c6aaa46e0a24cb0ce32ef0444130d07')
local username = 'Shedletsky'
local password = 'hunter2'
local tokenCache
local getWithToken

local http = game:GetService('HttpService')
local encode = http.JSONEncode

getWithToken = function (handler, retry, ...)
  local res = handler(tokenCache, ...)
  if res.status.code == 403 and res.status.message == 'Token Validation Failed' then
    if retry then
      error('Failed to get token')
      return
    end
    tokenCache = res.headers['x-csrf-token']
    return getWithToken(handler, true, ...)
  elseif res.status.code == 200 then
    return res
  else
    error('Login error: ' .. res.status.message)
  end
end

local createTokenHandler = function (handler)
  return function (...)
    return getWithToken(handler, false, ...);
  end
end

local loginHandler = function (token)
  return Proxy:Post('https://auth.roblox.com/v2/login', encode(http, {
    ctype = 'Username',
    cvalue = username,
    password = password
  }), Enum.HttpContentType.ApplicationJson, false, {
    ['X-CSRF-TOKEN'] = token
  })
end

local deletePrimaryHandler = function (token, cookie)
  return Proxy:Delete('https://groups.roblox.com/v1/user/groups/primary', nil, {
    ['X-CSRF-TOKEN'] = token,
    ['Cookie'] = cookie
  })
end

local login = createTokenHandler(loginHandler)
local deletePrimary = createTokenHandler(deletePrimaryHandler)

local res = login()
local cookie = res.headers['set-cookie'][1]:match('.ROBLOSECURITY=.-;'):gsub('_|.-|_', '')

deletePrimary(cookie)

print('Done')

Responses are different with ProxyService: instead of just the body, a table is returned with a dictionary of headers in the headers field, the body in the body field, and the status code and message in the status field.

Example response:

{
  "headers": {
    "set-cookie": "one=two;",
    "content-encoding": "gzip"
  },
  "body": "Success",
  "status": {
    "code": 200,
    "message": "OK"
  }
}

Notes

  • Requests use https by default, if the endpoint you are trying to access does not support https you must override the protocol (see API above).
  • Despite using https, server certificates aren’t actually validated. If you want to do so you’ll have to deal with installing client certificates.
  • Although the appending process seems simple at first (just write the extra data after the proxy response has been received), it becomes a lot more complicated when factoring in encodings. In order to add additional data the server has to first decode the response, append the data, and then re-encode the entire thing. Two alternative methods are available: one is to decode and not re-encode (sacrificing bandwidth for server performance), and the other is to append a separate gzip file to the request. The latter option seems to be the most ideal overall, but unfortunately it is not stable: that is, it is not supported by any spec, yet occasionally a client will support it because of the way they implement gzip. The Roblox client does not support this, unfortunately, but this proxy was created with non-Roblox clients in mind. To change the way the server handles encoded data you can change the GZIP_METHOD environment variable to any of these three values: ["transform", "decode", "append"].
  • Accept-Encoding is always overwritten to “gzip” because deflate is not supported. This is unlikely to affect anybody at all.
  • Heroku gives a generous number of free dyno hours, but note that without adding a credit card you are not able to run one dyno 24 hours nonstop. If you just add a credit card you’ll get enough hours for the server to be constantly on for an entire month, every month, even if you don’t actually spend any money.
203 Likes
HttpService should support requests for RESTful-based APIs
Any free Catalog Proxy that can take up a lot of requests?
Roblox Proxy works for every website except Roblox
HttpService "Trust Check Failed" Error
HttpService Cookies
Discord Webhook | Http Forbiddeon
I need help with the catalog API
ROBLOX Economy API
Catalog API for Homestore
Catalog API, how to use it?
Exporting Accessories To a Module
HTTP 410 (rprxy is no longer available)
I need a bit of help making this Catalog script
Inserting EVERY hat in existence
Returning as nil
Need help with getting player RAP
How do I get the amount of likes my game has?
Roblox to discord visit stats
Is there a way to get audios from the library with a LocalScript?
How do you get info about a player from a ID?
Whats the best CURRENT solution to getting a collectible's Best Price
How to make requests from Roblox to Discord
How Would I Get Place Likes and Dislikes?
Any devforum API endpoints?
How can I get a Random Catalog Item?
Loading assets quicker
Trust Check Failed
Alternative Roblox web API proxies to use?
My devforum notifications are coming in way too late
Help getting game icon
Can't send cookie in header
Discord Webhook | Http Forbiddeon
Retrieve information on my game every 1 second via a proxy without caching
Checking if a place is public/private/copylocked/uncopylocked?
Is it possible to get player's favorite places through scripting?
Is it a good idea to get an http request from api.roblox.com using javascript?
Is there an api or http accssibility to status.roblox.com?
Plugin not loading in correct game icons?
DataStore API - HttpService is not allowed to access ROBLOX resources
Roblox API 401 Authorization has been denied
How to add catalog items into a module with proxy
How to send HTTP requests TO roblox?
Need help proxying a Discord API request
How to find how many people own a badge?
How can I send post requests to Discord?
What's the URL for getting the players favorite games?
WebhookProxy | Discord webhooks go brrrrrrr
WebhookProxy | Discord webhooks go brrrrrrr
How to get Player Headshot URL
How do i get a game with a game id? And all the games data like likes name etc
How do you think this would be done?
[CLOSED] Looking for someone to fix a model
Presence API unwanted behaviour
How would I count all badges a user has?
Total Players with Badge
How can I query all of the hats made by a person?
Is there a way to collect game visit data?
This topic is closed dont click
Subscribers Goal
Getting "HttpService can't access ROBLOX resources" when trying to use a webhook
Get plugin version, or owner?
How to make community milestone
Having Trouble with Login API, Cannot Log Into Bot Account for Group Auto Role
Roblox API Code 403 (Token Validation Failed) when using the group join request user API with Python
Fetching Group Games from a Group ID
How do I fix my live like script
InternalServerError for Roblox API "/v1/games/list"
Music search bar
Trust Check Failed Https Service
Invalid argument #2 to 'random' (interval is empty)\\
Live group counter script not working
Invalid argument #2 to 'random' (interval is empty)\\
How to use the Twitter API inside studio?
Discord Requests
Roblox Scripting Roadmap & Learning Resource List
How can I get the value of an array?

Source code: https://github.com/sentanos/ProxyService

6 Likes

Nice tutorial :slight_smile:


You can edit your posts btw
Allows you to add more content to the tutorial in the future too

2 Likes

Pretty cool tutorial. Will definitely use this soon.

2 Likes

This is a great tutorial. I would use this if I were not sending all my data to / from / through my own website.

3 Likes

bookmark

5 Likes

There is no access key. Where can I find it?

it should be under your heroku app settings > reveal config vars, and there should be a value called ACCESS_KEY, but you honestly won’t be needing this soon by the looks of it, if you’re just looking to use currently unsupported request methods:

https://wiki.roblox.com/index.php?title=ReleaseNotes/343#FFlagHttpServiceRequestAsync

https://wiki.roblox.com/index.php?title=API:Class/HttpService/RequestAsync [NOT LIVE YET]

6 Likes

Thanks.

Good if Roblox finally heads in the right direction, but this will hardly become unnecessary unless Roblox also stops blocking requests to their own API from in-game (which I doubt they would do but would be glad if they finally did). It is true, though, that if they did do this the proxy could be a lot less complicated.

this is very true. I forgot about not being able to communicate with the roblox.com domain. RequestAsync, however, does appear to be returning more useful information too:

Doesn’t seem to be returning the response body at this time, though.

Will this be updated to use more of the new HTTP API?

You effectively don’t need this anymore, all it offers that you can’t do with the new API (as far as I can see from this post) is the proxy functionality, which you can implement stand-alone.

3 Likes

Yeah, what @buildthomas said. This isn’t needed needed anymore because the new API offers everything this does and more.This is confusing to set up and there isn’t much point to it now, but it was amazing before the new API was released!

1 Like

@grilme99 Alright, wasn’t too sure, thanks

Late response, but the one thing that this can do is send requests to the Roblox Web API. The new RequestAsync method does not allow you to access Roblox resources.

As someone who uses HttpService for nearly nothing but using a proxy to access Roblox endpoints, that’s crucial, and this provides what I need.

Yes, this is a repeat of what I already said:

5 Likes

How would you add a function to execute without doing a request if a request is incoming from the game and have a specific header ?

I recently tried the httpservice but noticed that it doesn’t properly decode like httpservice does (although with further looking on the client module, it does use the http decoder)
It would return this:

{“Id”:2470023,“Username”:“Froast”,“AvatarUri”:null,“AvatarFinal”:false,“IsOnline”:false}

instead of this:

{
[“Id”] = 2470023,
[“Username”] = “Froast”,
[“AvatarUri”] = null,
[“AvatarFinal”] = false,
[“IsOnline”] = false
}

1 Like

Have you figured this out? I’m having trouble to retrieve the value of each key/data.