Roblox Proxy works for every website except Roblox

Help and Feedback Scripting Support
#1

I’ve been trying to make a Roblox proxy for the catalog API for the past day-ish, and I got a simple system working. The only problem is is that it works for every website, except Roblox.

Example:
https://pixelgrid.games/proxy/twitter.com - Works
https://pixelgrid.games/proxy/youtube.com - Works
https://pixelgrid.games/proxy/roblox.com - Doesn’t work
https://pixelgrid.games/proxy/devforum.roblox.com - Works (???)

And here’s the code, I’m using Google Firebase Cloud Functions and I wanna keep using them because I’m happy with the price and I already have a bunch of stuff setup using them. I’m guessing there’s some config option I’m missing out of options but I have no idea

//Variables
const functions = require('firebase-functions');
const express = require('express');
const cors = require('cors');
const request = require('request');

//App
const app = express();
app.use(cors({ origin: true }));

//Endpoints
app.get('/**', function(req, res) {
    
    //Variables
    var url = req.url.substring(1).replace('proxy/','');
    var options = {
        method: 'GET',
        uri: 'https://' + url,
    }
    
    //Request
    request(options, function(err, proxyRes, body) {
        if (err) {
            res.send(err);
        } else {
            res.send(body);
        }
    });
});

//Functions
exports.proxy = functions.https.onRequest(app);
2 Likes
#2

What is the status code of the response? as well as the body

#3

By just visiting https://pixelgrid.games/proxy/roblox.com in the browser, the site keeps refreshing. Inspecting the response, it looks like Roblox is serving up a basic page that says “turn on Javascript and cookies”, tries to set a cookie, and refreshes.

Maybe your request states that it doesn’t support Javascript or cookies (through the user agent maybe?), so Roblox serves up a basic page. It refreshes so that if you change your settings, Roblox automatically starts working. You could try setting the proxy’s user agent to the user agent of your browser and see if it still won’t work.

4 Likes
#5

The ‘turn on your js and cookies’ message is wrapped in a <noscript> tag, which is probably included in all responses regardless of whether js and cookies are enabled or not. The only thing on the page is a script that sets a cookie and reloads the page. Perhaps your proxy/browser isn’t returning the cookie back to Roblox when it reloads, thereby causing the entire process to repeat? I found similar behavior when I opened an incognito tab, turned off cookies, and loaded roblox.com. The only difference is that it returned status 200 instead of 304, but this is because I’m not going through a proxy. The response had no body. Because of this, I suspect that it is actually your proxy that is writing the javascript to save the cookie and reload the page, rather than the Roblox servers.

I would suggest looking into why the saved cookies aren’t being returned back to Roblox. If you can fix this, then I suspect it will start working as expected.

#6

I don’t think this happens for all requests. If I visit Roblox in incognito mode with cookies and javascript on, then I never get an intermediate “turn on cookies and javascript” page, meaning that Roblox can’t be putting that page before every first site visit. They must be doing it only for requests that state that they don’t support javascript or cookies.

I think that stating that cookies and javascript are supported in the request is all that @gillern needs to do to get the right page to load. I know that other Roblox tools can request pages just fine without actually storing cookies or running javascript.

#7

As I said in my original reply, I don’t think Roblox is the one that is writing that noscript message. I think his proxy is doing it. Obviously, you wouldn’t get that message in incognito with or without cookies enabled, because you aren’t going through his proxy.

The reason I was experimenting with incognito with cookies off was to see what might be happening between his proxy server and Roblox.

Again, I don’t think it has anything to do with your headers. Rather, I still think that your browser/proxy isn’t returning previously set cookies to Roblox. And, for some reason roblox.com (or chrome) has this weird behavior to infinitely reload when connecting with cookies off. For me, visiting www.roblox.com instead can fix this issue.

#8

Couple things. First of all you have to access www.roblox.com and not roblox.com. Second, you have to have a User-Agent header in order to prevent their automatic DOS protection. It has nothing to do with cookies, any page can be correctly retrieved even if you aren’t a browser.

const options = {
  method: 'GET',
  url: 'https://' + url,
  headers: {
    'User-Agent': 'Mozilla'
  }
}

And I’m curious, why are you writing your own proxy when so many exist that are not only much more efficient but also much more versatile? [1][2][3]

4 Likes
#9

I’d like to add roblox.com redirects to www when cookies are enabled, but not when they aren’t, which is the weird behavior I was refering to in my previous post.

#10

I see. Of course Roblox would do something as useless as that.

1 Like
#11

Here’s a demo I recorded of this “weird behavior”:

Note how that infinite refreshing is remarkably similar to what OP is experiencing. Also, there’s not any of that noscript stuff in any of the responses.

1 Like
#12

Because I’d like to learn how to write my own