403 Forbidden response when sending requests to Devforum Api on Linux Server/VM

TheH0meLands · June 21, 2022, 3:12am

Anytime I try to send requests to the devforum, more specifically this endpoint on a linux server, I get a 403 Forbidden error. But when I send a request on a windows server, I get a 200 success response with the data I expect to get.

Although whenever I send a request via the browser on a Linux VM or via post man, the request goes through. Making me believe that this is some bug.

This issue is problematic as code I wrote months ago no longer works, with no change whatsoever with no description on how to fix.

Originally, I had thought this was an issue with my code, until I tried to test locally on my windows machine, the same code, nothing changed, worked. So then I tried on a linux vm, same issue once again.

How to reproduce
first, create a javascript file with the following code(using node.js) on either a linux VM, or linux server. Any distribution is fine.

// run npm init -y in terminal
// run npm install node-fetch@2.6.1 in terminal
// run node index.js in terminal

const fetch = require("node-fetch")

async function doSomething() {
    const url = "https://devforum.roblox.com/latest.json"
    const response = await fetch(url)

    console.log(response.status, response.statusText) // 403 Forbidden
}

doSomething().then(() => {
    console.log("test over")
})

Although when I run this exact code on a windows server/my main pc, I get a 200 OK response.

Expected Behavior
I expect to be able to send requests to the devforum api, no matter the OS im hosting my code on.

Actual Behavior
I get a 403 Forbidden response when my code sends a request to the devforum on either a linux vm or linux server. While the same code works on windows/windows server.

Specs

My VM specs(i dont know my server specs, only my VM specs of which I used to test the code):

OS: Manjaro Linux x86_64
CPU: Intel i7-10700F (2) @ 2.904GHz
GPU: 00:0f.0 VMware SVGA II Adapter
Memory: 7916MiB

Issue Area: Website
Issue Type: api
Impact: Very High
Frequency: Constantly

Judgy_Oreo · June 21, 2022, 6:25am

This seems more like an issue with your code than with Roblox, try making a post in StackOverflow.

Maybe it’s an issue with Linux’s network driver? A configuration mistake? I don’t think Roblox would just block any HTTPS requests because it’s using Linux, given that Roblox’s servers run on Linux.

Maybe try changing the URL to http://devforum.roblox.com/latest.json, it could be a connection encryption issue.

commitblue · June 21, 2022, 9:24am

No this isnt. The forum blocks requests from linux at that endpoint. Its not supposed to be in stack overflow really.

Judgy_Oreo · June 21, 2022, 9:25am

Did you get this information from somewhere other than this post? Personally I don’t use Linux and I don’t want to brick my laptop, so I apologize if Roblox does actually block Linux’s HTTP requests. I’m just a little skeptical, and I’d appreciate any sort of widespread evidence that this isn’t an issue with Linux.

commitblue · June 21, 2022, 9:32am

I can try linux on a vm and test the endpoint, And this forum bug request was supposed to unblock the endpoint from linux. So you cant just post it at stack overflow.

He used a VM.

Judgy_Oreo · June 21, 2022, 9:34am

For all I know this is some compatibility issue with Python and NodeJS modules, which in my mind seems more likely than Roblox blocking an entire family of operating systems. Wouldn’t this mean that anybody viewing the DevForum on Linux will get a 403 error?

commitblue · June 21, 2022, 9:36am

… He said it happens at THAT specific endpoint. Not the whole forum

Judgy_Oreo · June 21, 2022, 9:39am

This just makes me even more sure it has nothing to do with the DevForum specifically if it’s working in their Linux browser, but not their code.

Maybe an update to the Node module broke some stuff, idk.

TheH0meLands · June 21, 2022, 11:56am

If this was the case, my code wouldn’t work on windows. Instead, when I run on linux via a node.js server i get a “403 forbidden” response. But on windows, i get a 200 OK. Making this be an issue in roblox’s part which can only be fixed by them. I am making this post for this bug to be fixed because code I wrote 3-4months ago are now broken simply because they were hosted on linux. Keep in mind, those scripts have been unedited FOR MONTHS, and now breaks with no announcement from roblox?

It’s the same thing as Roblox randomly removing a really used api in the engine and then not saying anything. Pretty weird right?

Everything is up to date. With all my drivers being updated. If that was the case, I wouldn’t be able to send requests at all without getting a response( the 403 forbidden)

This issue is not a browser issue. This is a SERVER SIDE ISSUE, NOT client side issue. And as I said before, this issue doesn’t happen in browser or postman. Only happens when u host a node.js server on linux, but doesn’t when you host a node.js server on windows.

Excuse my poor wording there, I wrote that really late at night. But this is an issue anytime I send requests to the forum. I just used that endpoint for testing.

Client side works fine, server side doesn’t.

Client = browser
Server = node.js code

Theres 2 reasons why this is false:

I ALWAYS use node-fetch version 2.6.1, meaning it isn’t an update.
If it was an issue with the dependency, I would be getting a completely different error inside my console. Instead, it sends the request completely fine. but the response retrieved on linux is a 403 forbidden status. While on windows, I get a 200 OK response. With no edits whatsoever with the code.

Mods, PLEASE fix this. I recently made a twitter bot which notifies you of devforum posts(respects ratelimits) sending 1 GET request every 3 seconds. But I cant host it because of this bug, and many code I wrote in the past, kept unedited for months. Doesn’t work anymore also due to this bug.

TheH0meLands · June 21, 2022, 12:01pm

If it was an issue with linux, I wouldn’t be able to send requests to the api at all. The requests are succeeding at sending, but you get a 403 Forbidden status code as a response.

If you want to test, you can go to replit.com, sign up, create a node.js project and paste the code I provided in the original post. Run the code and you will see that you get a 403 forbidden error, but if you test on windows(locally or on a server) you get a 200 OK response. This is because replit uses linux for their servers. This is if you dont know how to test on linux without wanting to test on a vm or something

Judgy_Oreo · June 21, 2022, 12:22pm

When I used Python, I got HTTP 200 for some reason

import urllib.request

returnValue = urllib.request.urlopen("https://devforum.roblox.com/latest.json")

print("Status code: " + str(returnValue.status))

I got a 403 error on NodeJS though, only, small problem,

I got a 403 on the home page, not specifically on the JSON URL. I used the following code;

const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));

async function doSomething() {
    const url = "https://devforum.roblox.com"
    const response = await fetch(url)

    console.log(response.status, response.statusText) // 403 Forbidden
}

doSomething().then(() => {
    console.log("test over")
})

TheH0meLands · June 21, 2022, 12:26pm

Hmm very strange.

But I still believe it’s an issue with the devfourm api as its returning a 403 error code any time I run the code on a linux server, but not on a windows one.

Judgy_Oreo · June 21, 2022, 12:31pm

I have regained my confidence that this is not a DevForum issue. Surely, if a browser running on your Linux machine works fine but your code does not, that should mean it’s a problem with your code or some library it uses, right? Anyway, I’m not going to reply any further, I feel like there’s enough evidence this isn’t an issue with the DevForum, which is what I believe.

Although, I would still appreciate it if some staff member showed up and confirmed or denied anything on this thread. Hope all goes well for you in the end.

TheH0meLands · June 21, 2022, 12:37pm

the same exact code with the same library worked months ago. Code just doesnt randomly break without warning.

This issue is not a client side issue. It’s a server side issue. Meaning requesting the data on a browser would work, as this bug is related to server side rather than client side.

Again, the code works, I used the code MONTHS AGO and I would get a proper response, but now I simply get a “403 forbidden” error.

Hooksmith · June 21, 2022, 2:55pm

We don’t officially support automated access to the Developer Forum. You may see traffic being blocked outside of normal use cases.

Is this preventing you from using the forum normally on a browser on either Windows or Linux?

TheH0meLands · June 21, 2022, 3:04pm

This is not preventing me from accessing the forum using a browser on either linux or windows.

But please atleast remove the auto 403 errors. Its making all of my code break which rely on the devforum api to run. It seems really weird that you guys cut off support for those apis without telling anyone.

Hooksmith · June 21, 2022, 3:11pm

Thanks for confirming – will close this out since I don’t believe there is a bug here.

Please make a feature request in #forum-help:forum-features if you want us to officially support automated access to the devforum APIs. It would likely require allowing users to set up some sort of API key or other way of identifying the traffic so that we know who is making the requests.

Hooksmith · June 21, 2022, 3:11pm