Catalog API Proxy "Failed to open stream"

Help and Feedback Scripting Support
1

I’m using a proxy using file_get_contents() of the roblox catalog API to get Asset Ids for a group catalog. After a few requests, it says that the stream was failed. As of now, I have tested with different intervals and wait times to see if that would help. I was hoping to speed it up a bit however since it does take a while sometimes.

Script In Game -

   local assetIds = {}
local HttpService = game:GetService("HttpService")
local cursorAvailable = true
local cursor = ""

while cursorAvailable == true do
	local url
	if cursor == "" then
		url = "https://*****************.000webhostapp.com/CatalogReceiver.php"
		for i,v in pairs(game.Players:GetChildren()) do
			v:WaitForChild("PlayerGui"):WaitForChild("mainUi"):WaitForChild("status").Text = "Scanning Page: 1"
		end
	else
		url = "https://*************.000webhostapp.com/CatalogReceiver.php?cursor=" .. cursor
		for i,v in pairs(game.Players:GetChildren()) do
			v:WaitForChild("PlayerGui"):WaitForChild("mainUi"):WaitForChild("status").Text = "Scanning Page: " .. string.sub(cursor, 1, 1)
		end
	end
	local response
	response = HttpService:GetAsync(url)
	if not response:match("creatorType") then
		repeat
			response = HttpService:GetAsync(url)
			print("Request Failed. Trying Again in 5 Seconds")
			wait(5)
		until response:match("creatorType")
	end
	local data = HttpService:JSONDecode(response)
	for i,v in pairs(data.data) do
		table.insert(assetIds, #assetIds + 1, v.id)
	end
	for i,v in pairs(game.Players:GetChildren()) do
		v:WaitForChild("PlayerGui"):WaitForChild("mainUi"):WaitForChild("scannedIds").Text = "Scanned Ids: " .. #assetIds
	end
	if data.nextPageCursor then
		cursor = data.nextPageCursor
		cursorAvailable = true
	else
		cursorAvailable = false
	end
	print(url)
	wait(4)
end

PHP for website.

<?php
if (isset($_GET['cursor'])) {
    echo file_get_contents('https://catalog.roblox.com/v1/search/items/details?Category=1&CreatorType=2&CreatorTargetId=7115611&Cursor=' . $_GET['cursor']);
} else {
    echo file_get_contents("https://catalog.roblox.com/v1/search/items/details?Category=1&CreatorType=2&CreatorTargetId=7115611");
}
?>
2

Most likely a issue with their API.

3

Doesn’t seem to be as it mostly works, however while going through the catalog it randomly stops usually around 7-8 requests and fails to stream. After a few attempts later it works again. I’m guessing it is most likely a rate limit and I’m not sure if there is a “bypass” to it.

1 Like
4

Oh then, maybe it is a rate limit, I am not a expert upon guessing what’s happening. You can’t bypass a rate limit unless you,
Switch IPs(Often, servers tries to use your IP to keep data about how many times you’re requesting.)