CatalogScraper
Scrape assets from the Roblox catalog with ease
Author: @Starnamics
Version: 1.0.0
License: MIT License
Download on GitHub
Download on Roblox
View Source
Introduction
CatalogScraper provides developers with a way to scrape assets from the Roblox catalog.
CatalogScraper has support for all functionality of the catalog search API and allows for developers to fetch a huge amount of assets from the catalog, with virtually no limit to the amount of assets that can be fetched.
CatalogScraper is also fully typed, making it extremely easy to start using.
Installation
Installation of CatalogScraper is extremely easy, all you need to do is insert the module into ServerScriptService!
You can get the module from either the Roblox library or from the GitHub releases page
Once you’ve downloaded the module, open up studio and either insert it from the toolbox or from the file explorer.
Examples
Scrape the Top 100 accessories from the catalog
This example shows you how to scrape the Top 100 accessories from the catalog and print them out in the console.
--// Require the module
local CatalogScraper = require(script.Parent:WaitForChild("CatalogScraper"))
--// Create scrape parameters
local ScrapeParams = CatalogScraper.ScrapeParams.new()
ScrapeParams.Category = "Accessories" --// Set a category
ScrapeParams.Subcategory = "Accessories" --// Set a subcategory
ScrapeParams.Limit = "10" --// Set the limit of asset results per request
ScrapeParams.SortType = "Favorited" --// Set the sort type
local Accessories = CatalogScraper:Scrape(ScrapeParams,100) --// Scrape 100 assets using the scrape parameters
for Number,Data in pairs(Accessories) do --// Loop through all accessories returned
print(Number,"-",Data.Name) --// Print the accessory and it's ranking
end
Documentation
Scraping Assets
To scrape assets, you can use the CatalogScraper:Scrape()
function.
This function will send requests to the catalog search API and will parse and compile the results into an array for you to use.
Due to limitations with the search API, you can only get up to 30 results per request currently, however you’re still able to request as many results as you’d like as the module will continue scraping until the amount of assets you wanted have been returned.
You are also able to scrape continuously until there are no more results by entering no amount to the arguments, however this may take a while to complete depending on the scrape parameters you entered.
Creating ScrapeParams
To create ScrapeParams, you can use the CatalogScraper.ScrapeParams.new()
function and then edit the values of the returned dictionary.
Functions
CatalogScraper:Scrape()
Description:
Scrapes assets using the catalog search API
Usage:
CatalogScraper:Scrape(SearchParams,30,true)
Arguments:
- Parameters:
ScrapeParams
– Required
– Parameters for scraping - Amount:
number
– Optional
– Amount of assets to scrape - LogOutput:
boolean
– Optional
– Should the amount of assets scraped be logged on each request
Returns:
- Assets:
{...Asset}
– An array of the scraped assets
CatalogScraper.ScrapeParams.new()
Description:
Creates an empty ScrapeParams
dictionary
Usage:
local ScrapeParams = CatalogScraper.ScrapeParams.new()
ScrapeParams.Category = "Accessories"
ScrapeParams.Subcategory = "Accessories"
ScrapeParams.Limit = "10"
ScrapeParams.SortType = "Favorited"
Arguments:
- None
Returns:
- ScrapeParams:
ScrapeParams
– EmptyScrapeParams
dictionary
Classes
Asset
AssetType: string,
BundleType: string,
Description: string,
CreatorHasVerifiedBadge: boolean,
CreatorName: string,
CreatorTargetId: number,
CreatorType: string,
FavoriteCount: number,
Genres: {...string},
AssetId: number,
ItemRestrictions: {...string},
ItemStatus: string,
ItemType: string,
LowestPrice: number,
Name: string,
Price: number,
PriceStatus: {...string},
PurchaseCount: number,
UnitsAvailableForConsumption: number,
ProductId: number,
ScrapeParams
Category: string
CreatorName: string
CreatorTargetId: number
CreatorType: string
Cursor: string
Genre: string
Keyword: string
Limit: string
MaxPrice: number
MinPrice: number
SortAggregation: string
SortType: string
Subcategory: string
Thank you
If you encounter bugs, need support or have feedback for me, please let me know in the replies of this topic!