HTML/DOM Reader module

[MainGag (Gag)]

The item: HTML Module [v2 in the making!] - Roblox

I’ve made a module today, that replicated js DOM functions as lua methods:

• module:getElementsByClassName()
• module:getElementById()
• module:getElementsByTagName()

They are used with html pages. If you want to get a certain data off a website page, but cannot easily access or cannot access apis or JSON like twitter for example, you can use this.
How to use:

• Get the html content of the page with httpaccess (httprequests turned on)
• Get the precise data you were looking for, with the module I created.

Example:

--httprequests turned on. 

local http = game:GetService'HttpService'; 
local url = 'https://www.twitter.com/Roblox'; 
local x = require(workspace["HTML Module"]); 
local data = http:GetAsync(url); 

if data~=nil then 
	local requestedClass = 'TweetTextSize TweetTextSize--normal js-tweet-text tweet-text' --html class of a tweet, found with inspect element
	local tweets = x:getElementsByClassName(data, requestedClass, 3)--third parameter is 3, because I don't want my table to be longer than 3 rows. 
	
	for i,v in pairs(tweets) do
		print(i..': '..v); 
	end	
end

Result:

As stated inside the ‘read me’ part, the scripts should work most of the time, but it is not guaranteed it’ll always work due to the fact that it’s simple string manipulation, and in some cases, html can be really confusing for example, when there are lots of same ‘tags’ inside other tags.

Don’t hesitate to correct me, as string manipulation is not my strong point in lua. I might have used in-built functions in a wrong way, or forgot using a more efficient one.

Feel free to use the module, would be kind if you credited me.
Also I’m not responsible of what you’re doing with my ‘creation’.

Many features off this could be done. For example an in-game twitter feed UI.

[Dekkonot]

As a head’s up, you can always send a message to @Post_Approval to request a post to be placed somewhere else:

@Roblox

What if I want to post in a category I don’t have access to?

If you want to post a feature request, bug report, or some other issue/request as a New Member, make the thread in the #bulletin-board category first. Then, please click here to send a message to the @Post_Approval group, and include the link to your Bulletin Board thread. We will review your thread and move it to the right category after we approve it. (This whole process is commonly referred to as the “post approval process”.)

That being said, neat! HTML reading is always annoying in Lua.

[MainGag (Gag)]

Thank you, I don’t know if it’s still time for this post, but next time I will do it. Might not change this post, as not so much of a ‘off-topic’ in this subcategory I guess.

I should be more aware of the protocols here.

[buildthomas]

This is fine in Community Resources because you have the module open-sourced.

[MainGag (Gag)]

@UnlimitedKeeping

please don’t get mad at me if this doesn’t work, I just didn’t wanna test it…

Why would I or someone get mad

However, I stated that I’m not responsible of the usage of this module.
I haven’t tried doing that and seeing what it returns, I think it would return a ROBLOX server ip, if ran on server. I also think that HttpService getAsync’s can only ben sent from servers, so we are fine.

If your method does work, then my module isn’t reponsible in the extent that you can do it with anything.

[UnlimitedKeeping (Avenzey)]

yeah, because HTTP cannot be used clientsided, so then it’s safe.
and getting the content of the ifconfig.me website serversided, it’ll just return the server ip,

okay, problem solved! thanks m8!

[AxoLib]

Lol I just realized that we we could make an in game web browser with this . That would be so cool.

[TechSpectrum]

I think someone made a module to interpret HTML into a GUI a long time ago. It may still be around here. That or it was CSS.

[MainGag (Gag)]

We could actually make a module reading css,another reading html tags and the dom one, for gadgets and animations.

[TechSpectrum]

For gadgets and animations you mean jQuery and JavaScript? There’s also the matter of loading the images and fonts, yikes.

[MainGag (Gag)]

This could be done with a server held on the side with some node.js I think.

[SoCalifornian (Cali)]

Loading the images isn’t the problem, it’s more the fonts. Which I guess you could convert the font file into a spritesheet and use that lol

[MainGag (Gag)]

Another way, would be to make a list of all the fonts in roblox: and link like to most 50 fonts on the internet to the most similar roblox font. And add a default font if it is not recognized. Wouldn’t have the same aesthetic aspect, but wouldn’t look bad.

[TechSpectrum]

Thats a pretty nice alternative. And it could default to the most likely font based on the html tag if it doesn’t exist on the list.

[valchip]

True but it works on ROBLOX Studio however . Backdoor plugins are able to log your IP.

[cxmeels]

this would actually work pretty well with my request module, as the response body transform function to return parsed HTML

local request, htmlParser = require(path.to.request), require(path.to.parser)
local requestedClass = 'TweetTextSize TweetTextSize--normal js-tweet-text tweet-text'

request({
    url = 'https://www.twitter.com/Roblox',
    transform = function(body)
        return parser:getElementsByClassName(body, requestedClass, 3)
    end
}):andThen(function(tweets)
    for key, value in next, tweets do
        print(key, '=', value)
    end
end)

[MainGag (Gag)]

I currently don’t have the time, to do any of these small projets. However, if nobody does it by then I’d gladly make some features for games, such as twitter-feed ui, or stuff that could be really cool.

There are thousands of awesome feature’s ideas that are waiting for their creator.