[OS/MIT] Advanced chat filter for fraud messages

Resources Community Resources
,
#1

I was planning not to create a topic in DevForum for only this, but I did it anyway.

I’ve created a chat filter to prevent fraud messages from appearing in your game, this chat filter is similar to the another chat filter found in DevForum, but this one adopts a much special design to avoid cases where you want to talk about something related to robux, and the system misunderstood that you are sending a fraud message.

Example: “I earned a lot of robux via commissions”
Most other filters’ result: “I earned a lot of ##### via ##########”
My filter’s result: “I earned a lot of robux via commissions”

Example 2: “I earned a lot of robux via fakelink.xd”
Most other filters’ result: Blocked
My filter’s result: Blocked

For those who want to look further into the design of my filter, here’s a short brief description of the design:
When there’s a message going through the chat system, my filter will react to it and scan for any malicious words in the wordlist in the module, and if there is, the system will continue looking for a link, and if there is, the filter will block the message from going through by replacing the whole message with something else that will not catch the user’s attention.

With this design, accuracy will be increased by a lot and you do not need to add new links into the filter, making the whole moderation process less labour-intensive. However, because of that, no matter what link it is, it will get blocked if malicious words and link are found.

Apart from that, another pro of this design is I don’t have to upload the wordlist in a server or git repository like GitHub, so, there’s no chance that the filter can be broken as no HTTP requests were made.

There’s really nothing to talk or explain about a filter, so I’ll just drop the GitHub repository to the source code here:
https://github.com/va1kio/roblox-assets/blob/master/src/chatFilter.lua

Simply copy the code into a ModuleScript and put it into ChatModules in Chat. Create a folder called ChatModules if you don’t have that folder in Chat.

Additional notes

As I’ve mentioned about the main drawback of the design system I’ve adopted for my filter, please do let me know if you have any solution to solve that issue without having to add every malicious link to the list or anything like that. Thank you.

10 Likes
#3

Why would this module be preferable rather than the many other chat filters? What makes this unique and separates it from the others? Are there any benefits to this than using another chat filter?

From the looks of it, it functions just about the same as every other chat filter and I don’t really see the point of using this instead of the many other original chat filters.

#4

As mentioned in the topic, it uses a distinctive design to prevent cases where the system misunderstood the context of message, thinking it is a malicious message.

#5

If possible, could you give a snippet of the code doing what you described? After looking through the repository, the only thing I found different was the string.match.

#6 
	for i,v in pairs(sensitiveWords) do
		if message:find(v) then
			sensitiveWordsFound = true
		end
	end
	
	if sensitiveWordsFound then
		-- just finding the sensitive words are not enough
		-- in this section, we'll figure out whether the message contains the link or not
		-- this isn't very accurate considering we are using taking a bold guess
		-- thinking all the links must be suspicious when sensitiveWordsFound are true, rather than actually reading the domain name
		-- but the chance of inaccurate filter result are pretty low with this design -- @nana_kon
		
		if string.match(message, "%S+%.%S+") then -- %S dot %S

Even though the actual difference isn’t very big, I believe it’s worth creating this topic as most of the filters did not implement something similar to this.

2 Likes
#7

Is that the same exact string.match being used in the filter right now? If so, it will currently not be an accurate filter – right now spammers are filling their messages with ZWSJ and other similar zero-length ASCII characters in order to confuse the filter.

#8

Hm, in that case, I might make the code to check whether the message contains the word by this:

Separate the words into a table, and then check if each word contains any characters found in each suspicious word listed in the list, and if it matches around 75% or higher, and the length of the word is close to the suspicious word, it will be marked as a malicious message

#9

This is absolutely against terms of service and any game using any kind of filter bypass which displays anything other than the filtered version can be taken down.

You can only go stricter than the filter, you should never bypass it, regardless of whether you think your result is better or not.

Hopefully that’s just me misunderstanding your example, or a misleading example on your part. I strongly suggest changing it.

Edit: Confirmed misunderstanding. “Most of the filters’ result” is supposed to say “Most other filters’ results”

#10

You are misunderstanding the example, what I am trying to point out that, most of the custom filters to prevent is fraud messages only read the content but not the context, from that being said, even though there is no link in the message, if the filter finds a word that is in the denylist in the message, it will block the message from going through the chat system, no matter what the context is.

Meanwhile my filter does read the context (a bit), as my filter check whether there’s a link, and whether there’s a word which is in the denylist.

#11

Ah I see, I definitely misunderstood it as referring to the Roblox filter with the #### rather than other people’s filters, with both cases sitting on top of, not instead of, the Roblox filtering system.

Thanks for clarifying

2 Likes
#12

To demonstrate the problem, copy this single word by itself (triple click to highlight all, inside the text box) and try to backspace it out:

n‍‍‍‍‍‍‍‍‍‍o‍‍‍‍‍t‍‍‍a‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍s‍‍‍‍‍‍‍‍‍‍‍c‍‍a‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍m‍‍‍.‍‍‍‍‍‍‍‍‍‍‍‍b‍‍‍‍‍‍‍‍a‍‍‍‍‍d‍u‍‍‍‍‍‍‍‍r‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍l‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

You may notice that pressing backspace doesn’t work, but holding it does, very slowly and at random amounts – that’s because the word is filled with random amounts of ZWJ characters that the filter can see, but a human cannot. In order for your filter to be effective, you need to detect and eliminate ZWJ characters inside the text string.

1 Like
#13

You may want do check if it’s a valid link instead of checking if there’s peroid. What if I just would like to use a period? That really seems kind of annoying.

#14

The chance of that is rare, as the %S means any character except whitespace/space, but I’ll consider doing something like using HttpService:GetAsync to check whether the link leads to a website or not, but that will probably cause extra delay when processing

#15

Scam bots will get around this, look into string patterns and try to block links themselves, if you want you can go onto wikipedia and find an entire list of TLDs you can filter.

Make sure to avoid ones like .xxx for obvious reasons.

And to get around them using ZWJ characters, use gsub with utf8 to remove them.

Opened a github issue here

4 Likes
#16

I appreciate your work on this, but this does the same thing as all the other ones. I’m not referring to your keywords or anything like that, but the actual implementation being based off chat means that bot developers can always randomize the message or change it one way or another. This is a bandaid solution based on values that a client can very easily change rather than constants which may actually indicate a bot is in the game. If you truly want to stop bots, implementation has to be individual per game. Let’s say you have a tycoon game, don’t allow the player to chat until they load/claim a tycoon and get 1 upgrade. Perhaps you have an obby game, disallow the player to chat until stage 4. Maybe disallow chat until they complete a tutorial. There is no universal solution.

#17

Great job, I would use this if I had not already made my own. :smile::+1:

#18

The main purpose of this chat filter is to avoid scam-likely messages, not to stop bots joining from your game, or whatsoever.

Also, considering the system will determine whether the message is a scam-likely message based on whether there’s a word from the list in it and whether is there a link-likely text inside the message. It is theoretically impossible to bypass by just randomising the message or changing it one way or another.

#19

But there’s an infinite amount of ways you can express a scam message differently. You need to solve the underlying issue of bots joining and chatting in games. It’s much easier to prevent a bot from chatting by implementation per game.

if string.match(message, "%S+%.%S+") then
		-- for those who do not understand string patterns,
		-- %S means any characters without that/those character, and s is whitespace/space
		-- + %. is basically "."
		
		object.Message = "[Content Deleted]" -- too lazy to do the hashtag lolol
	end

Can’t this just be avoided by a bot saying something like “goto scamlink dot com in your browser”? A bot creator could create a character table which can replace characters randomly such as replacing “O” with “0”. A bot could also use URL encoding to trick the user to visiting a site, like saying “search scamlink%2Ecom on google for tons of bux”

1 Like