Anti-Toxicity Filter v2

Anti-Toxicity Filter v2


Error has been resolved.


About
Previously, I have (attempted) to make a script which stopped people from making toxic comments in games, such as “ur a noob” or etc. Of course, this didn’t stop people from continuing to make these such comments. Since then, I have improved it more.


FAQ

Why should I implement this?
This new version of my previous anti-toxicity filter is not only faster, but detects toxic comments more accurately then the previous version.


Your code is scuffed. Please rewrite.
Make your own. Use my API if you want: https://toxicity.plight.digital/


What are the use cases?
For example, games like “Da Hood” or FPS games are primary targets for toxic behaviours. If implemented, this could decrease it by a lot.


What does it run on?
Cloudflare Workers. Which is what makes it so fast.


Will you ever make it go private?
Maybe. If it gets enough traction, I may have to shut it down and make people sign up for it.


How do I change the sensitivity?
Set the MaxScore to a higher (making it less sensitive) or lower (making it more sensitive) number.


Why can I still see the message?
Either the API broke, the code broke or, due to the change to TextChatService, you can still see the message but others wont.


How fast is it?
I would say around 500ms, you can test it if you want.

Code
local TextChatService = game:GetService("TextChatService")
local Players         = game:GetService("Players")
local RunService      = game:GetService("RunService")
local HttpService     = game:GetService("HttpService")
local Channels        = TextChatService:WaitForChild("TextChannels")

local TextClassifier  = "https://toxicity.plight.digital"
local MaxScore        = 0.7

local function isMessageToxic(message: string): boolean
	if message == "" then return false end
	
	pcall(function()
		response = HttpService:PostAsync(
			TextClassifier,
			HttpService:JSONEncode({
				text = message
			}),
			Enum.HttpContentType.ApplicationJson
		)
		response = HttpService:JSONDecode(response).response
	end)
	
	if response.Error or not response then
		print("There has been an error.") -- err code?
		return false
	end
	
	if (tonumber(response[1].score) or 0) > (tonumber(MaxScore) or 0.7) then
		return true
	else
		return false
	end
end

local function channelAdded(channel: TextChannel)
	if not channel:IsA("TextChannel") then return end
	channel.ShouldDeliverCallback = function(message: TextChatMessage, speaker: TextSource)
		if isMessageToxic(message.Text) then
			print("Message not sent.")
			return false
		else
			print("Message sent.")
			return true
		end
	end
end

for _, channel in pairs(Channels:GetChildren()) do
	channelAdded(channel)
end
Channels.ChildAdded:Connect(channelAdded)

You can test it here.
In case you didn’t read the FAQ, the link to the API is here.

9 Likes

Can you please let us know more about your API, privacy policy, and practices?

It seems like http://plight.digital/ and https://toxicity.plight.digital/ don’t even have a website (red flag for me).

How can we trust you to properly handle having all our player’s messages and making sure you do not store, sell, or share such data?

2 Likes

Hi,
I’m still setting up the host for my website, https://plight.digital, and I have school, so I don’t have much free time to do that currently. I’m working on a privacy policy as we speak. So far, I don’t sell or share data and I don’t knowingly store them for training purposes (unless Cloudflare stores the data, although no player data is sent to the API, so it would be completely anonymous). No data is stored as of now, and no player data is sent to the API. You can read the script if you want.

If you want to read Cloudflare’s privacy policy (considering I host on Cloudflare Workers), you can find it here

1 Like

I don’t quite understand, how exactly is a developer supposed to know how your filter works
and are slurs flagged as well?

And how is the behaviour if I for example say I am such a noob

What exaclty was improved?

2 Likes

Hi,
It can be tested at sites such as https://reqbin.com or the program Postman and such, and you can also play the game (provided you have one other person to test it with). Slurs should naturally be filtered by ROBLOX’s text filter, as this mainly deals with toxicity (although should prevent it should going through).

It will display the chat message on the client of which sent the message.

Faster classification (thanks to Cloudflare workers), and better accuracy.

1 Like

Does your API can handle the large number of traffic?
Because its gonna be slow if theres alot of requests…

Well, doesn’t roblox tags toxicity for you?

if you are using cloudflare doesn’t it gonna block roblox if it gets alot (like alot) of requests since players likes to spam the chat and such? because same thing happened with discord.

2 Likes

Yes, it should be able to, considering it’s on Cloudflare’s infrastructure

No.

The script should, if Cloudflare does block ROBLOX IPs temporarily, just let the unfiltered messaged through.

1 Like

This feels like you just put Perspective API or a similar model in a wrapper, soon to be faced with ratelimits and expenses.

Also, given the long roundtrip (500 ms), you should use such APIs after sending the message, otherwise chatting will feel really slow.

3 Likes

Does your endpoint contain of premade messages a user could say?
If so, wouldn’t it be better to match string and/or modify a message in a way to get rid of a bypass

For example a user says nooooooob, I haven’t tested but it might go around the filter
And you’d modify and check for multiple characters in a row, ex: slice it down to where it would only say noob?

Im curious, since it does seem like it only matches whole text messages

1 Like

Roblox has this built-in, though. You know, the ####ing chat filter?

2 Likes

Are you sure about that?

image

(I copied bluebxrrybot’s reply to this topic and replaced the hashtags with a swear word and this came up)

1 Like