Does this means less hashags?
If my player types gaem will it still be flagged?
You can test the function yourself for two players to see what gets filtered or not, in case you’re that curious. They also applied it to the website and if you have a look there you can also see its less hashtags if any in general.
Good news:
This function should work in studio server/client test mode. This fix will be live by next wednesday, and only applies to the new “FilterStringAsync” method (as if you needed any more motivation to switch over 
)
9 Likes
I get it’ll cache results and all, but changing over to this is upping my filtering calling from 1 per received message to X number of players, how is this an improvement? The old one has hang time anywhere from nothing to 2+ seconds, is this one guaranteed to be a lot faster or will it have the same problems under load?
My game used to filter every chat message, and with even a small portion of a 30 player server chatting the delay caused from it being asynced could get as bad as 3 seconds. My solution around that was to filter from my own list and only use the filtering function when my own system detected bad words, but even doing that I had some pretty noticeable chat delay at times.
My current idea to keep things relatively smooth involves wrapping all my filtering and chat bounces in coroutines and attach time stamps, but then client side delivery time is inconsistent and I’d probably end up with messages popping up after ones that already arrived, which is far from ideal.
Are my concerns at all warranted or am I just over speculating? I want to keep my chat experience as smooth as possible but I’m pretty skeptical that this new filtering method is going to help me preserve what little smoothness I already have.
Even though you’re calling this new function on every player, it doesn’t actually result in any more lag time. It won’t make a bunch of unnecessary extra web requests like the old function did. Once the server already has the required filtered messages, the new function will deliver those messages instantly.
I experienced the same issues as you with FilterStringForPlayerAsync, but with the new function I tested with 20 players flooding the chat with messages and there was no lag or errors whatsoever.
I’ve also stopped wrapping the filtering function in coroutines, since only one or two of the calls will actually need to yield for a web request - the rest of the calls seem to work instantly.
1 Like
Internals-wise, it keeps the latest 30 message+sender combinations and removes them in least-recently used order. Any call with the same message+sender combination that is in this cache won’t need to do any waiting.
So I converted over to the new filtering and as I expected the chat performance absolutely tanked. In an empty place I had no problems at all but as soon as I dumped it into the game I was instantly hit with 2-3 second delays and 503 errors when trying to filter. The worst part is it’s totally on and off, sometimes it works well and other times it just totally dies.
If you want to test, click here and click the reborn server button. (it’s a bad repro but this is how it’s going to be used in game, I don’t have another way to emulate this). If you type into chat Toggle Filtering it’ll turn the filtering on/off and you can get a good idea how much lag is just from the remotes, and have much lag is from using the filtering method. If your message doesn’t go through at all it was probably a 503 error. I’d recommend spamming chat or just getting a few people in game to casually chat to see the effect it has.
Here’s my server end of the chat system if you want to peak at that. Again, I understand the importance of filtering the chat, but this kind of performance is totally unacceptable considering I can just make my own filtering functions and still filter out 99% of the bad junk, with a 99% performance increase. I wanna use this method but it’s not exactly working well for my game.
1 Like
Hey, so I’m unable to connect to your AR server with two characters simultaneously (connecting with a second always disconnects the first). Any chance you’d be able to make a minimal copy-unlocked repro with that hastebin code you linked? Everything looks fine at first glance.
In addition, does what you mentioned only happen with a large number of users connected, or should I have been able to see it with just one player?
I think you should for each player have a thread, something like:
if not filterOn or isStudio or message == "" then
data.Message = message
return chatRemote:FireAllClients(data)
end
for _, player in next, players:GetPlayers() do
spawn(function()
data.Message = chatService:FilterStringAsync(message, sender, player)
chatRemote:FireClient(player, data)
end)
end
At least you’ll be sure the second player doesn’t have to wait until the first is done.
(and the third doesn’t have to wait for the second, etc, in case the request does get redone)
1 Like
That shouldn’t be necessary with the new function, the very first chatService:FilterStringAsync(message, sender, player) should trigger a web request but the following iterations of the loop should all be instant.
But with the 2 players as arguments?
Would seem very weird if one request would be good for everything…
Yeah. The message+player source will locally cache all relevant versions of the filtered message, and the message to return is selected using the second destination player.
1 Like
To be clear, here’s an image showing what the behaviour should be. I’ve got the default sample code and measure the time a FilterStringAsync takes using time().
I’m sending 3 chats from my main character (103972519), the third message is instantaneous because the message was a duplicate and already in the cache.
2 Likes
I’m more interested in all the spotcos.
Is this a new thing of Start Server / Start Player or…
Tiny bit more on-topic:
Why did it take 2-3s for WhoBloxedWho?
(maybe the servers were slow just that moment)
(“chat servers” slowing down doesn’t seem good at all)
Actually maybe he’s doing a lot of (internal) HTTP stuff, we don’t know.
1 Like
I’ve got a couple of computers, VMs and Win10 client.
@WhoBloxedWho Let me know if the problem reappears or you’ re able to repro it.
How good is it?
a yies ui ce wut u mean leolollolol coud it uderstan dissh!t and blcki tout?
It doesn’t block any part of that string. It doesn’t seem to check for “!” being an upside down i, which seems like an obvious thing to check, but oh well.
I can’t remember where I read this (I think it was an offshoot of one of Jeff Atwood’s blog posts), but the author worked on various profanity/offense filters to make online games perfectly safe. With menu chat, players would give others their phone numbers (somehow related to game rewards or something IIRC) by spelling it out with blocks. Another thing was players spelling out profanity using furniture they used to decorate their house with. The blog listed a million instances where they tried to filter bad player communication out, over the course of a lifetime of jobs, and in the end they said they pretty much gave up. It’s impossible to catch it all. I imagine the new filter is a lot better at catching stuff than the old one though, given that it’s made by a company dedicated to making a filter.
As I mentioned earlier, CommunitySift is only on for users age 13 and younger. You have to make a young alt account to test it right now. The phrase will be blocked if you do that.
When I type the word with the proper spelling it does get blocked for my own account though? Are you guys running a custom blacklist filter on top of it?
EDIT: So if I understand correctly, it’s just the whitelist filter being replaced with the Community Sift then?