So I converted over to the new filtering and as I expected the chat performance absolutely tanked. In an empty place I had no problems at all but as soon as I dumped it into the game I was instantly hit with 2-3 second delays and 503 errors when trying to filter. The worst part is it’s totally on and off, sometimes it works well and other times it just totally dies.
If you want to test, click here and click the reborn server button. (it’s a bad repro but this is how it’s going to be used in game, I don’t have another way to emulate this). If you type into chat Toggle Filtering it’ll turn the filtering on/off and you can get a good idea how much lag is just from the remotes, and have much lag is from using the filtering method. If your message doesn’t go through at all it was probably a 503 error. I’d recommend spamming chat or just getting a few people in game to casually chat to see the effect it has.
Here’s my server end of the chat system if you want to peak at that. Again, I understand the importance of filtering the chat, but this kind of performance is totally unacceptable considering I can just make my own filtering functions and still filter out 99% of the bad junk, with a 99% performance increase. I wanna use this method but it’s not exactly working well for my game.
Hey, so I’m unable to connect to your AR server with two characters simultaneously (connecting with a second always disconnects the first). Any chance you’d be able to make a minimal copy-unlocked repro with that hastebin code you linked? Everything looks fine at first glance.
In addition, does what you mentioned only happen with a large number of users connected, or should I have been able to see it with just one player?
I think you should for each player have a thread, something like:
if not filterOn or isStudio or message == "" then
data.Message = message
return chatRemote:FireAllClients(data)
end
for _, player in next, players:GetPlayers() do
spawn(function()
data.Message = chatService:FilterStringAsync(message, sender, player)
chatRemote:FireClient(player, data)
end)
end
At least you’ll be sure the second player doesn’t have to wait until the first is done. (and the third doesn’t have to wait for the second, etc, in case the request does get redone)
That shouldn’t be necessary with the new function, the very first chatService:FilterStringAsync(message, sender, player) should trigger a web request but the following iterations of the loop should all be instant.
Yeah. The message+player source will locally cache all relevant versions of the filtered message, and the message to return is selected using the second destination player.
To be clear, here’s an image showing what the behaviour should be. I’ve got the default sample code and measure the time a FilterStringAsync takes using time().
I’m sending 3 chats from my main character (103972519), the third message is instantaneous because the message was a duplicate and already in the cache.
I’m more interested in all the spotcos.
Is this a new thing of Start Server / Start Player or…
Tiny bit more on-topic:
Why did it take 2-3s for WhoBloxedWho? (maybe the servers were slow just that moment) (“chat servers” slowing down doesn’t seem good at all)
Actually maybe he’s doing a lot of (internal) HTTP stuff, we don’t know.
It doesn’t block any part of that string. It doesn’t seem to check for “!” being an upside down i, which seems like an obvious thing to check, but oh well.
I can’t remember where I read this (I think it was an offshoot of one of Jeff Atwood’s blog posts), but the author worked on various profanity/offense filters to make online games perfectly safe. With menu chat, players would give others their phone numbers (somehow related to game rewards or something IIRC) by spelling it out with blocks. Another thing was players spelling out profanity using furniture they used to decorate their house with. The blog listed a million instances where they tried to filter bad player communication out, over the course of a lifetime of jobs, and in the end they said they pretty much gave up. It’s impossible to catch it all. I imagine the new filter is a lot better at catching stuff than the old one though, given that it’s made by a company dedicated to making a filter.
As I mentioned earlier, CommunitySift is only on for users age 13 and younger. You have to make a young alt account to test it right now. The phrase will be blocked if you do that.
When I type the word with the proper spelling it does get blocked for my own account though? Are you guys running a custom blacklist filter on top of it?
EDIT: So if I understand correctly, it’s just the whitelist filter being replaced with the Community Sift then?
So just a quick update, the web request limit is 1000 requests per minute per instance (which is similar to the previous filterstring function).
This does not mean that you can only call FilterStringAsync 1000 times per minute, but that, with correctly implemented code, you should be able to handle 1000 custom chat messages per minute (see my last post a few posts above for more details).
If you run into that limit regularly for your game (you’ll be seeing a friendly red 503 Please try again later error) please let me know
It should be fine. All Lua code is running sequentially in a single thread, so you don’t need to worry about any race conditions. You can time it to see if there are any abnormalities.
This isn’t true. Lua threads aren’t real threads but if one lua thread yields the other ones will continue running. So lua thread A could be the first caller, but if tomarty has another lua thread B that makes the same call before A’s call finishes then it would make two separate requests if it doesn’t recognize that it has one already pending.
