7,000 CCU game experiencing major server instability

Our game, Animations: Mocap had 7,000 CCU. Players were reporting severe connection issues so we took the game offline for about 15 minutes to see if that would fix the issue to no avail.

We only have 800 CCU at the time of writing, and a poll in our Discord server (a small subset of our community) shows that a lot of people are unable to connect, and those that can are having various issues

It is unlikely an issue on our end as the only changes made were related to chat modules. We started recieving reports of these issues today, a few hours ago.

Error logs and reports sent by players

https://media.discordapp.net/attachments/726161212517122190/748985425355604108/unknown.png?width=744&height=684

  • Players can connect to a VIP server after a few minutes and the server seems to run fine
17 Likes

We reverted to a version from 22 August and restarted the servers via migrate.

It did not fix the issue.

I’ve been in the server for a solid two minutes now and only a few things have changed (mainly animations playing), everything on the server is just frozen, I can’t even view the server log.

2 Likes

I quickly tested on Adopt Me, everything there works fine. You can also see from a Discord server poll that this is a largescale game-specific issue impacting servers.

Please do not reply for the sake of replying, we have already exhausted most common solutions.


With that said, the player count appears to be recovering, but most of the servers are marked as ‘Slow game’ and our players and staff are reporting major lag issues.

EDIT: Nevermind… Just dropped 400 players right now.

Please do not continue replying if you are going to ignore everything I’ve said and the OP. These issues appeared without an update and reverting to a previously working update does not solve the issue.

Moreover, this is a front page game; we are absolutely certain there are no backdoors as all of the code was originally written and is hosted on the game page.

4 Likes

Make sure that your original code is optimised and not using stuff like while true do when the client is loading, I tested 10 front page games and none of them have your issue of not loading quick.

(and before you reply with ‘Please do not continue replying if you are going to ignore everything’, I have read everything you said.)

1 Like

If you had you will know that we’ve tested a (little under) weeks old version which previously had no issues, and the issue persisted.

1 Like

If you had read what I said, I said to make sure your original code is optimised and that I read everything you said.
But if the issue was not a thing in your week old version (when it first released) then you probably have a plugin with a backdoor or causing the lag.

3 Likes

No plugin changes recently on myself, likely the same for the 1-2 other people with game access. Also reverted from the web page rather than studio, meaning plugins would have had no opportunity to do anything.

Now that you sent pictures of the console,
game.Players.LocalPlayer.PlayerScripts.ChatScript.ChatMain is your problem.

What I mean by your problem is that the script is causing the roblox client to freeze and after a certain time it causes one of roblox’s protections to fire and cancel the script from running giving the error and letting the game persist

Strange that rolling it back didn’t help. I’ve had the same errors your players showed before when I made a change that tried to do too much without yielding.

Depending on what the changes were to the chat script, consider adding some additional ways for it to yield.

An example could be trying to load the entire chat history since the start of the server into a GUI, including some expensive tasks, without yielding at all.

As you mentioned though, this should have fixed with the rollback if you published the old version from studio and shut down all the servers afterwards.

4 Likes

Did you audit your code for this change?:

1 Like

Yes – our client source is based from StarterPlayerScripts.

Players are still reporting connectivity issues and our player count is still horribly low compared to the same time yesterday.

1 Like

@fovvss restarted the server after adding some new content to the game (not script) and players are reporting similar issues.

1 Like

Thanks for the report! We’ve filed this internally and we’ll follow up here when we have an update for you.

5 Likes

I am in the process of checking over bug reports and following up on some bugs that haven’t received any activity in a while.
Is this issue still occurring or can you confirm that this bug has been resolved? We are having a hard time reproducing this issue

The issue just randomly stopped happening sometime in October.

Feel free to mark this bug as resolved.

1 Like

Thanks for the info. This issue should now be resolved! If this issue is still occurring, please create a new topic for us to look into.

2 Likes