Servers Keep On Crashing, Cause Unknown

Hi everyone,

I’m the Lead Developer at Club Lumi, and we’ve been encountering a frustrating issue recently with our servers consistently crashing.

This has only started happening in recent times, and we’ve noticed that the main server will randomly show it is “updating,” even though none of the developers have published any new updates during those moments.

I’ve thoroughly reviewed all of my plugins and checked to see if there is any malicious scripts in the game, and including tools! Everything is safe and free from malicious scripts.

I have also looked at our analytics page and there is also nothing that would make the servers randomly update!

We’ve taken every precaution on our end, but the problem persists. I’m posting here in hopes that someone may have experienced a similar issue or that Roblox staff could shed light on what might be causing this unexpected behaviour.

3 Likes

Can you show the micro-profiler? I have a feeling it’s poorly written scripts that is crashing the game.

1 Like

To be honest, I don’t think that it is my code, as the servers just randomly update! Here is the micro profiler
image

1 Like

It would be nice to see the profiler as the server is going through a crash. I understand how unfeasible that is to do, though

Its probably just a single script somewhere causing this issue. Is it a roblox update message, or a custom implemented update message?

1 Like

The crash only really happens when we have a full server so usually on a weekend, when we have about 50-70 people in one server

1 Like

We do use a custom updated screen, and its that. The custom module we use is Quenty’s soft shutdown

It looks like that module automatically detects server shutdowns and performs the soft-shutdown behavior, am I correct?

If this is the case, and you are not manually shutting down the servers anywhere in code, there is a script (or maybe a culmination of many scripts) causing your server to crash.

It could be memory leaks, it could be quadruple nested for loops with no waits, it could be anything. We can’t really detect the source of these crashes without knowing more about your setup

actually, take a look at this: Servers shutting down on their own - Bug Reports / Engine Bugs - Developer Forum | Roblox

This could be exactly what you are going through

We do not have any looped scripts without a wait in them, The only looped script we have is the levels script and that waits once every 25 seconds before giving out a level and currency to players.

Yes that is correct, it detects when the server is BindToClose Then Functions.

1 Like

Here is the issue you are running into:

We have made a recent change that will now shutdown the server for an experience if it runs out of memory. Previously when servers ran out of memory they would “hard-crash”.

So your experience is running out of memory. It’s probably a memory leak issue. They made it so that servers perform “soft-shutdowns” invoking BindToClose when this happens.

Ensure you are properly using Disconnect() on connections where they are no longer needed, and use Destroy() to properly destroy instances. The most common memory leak is creating new connections and never disconnecting old ones

1 Like

Also maybe you to further narrow down this issue, could you possible add a webhook and detect when the server crashed? Just make a request and post it with a few stats like current amount of players, uptime e.t.c if the uptime is long (or even short) you most likely have a memory leak that is unknown to you.

An example could be a simple memory leak like a table holding all the players, when the players leave you never remove them from that table and set their instance to nil causing a memory leak

I have looked into our analytics and we are only currently using 27.13% of memory according to roblox this should be below 50% in their Identify performance issues documentation. If this is really the issue then I would speak to the development team about cutting it down.

If it is possible, you could try and snoop other games and see how their memory usage is. Go from club games (your genre). To Battle Games. To other complex games that you think will have more memory usage. Then you can get the average and then talk to your development team.

1 Like

I would say it is your issue. Memory leaks are tough to spot because they happen over time - and/or in circumstances you cannot normally test (70-80 players).

I can almost guarantee that a memory leak exists in your game, because the only other explanation would be that you are running A LOT of code per-player all the time. It’s not easy to do that second part on accident in my opinion, that’s why I’m pretty sure it’s a memory leak.

A good test for a memory leak would be to play in a well-populated server, and take notes of the memory usage over time. If the memory usage increases over time while the player count remains relatively steady, you can confirm the memory leak

1 Like

Script memory’s are actually the least looking into this its actually mostly building stuff, moving parts, physics parts! Thank you for your help I will send this stuff to the development team!

I will also look further into this with populated servers, as we don’t have a 3d modeller for building, most of our build is made from mostly parts!
:smiley:

You should still be fine using roblox parts but just to let you know, if you are animating these parts of the server that can create HUGE memory usage and cause huge performance lost! Make sure your development team is running these on the client through a remote event to determine which State should be played (Moving Lights, Dance Floor, e.t.c).

Also, if you could further narrow down which parts are causing the most memory usage, they most likely can be optimized. But first start with verifying your development team is handling animations, on the client. Then go from there.

I see the issue now, other developers in the development team have put in moving parts for their build without my knowledge, and it has been messing with memory issues thank you for helping :smiley:

1 Like