Tremendously bad memory leak on the server, what can I do?

Help and Feedback Scripting Support
, ,
#1

PLEASE READ, EVEN IF YOU CAN PROVIDE AN IDEA I WOULD BE SO GREATFUL

I’ve been working on a game for nearly a year now, I was just getting ready to release and then the server crashed while doing a long testing session. I think the server crashed due to high memory use. After an hour of searching, I’ve found the main users of memory:
1: network/megaReplicationData

image
image962×230 65.8 KB

2: netowrk/streamingSolver
image
image982×241 74.4 KB

3: internal/scriptContext
image
image956×228 77.3 KB

4: PhysicsParts/Instances
image
image955×228 69.8 KB

5: Signals
image
image976×226 76.9 KB

All of these, along with other smaller bits, cause my server to take nearly 3GB with only me in it!!

This is already a very stressful game on the servers, since it is a farming game with other 150k crops ingame at once. I have tried my best to optimize the crops/collection code, and code performance isn’t much of a problem anymore, it’s now memory usage.

My goal is to get down to around 800MB with 1 person in the server, but with the large amount of instances, I would also settle for 1-1.5GB

What are some ways I can lower memory usage in each of these areas? I’m willing to provide code samples, explain how the game works more deeply, even provide access!!

Even if a solution will take a week to implement, I’m willing to do it to get this game preforming how it should.

Things that are not the problem:

  • Connection leaks, I very carefully disconnect all connections explicitly
  • I will update this as I get more ideas/suggestions

Please help me out!! I know it’s a lot to ask since this is a difficult problem, but I’m losing hope :sweat:

2 Likes
#2

why do you have 150k crops loaded in at once? it makes no sense for the server to handle all those parts while the player is not even looking at them. also, i know that this is not related to the server but have you checked if you’ve enabled content streaming?

you could maybe use a chunk-loading system or a system that procedurally generates nearby terrain like minecraft

however, you can also check and monitor the script performance via the ScriptProfiler tab in the console. it could help you figure out which scripts are causing the most memory usage :3

1 Like
#3

Content streaming.

Also, render crops client-sided, and on the server it’s just a table of their position or a grey singular part that you’ll use for sanity checks and such.

1 Like
#4

For signals, use a module like GoodSignal. It’s very good with memory leaks

#5

If this is in studio, don’t worry about the replication solver. Else, make sure you don’t have a run service being called inside of another runservice in a module or something. That can be very bad.

#6

You might want to read more about Garbage Collection and Memory Leaks

1 Like
#7

Just adding this out there, you can use amazing modules like Janitor | Janitor, or Maid to help with garbage collection, and make your life easier.

1 Like
#8

Would Octrees technically help? Just with performance, not memory leaks

#9

I believe it should, although i have never tried myself, i have used a Quadtree before, and since an octree is a 3D version of a quadtree, it should help in various aspects in your game

#10

Hi! Just begun work for today so sorry for the slow response :sweat:
I do have streaming enabled, and I have thought about a chunk system instead, but I’m not sure how that would help memory wise.
Usually only 50k crops are loaded and interacted with on the client, the rest still exist ingame for the server though.
I’ll go look into script performance in a sec, and reply with my findings!

#11

If the server has a table of crops, and they are only rendered on the client, wouldn’t that be bad to transfer data between the two?
There are lots of players in each field, so I would have to constantly update and send large tables to the client, doesn’t seem like a good solution to me, or maybe I’m wrong!!
I doubt the crops are the problem though, they do take up a lot of instance and collision memory, but that’s about it to my knowledge.

#12

I’m not quite sure what the sciptprofiler tells me, I know it tells me CPU resources, but not memory

image
image1001×268 122 KB

#13

have you connected a remoteevent for every single crop in your game? the amount of OnServerEvent usage is not normal, like at all :sob:

if you’re doing that, please consider removing that many amount of connections and just use 1 remoteevent instead pls

#14

It is high, but its the best solution I could come up with…
I’m sending the players tool position to the server every renderstep (just reduced it to a max of 45 times a second for better performance), then the server checks if any crops are in the tools hitbox, and then collects them. I can’t just use what the server thinks the position is because it lags behind loads and doesn’t collect all crops that were in the hitbox… if that makes sense

Is there something better I could do? It is just one remote handling each collection.

#15

with tool position do you mean the mouse’s hit? (the 3d position in the world) if so, you should just do this calculation on the client everytime UserInputService.InputChanged fires, since any exploiter is and will be able to falsify this position

#16

Uhhh, the tools position is replicated automatically to the server by Roblox, no? Can you not use that instead?

By sending the tools position to the server, the remote will get there just as fast as roblox can replicate that tools position to the server. So there’s basically no point, there’d be no desync in that. Basically think of Roblox replication as firing remote events to the server.

Also, if you NEED to send the position on the remote, (i.e, the tool doesn’t exist physically, it’s a clientsided GUI) do it every 1/10th of a second, or maybe even slower. 1/40th is fast!

#17

Do you really have to send the entire table? Send the crops that need to be changed.
Either way, the table should stop existing when the function ends. I hope.

#18

There’s a very useful video on Octrees, which are basically just chunks. Octrees used by big RTS games with hundreds or thousands of AI units, which are certainly intensive to run. Minecraft somewhat uses octrees with chunks.

#19

hia so i have made some progress
using luauheap i found 1 major cause of the leak (around 500mb of 3gb) and removed it
but… now im left with nothing.
luau heap doesnt report much, and memory still grows around 1mb a second
what else can i do from here?
i wont be able to work saturday - monday due to school, but i will try to reply and answer questions!

#20

UnreliableRemoteEvents might be able to help, since you don’t absolutely need every single crop picked up to go through…
might not help with the memory leak but is a suggestion

1 Like