Server network/raknet Memory Leak?

Thank you for the quick response. I will let you know in the coming days if this improves on our end. It may be a bit as I want to ensure I report issues related to network/raknet and not other memory problems that may be caused by our end (unless you want that reported as well)

Also, I look forward to seeing the new memory-capturing system! Thank you for letting us know

I’ve noticed network/ReplicationData increases overtime. In 53 minutes, it increased ~80 MB.

Here is a before & after picture (53 minute difference). I’m not sure if its related to network though for most.


1 Like

Awesome!

Here’s our current raknet usage - it’s looking a lot more stable (although still generally high). Not seeing the big curve that we used to. I’ll set up our logging again to get an idea as to server lifetimes :slight_smile:

Thank you again for the engagement here and for pushing out a fix so rapidly. I know how challenging these things are to diagnose and our playerbase absolutely appreciates it just like we do!


edit: Update - I’m still seeing some fairly large growth - I think it’s correlated to number of users in game, but don’t have any hard proof on that.

4 Likes

Of course! Thank YOU guys for reporting. Yeah memory leaks are no fun to find, but super important to solve.

Glad you might be seeing some improvements. It sounds like there might still be some work to do though, so I’m going to keep an eye on this thread~ hopefully more devs report back. If you guys are still experiencing the problem, I will of course continue the investigation.

8 Likes

Looks like raknet is still growing, I don’t know if this is the expected memory level, this is what I got in 5 hours of runtime.

just as it seems to me, it began to grow more slowly


Also, tell me, are there any problems with UntrackedMemory? Is this growing insanely fast, is it related to raknet, or do I need to create a separate post about UntrackedMemory? Here’s what I got in 5 hours. Could this be a problem on my side?

1 Like

Hey PozziDev, thanks for checking. I’m glad it looks like a sizeable improvement compared to some of your previous reports. We’re going to continue working on improvements until the problem is fully mitigated.

As for the UntrackedMemory, this would be a separate issue. I believe there is a recent post which some other engineers (who work more closely in that area than I) are paying attention to. Please feel free to contribute there (or make another post if you feel your experience is different enough).

I would also suggest poking around some other posts in case there are solutions related to lua scripting while we work on a better system for diagnosing and repairing these types of prolems.

4 Likes

Hello!


The perpetrators are exactly the same as 6 months ago. Nothing has changed.

  • network/Raknet.
  • network/replicationcoalescing
  • network/sharedQueue
  • UntrackedMemory.

All the others within CoreMemory are pretty stable, regardless of playercount & age of server. PlaceMemory is irrelevant & so seems PlaceScriptMemory to be.

It’s now been going on for more than 6 months, with no clear end in sight.
What’s the status on this getting resolved? Is it even a priority anymore?

It’s very discouraging as a developer, having our servers crash every 1-2 hours of heavy usage, without being able to fix it, as it’s out of our hands to change.


Our game relies heavily on long play-sessions, as it takes hours to grow & survive your dinosaurs. But servers crashing constantly, interrupts out gameplay-loop intensely. We’re severely limited at this point & cannot do anything about it.

Thank you,

  • Jaco

Hey Jaco! Thanks for sharing your data. How old was your “Old Server”? We are actively investigating and also investing in the tooling to make this process more efficient. Unfortunately it sounds like our last set of patches did not help your experience, but we are still prioritizing this issue.

2 Likes

It was 12 hours old, but had only gotten above 10/100 players in the last 2 hours of its life, where it was at 80/100 for the entire 2 hours.

1 Like

Hello!

I’m wondering whether or not this is happening to EVERY game on the platform, or just a few ones.

Perhaps it only effects games with plenty of server activity & high server sizes to a point where it is noticeable.


When our servers have 10/100 players on them, they last up to 2 days, but only a few hours at 80+/100. As playercount in each server gets higher, the memory leak becomes exponentially worse. That may be why only a few developers are noticing the issue.

I’m also starting to doubt whether or not it is a Roblox issue at all, considering you have published so many patches to attempt to fix it. Even with all our own memory loggers & other tools showing no leaks, could the memory leak not still be of our own creation?

Thank you,

  • Jaco
2 Likes

correct me if im wrong but what if you just set the values to nil after using them so it doesnt hold the references to them forever?

function FireClients(clients, event, ...)
   for _, client in pairs(clients) do
      event:FireClient(client, ...)
      task.wait() -- artificial delay
      client=nil
   end
   event,clients=nil,nil
end

please let me know if this works or not

1 Like

It didn’t affect anything-----

  • Setting a local variable or a function parameter to nil doesn’t free the memory or dereference the object it’s pointing to. Lua’s garbage collector will automatically deallocate memory for objects that aren’t reachable anymore.

  • client=nil within the loop is unnecessary. It doesn’t impact the actual clients table; it just breaks the reference between the local loop variable client and the current table item.

  • Similarly, event,clients=nil,nil at the end of the function is redundant. Once the function execution completes, local variables (including function parameters) go out of scope, and if there are no references to the objects they pointed to, they are eligible for garbage collection.

Hope this clears that up!

Hey guys - just wanted to pop in to say we’re still working on something internally which will help us narrow down the problem. Thank you for your patience!

As for the question regarding the types of experiences this affects, it seems likely this problem is exacerbated for experiences with many concurrent players for two reasons: these servers tend to live longer with constant player activity, and these servers serve exponentially more traffic.

10 Likes

I want to mention this might be Player instance related issue!

I noticed that when a player leaves the game connections like .CharacterAdded are still connected

to fix this you have to do something like this:

local Players=game:GetService("Players")
Players.PlayerRemoving:Connect(function(p)
	if p.Parent==Players then
		p.AncestryChanged:Wait()
	end
	p:Destroy()
end)

after calling “Destroy” on Player my .CharacterAdded connections get disconnected

When Luau destroys an event’s object all of its connections disconnect automatically ~

How did you come to the conclusion that the event was not disconnected? If you stored the connection in a local variable, that variable may not be nil but the connection should be disconnected

1 Like

When Luau destroys an event’s object all of its connections disconnect automatically

correct but what im trying to say is that when player leaves the game all of connections related to that player (like .CharacterAdded) are still connected

this can be fixed by connecting to PlayerRemoving and then calling :Destroy on that Player instance just to disconnect .CharacterAdded

local Players=game:GetService("Players")
Players.PlayerAdded:Connect(function(p)
	local connection=p.CharacterAdded:Connect(function()
		
	end)
	Players.PlayerRemoving:Wait()
	p.AncestryChanged:Wait()
	print(connection.Connected,p.Parent)
end)
game:BindToClose(function()
	task.wait(2)
end)

Zrzut ekranu (124)

so in order to fix this

you have to add this few lines of code:

Players.PlayerRemoving:Connect(function(p)
	if p.Parent==Players then
		p.AncestryChanged:Wait()
	end
	p:Destroy()
end)

I thought this was expected behavior? I have noticed for a while that this has been a feature/bug for a while now with all sorts of instances if I’m not wrong. Like the fact that Humanoid connections stay alive even after the character is destroyed.

yeah thats what probably causes the memory leak (connections are not supposed to stay alive when Instance got reparented to nil! example: player left the game and instead of destroying player roblox engine just parents player to nil and connections related to that player stay alive)

I think that its not expected behavior correct me if im wrong

1 Like

also one thing

roblox engine does same thing to player.Character (just like when player leaves the game)

instead of destroying it its just gonna reparent it to nil and that means humanoid connections will still stay alive (grrr roblox should fix this and start calling :Destroy on those instead of just reparenting to nil)

EDIT: yeah looks like im probably correct with this one

im not the only one who figured out about this stupid memory leak

1 Like