Roblox's core scripts are memory leaking on the server

Hello,

So Roblox’s System allocation goes from 684MB to over 1.3GB over 24h… I’ve provided a video that shows the graph and a picture of the detailed “System” graph


Related game: [UPDATE] Bot Commander - Roblox

* On an another note, the engine/core/react/rodux/roblox scripts also creates a lot of errors/warnings that I see in the error report, but they’re all on the client side so i’m like 99% sure they’re not related… I can provide them if needed

1 Like

Hi there,
thank you for reporting this - we are looking into it!
In the meantime, you could create a Luau heap usage snapshot in Dev console, in order to narrow down where the memory seems to be going Memory usage | Documentation - Roblox Creator Hub
Cheers!

Hello, I included the “first page” of all tabs, if you want me to dig into any specific one, let me know and I’ll provide more details

Graph:

Object Tags:

Memory Categories:

Object Classes:

Unique References:

Thanks

Thank you very much - this helps a lot.
I’ll post an update as soon as we have any.
Cheers!

2 Likes

I apologize for the inconvenience, but could this issue be investigated too?Unreasonbly high RAM usage on roblox mobile client
The client seems to just saturate itself with RAM and waste ressources, even if the client doesn’t need that much, this happens as early as the homepage (the memory leak issue stated by the OP may also be slightly related to this one)

After taking a look at the information you have provided, we did notice an issue on our side with too many separate Actor VMs being allocated.
We have now fixed the number to correspond to the CPU core count correctly.

Aside from that, it seems that you are not cleaning up data of the players that are leaving your experience.
Joining an instance with only 9 players shows a lot more of them being referenced from your scripts, preventing collection.
Consider cleaning up signal connections more carefully using Memory usage | Documentation - Roblox Creator Hub and if it’s compatible with your experience, enabling New Player and Character Destroy Behavior

Also note that in the graphs you show, memory reaches the highest point at 6h and does not increase over time as dramatically after that, so it might also signal that your experience has reached the steady state.
Server memory limit starts at around 6GB, so your experience is far from hitting it.

If you have made the steps to fix memory leaks in your scripts and you believe you have information on how core scripts keep leaking memory, please provide addition information.

1 Like

I have a quick question regarding what good practices we should be keeping about signal connections. It was my understanding that signal connections are automatically disconnected and cleaned up when the instance they are connected to is :Destroy()'d. With the new player/character destroy behavior turned on, it seems like the players and characters are being :Destroy()'d now when the player leaves/respawns. Would that not automatically disconnect signal connections tied to the player and their character?

Yes, Destroy would disconnect them all, so it can be used instead.

New Player and Character Destroy Behavior is implemented today by engine calling Destroy (server-side).
The details of this auto-destroy might change in the future. We are considering changing it to only disconnect all signals while keeping the instance tree intact as some developers expressed their need to explore that data.

2 Likes

Hello! Thanks for your reply!

Thanks, I identified the memory leak on our end, I’ll try to fix it asap to see if it helps with the memory issues *

This is already enabled, it’s a really good feature

If I’m not wrong, you’re referencing the “Total” graph, in the “System” graph it consistently went up over 18h, where it stabilizes (in last 7 days, it just goes up for the full 24h and is still the PlaceMemory:Navigation that is growing the most)

EDIT:
* Actually, despite having a memory leak on my end, PlaceMemory:Navigation shouldn’t be affected by it, your actor fix will definitely help with memory usage as well, but still won’t affect navigation (I think)

EDIT2:
I fixed a memory leak that affected Instances System memory, the one that keep references of players in memory more complex to fix. We can still see Instances grow by only 24MB (90 to 114) while Navigation grew by 145MB (45 to 192, and I’m being generous here, I started from 1 hour in.. Navigation starts at 2MB otherwise and instances 80MB)

Reference point of the last 4 weeks:

Thanks for reporting this issue.

We’ve received a report of a suspected memory leak related to the game’s pathfinding. We’d appreciate it if you could provide some more details to help us investigate.

  • When did you first notice this issue occurring?
  • Can you describe how you’re using pathfinding in your game?
    • If you’re using it for NPC navigation, how often do you call path:ComputeAsync for a single NPC?
    • We’ve noticed a large number of PathWaypoint objects in your reported Object Classes. When you re-compute a path, are the previously generated Waypoint objects still being referenced?
    • When an NPC is removed from the game, are its corresponding Path and Waypoint objects still being referenced?

Hello!

When did you first notice this issue occurring?

We noticed it about the same time I opened this ticket, we were investigating the reason of random server crashes and we ended up stumbling onto this

Can you describe how you’re using pathfinding in your game?

We’re using it when we detect our bots are stuck (speed < 3, distant from intended position and no LoS). At the moment we’re using the SimplePath module for our pathfinding needs (it was done in an effort to potentially address an issue we have with pathfinding.

If you’re using it for NPC navigation, how often do you call path:ComputeAsync for a single NPC?

We call it in the scenario mentioned above, if my understanding of the SimplePath module is correct, it would be computed every 0.07 seconds until the LoS of its intended position is acquired back

We’ve noticed a large number of PathWaypoint objects in your reported Object Classes. When you re-compute a path, are the previously generated Waypoint objects still being referenced?

So, since we use SimplePath, if my understand of it is correct, the waypoints are not cleared until a new path is calculated, in which case it overrides the old waypoints (self._waypoints = self._path:GetWaypoints()).

When an NPC is removed from the game, are its corresponding Path and Waypoint objects still being referenced?

I don’t do it manually, but each bot’s logic is handled via a script parented to the bot, once it dies, the script is Destroyed, which if I remember correctly, should clear all references within it? (Not sure if it would apply to the SimplePath module?

I can send you the code in private (or any other way) if you think it can help

SimplePath will memory leak the Waypoint objects if you do not call :Destroy on it as it connects events to the PathWaypoints (self._path.Blocked:Connect(function(...)), those connections will hold a reference to the waypoints even if the variable is overwritten (because the instances are never Destroyed so the connections are never disconnected), hence your memory leak with pathfinding. SimplePath hasn’t been updated since 2022 so it has no active maintainer hence the fact no one thought of this issue.

This probably also explains your high Signal memory leak that was stated earlier in the thread.

A fast but inefficient fix is to call SimplePath:Destroy whenever your NPCs either die or are de-parented from Workspace, the proper fix is to call SimplePath:Destroy after completing the path.

Sorry for late reply, I totally forgot to hit send.

I will try the SimplePathAgent:Destroy() method when the bot dies, I didn’t know this was something with SimplePath, the game will be updated next Thursday and I’ll monitor it for any changes in the Navigation memory usage

Has the memory use improved after you made the latest fixes?

hello! Sorry I forgot to reply with the update!

Here’s the graph, it still seems like the navigation is still leaking despite being cleared properly on NPC’s deaths. It even seems to be using more now that im destroying stuff than it did before?

  • Events are destroyed, which should automatically unbind them
  • Path is destroyed
  • Everything else set in the SimplePathAgent is explicitely set to nil (Including waypoints)

I also started cleaning a lot more data when the NPCs dies and when it gets destroyed, making sure stuff are cleaned. When an NPC dies, the script referencing all those things is Destroyed as well

Some Luau Heap details Compared to my initial report, Unique references and Unparented Instances are now both empty.


EDIT: I will try to completely ditch SimplePath in the next update (will be on the 28th) since SimplePath didnt fix the issue we had which was pretty much its sole purpose. I’ll send another update with new graphs 3 days later (I’ll set a reminder this time to make sure I don’t forget)

Thanks for the update and the graph!
Using FindPath creates NavMeshes for the searched area; those NavMeshes are expensive to generate and it remain in memory after creation for now.
Because NavMeshes persist, clearing NPCs or nil-ing SimplePath data won’t bring Navigation memory back down to pre-navigation levels. Memory will grow as the area/complexity of FindPath calls increases. From your graph I can’t definitively say there’s a leak.
If you can DM me the place file, I’ll be happy to look into it. If possible make sure the issue is reproducible by running it in Studio → Run.

Could you also send me the repo for fix the issue we had? This is the associated private message .

Hello, thanks for the answer!

Currently we have a set of different part maps, and with time, we’ll implement smooth terrain maps exclusively. Are the NavMeshes regenerated if the map is removed and reinserted later?

For maps, we dynamically load/unload the map using clone and inserting part-by-part (in short: for _, Part in Map:Clone():GetChildren() do Part.Parent = workspace.Map end) and for terrain maps we also Load the terrain then clear it at the end of each round.

So on paper, its always “new terrain and parts” that are inserted, can the engine understand they are linked to an existing NavMeshes?

To run directly from the Rbxl file I’d need to make changes to the games logic, I’ll DM you the place file as soon as the changes are done

You could actually use the same place for that as well, if its alright? Otherwise if you prefer, I can make a MRE place instead

The navmesh manages the world by dividing it into 64x64 tiles on the XZ plane. When parts or terrain within a tile change, the corresponding navmesh for that tile is deleted and a new one is generated. This process involves releasing the old memory and allocating new memory. We plan to add a feature in the future to remove navmeshes from tiles that haven’t been used for a long time. However, for now, once a tile’s navmesh is generated, the memory remains allocated as long as the parts and terrain within it don’t change.

Currently we have a set of different part maps, and with time, we’ll implement smooth terrain maps exclusively. Are the NavMeshes regenerated if the map is removed and reinserted later?

Yes.

So on paper, its always “new terrain and parts” that are inserted, can the engine understand they are linked to an existing NavMeshes?

Yes.

You could actually use the same place for that as well, if its alright? Otherwise if you prefer, I can make a MRE place instead

It’s nice to use a separated place. It seems like stuck to a wall issue could be reproduced in a simple/small place.

3 Likes

Hello!

So it’s been couple of days since our last update and we removed SimplePath entirely in favour of a manual pathfinding implementation. Navigation graph seems to be even worse than before. (Now goes about 50 higher than before, we have a new map, so might be that if nashmesh are not removed?)

Our code in case it's on our end
Target = FormationsHandler:GetTargetPosition(Formation.Value, IsLeft, Position.Value, OfficerTorso)
local Success, _ = pcall(function()
    PathfindingAgent:ComputeAsync(Torso.Position, Target)
end)

if not Success or #PathfindingAgent:GetWaypoints() == 0 then
    task.wait(LOOP_TIME)
    continue
end

for _, Waypoint in PathfindingAgent:GetWaypoints() do
    Humanoid:MoveTo(Waypoint.Position)
    Humanoid.MoveToFinished:Wait()

    if HasLineOfSight(Target) then
        break
    end
end

And it’s cleaned up like this:

Humanoid.Died:Once(function()
    PathfindingAgent:Destroy()
end)
Bot.Destroying:Connect(function()
    -- We make sure
    PathfindingAgent:Destroy()
end)

The navmesh is not automatically removed after it’s generated. It’s only removed when a part or terrain in the nav tile changes, but it’s immediately replaced with an updated navmesh. Because of this, you won’t see a reduction in memory over time.
Instead, as the pathfinding range in your game expands, the memory usage will increase accordingly.

1 Like