Every time a new unique character mesh loads in it leaks memory (200-300 MB)

Reproduction Steps

There’s still an existing bug where we can get leaks when new avatar appearances are loaded. The original thread can be found (here)[Every time your character dies it leaks memory (up to 500 MB, probably infinite)], where dying would lead to memory leaks. Now, loading character appearances still leaks memory. The reproduction can be found (here)[Die every second - Roblox].

  • Occurs with R15 (see my avatar), but reproduces with anyone’s avatar, including custom avatars
  • Occurs on both memory constrained and un-memory-constrained platforms
  • Occurs consistently

Reproduction steps

  1. Join a game
  2. Change your characters outfit using some preset character outfits (you can also have new players join the game, which is the actual issue)
  3. Reset your character
  4. Expected result: Memory stays relatively consistent
  5. Actual result: Memory is leaked per a costume (Here’s the result of 10ish costume changes, or 10 player joins).

image

You can also reproduce this by joining this game and clicking around your preset costume outfits OR you can also use the “Randomize costume” button which will randomize your costume.

Note: Eventually the reproduction levels off since it’s not loading more hats, but with a better list of Ids, I’m sure I can get it higher.

image

Effectively, every time a new player joins the game, it leaks memory if their outfit is different. Reproduction in live looks like this:

  1. Join the game
  2. Have other players join the game with different outfits
  3. Memory leaks slowly but consistently over time

Expected Behavior

I expect that the memory should not leak when a new character appearance loads, and then gets unloaded. Players loading with different appearances is a very consistent thing in Roblox and should not leak memory.

*Fixing this bug is important because this leads to crashes on many low-end devices in games with any amount of player joins

Actual Behavior

The memory appears to permanently leak per a character outfit.

You can see a video of this here:

Issue Area: Engine
Issue Type: Performance
Impact: High
Frequency: Constantly

5 Likes

I’m pretty sure I’ve observed that assets loaded on the client (meshes, textures, etc.) are never actually unloaded (because otherwise the client would somehow need to know if it ever needs the asset again, and if it guesses wrong it needs to redownload the asset potentially many times). Maybe that same behavior leading to this effect.

2 Likes

Wait, is this not just model/asset caching for things so they don’t need to load again? This sounds intentional.

Correct me if I’m wrong as I have some doubts I am right on that. If I am right; there definitely is a problem with the current system of caching assets for later use.

Does this happen with changing a character’s outfit’s scaling (which should not affect memory at all)? If someone can manage to crash a server by changing their scale using the Roblox avatar editor and resetting; this is a massive problem. Heck, someone can create alternate accounts that join off each other and just keep having new outfits to slowly add onto memory and crash the server.

4 Likes

This shouldn’t actually be an issue;

Apparently the engine will not opportunistically unload resources, A.K.A. it’ll only worry about resources that aren’t being used if it’s getting too close to it’s memory limit.

If it’s well under it’s memory limit, then it is sort of wasting network usage by constantly re-downloading assets. The first example that comes to mind for me is CS:GO or TF2, which have “texture streaming” options. It’d be incredibly annoying to see textures that you’d expect to be cached to some extent constantly be blurry if you haven’t seen them for a few moments.

2 Likes

Hm, yeah, it probably is just optimistic meshes not being unloaded. I’ll need to test under more memory pressure.

1 Like

Hi Quenty!

Thank you for the excellent repro case! I was able to jump in and look at the issue quickly and directly.

I can confirm we are seeing cache behavior here. The test case is small, so it takes a while to see the various caches all even-out but they do. I’ve got some internal tooling to flush the caches and yep, the memory dropped back down to where it was during the first few cycles. Reminder: it’s not only meshes. LUA, network messaging, the instance count maintenance all have independent cache budgets, garbage collection periods, and retention behaviors so summarizing overall memory usage is a tricky business!

Let me know if you have any new observations or test cases because so far I’ve not been able to find any legitimate leaks. My takeaway is that Roblox memory tools would benefit from a “flush all caches” button, even if it merely serves to remind the user of caches existing and give that whew! feeling of seeing the memory come back down.

7 Likes

Hello, I’m not sure if this issue is related but my game’s (Murderers VS Sheriffs Duels - Roblox) servers are crashing after a few hours of being up, and in the console I see “Animation” memory steadily increasing and never going down.

In a new server you can see the “Animation” memory is low (~6MB) but is steadily increasing:

In an older server you can see the “Animation” memory is much higher(~191MB) and is steadily increasing as well:

Update:
Here are both servers only ~40 minutes later:
~6MB to ~116MB

~191MB to ~321MB

3 Likes

I’m not an expert on this, but according to the Developer Console documentation:

It might be that you are creating tons of Animations or AnimationTrackss (created through :LoadAnimation) instances.

It could be that you are calling :LoadAnimation every time you want to play an animation instead of calling it once, leading to a build-up of AnimationTrack objects.

1 Like

wouldn’t that affect the client’s memory? I’m pretty sure animation tracks would be destroyed when the humanoid is destroyed. Also, I only call it once per humanoid for each animation. Also, the instance memory is increasing as well so I think its prob related to this issue Significant memory leak on blank baseplate when resetting character

2 Likes

Historically playing animations seems to leak memory, although a little bit at a time. We tend to try to cache loading animations so we only do it once per a character, which definitely helps.

I should probably file a bug report, but it’s a lot of work.

3 Likes