Memory Leak - Client not releasing memory used by textures even when all references to the given texture have been removed

[missmudmaam]

Reproduction Steps

Context
My brother (mrmudman) and I are in the process of creating a mega mall shopping / avatar dress up game which will contain a large amount of humanoid mannequins, each displaying both 2D and 3D clothing and accessories.

There are multiple ways of reproducing this issue, listed below.

Using a decal

1. Create a baseplate game in studio
2. Insert a new decal instance
3. Enter play mode
4. Load a unique texture into this decal (a texture that has not been previously used within the place)
5. Remove the texture from the decal, leaving the decal instance

Changing avatar look / clothing

1. Join a game using an avatar that has clothing
2. Change the clothing worn on the avatar through the site customization tools.
3. Reset character to load this new appearance
4. Every unique combination of clothing items worn adds approx 1mB - 2mB of memory usage
   Observing another player performing this sequence of actions will also cause this issue

Using a MeshPart with texture (both embedded & non-embedded)

1. Create a baseplate game in studio
2. Insert a meshpart with a texture into this place
3. Enter play mode
4. Remove the meshpart & associated texture from the client

These are only a few examples of how to reproduce this memory leak, there are a multitude of other methods to also reproduce this. Everything that has a texture will cause this issue.

Expected Behavior

When a texture is no longer in use on the client (referred to by any texture instances or in any other way), it should be removed from the client’s memory.

Actual Behavior

The texture stays loaded within the client’s memory, even when no longer in use on both the client AND the server. This means that when a new texture is sent to the client, it will permanently be added to the client’s memory. Without any means to remove this memory usage, the client’s memory will rapidly increase as more textures are sent to it, leading to crashes on low end devices.

Memory usage graph - texture was inserted then removed 5 seconds later

Memory has not decreased even though texture was removed

Texture in decal & using GraphicsTexture memory

Texture removed from decal & still using GraphicsTexture memory

This memory leak imposes a hard limit to how many textures can be implemented into into a place in total.

Every game that allows users to utilize their avatar’s appearance is susceptible to this memory leak.
Every game that uses textures temporarily is susceptible to this memory leak

Further related issues discovered
Using the client to join a new game while not closing the executable does not unload the textures used by the previous game from memory
Here is the same place, with the 1st screenshot showing the game opened directly from the website and the 2nd screenshot showing a memory increase after transferring from a different game without closing the executable

Issue Area: Engine
Issue Type: Performance
Impact: High
Frequency: Constantly
Date Last Experienced: 2022-08-27 00:08:00 (+10:00)
A private message is associated with this bug report

[Meowsionist (Meowsion)]

This is true and I also reported this bug to #bug-reports, I didn’t know it was due to textures and decal memory never being cleared.

On some Roblox Studio sessions I was hitting up to 5k mem usage, but when you close out and go back it it’s back to 2k.

This is still a very serious issue that needs to be resolved.

[GamEditoPro]

Probably, this to faster re-load texture. So if you delete it, and then, after half of hour somewhy decided to return it, it will load back momentally, instead of loading again.

[mrmudman]

This is a possible reason for this behaviour, but overall, I think it would be more beneficial to have to manually preload a texture to get quicker responses than to have the texture permanently occupy memory. If this was changed, it would be possible to make significantly bigger or more visually appealing places through unloading textures that are nowhere near being able to be seen by the player.

[missmudmaam]

Also concerned as how this affects streaming enabled. When a texture instance is streamed in, it now permanently occupies a space in memory, even when it is eventually streamed out again.

1. This means that the “Low Memory” option for streaming enabled is practically unable to complete it’s job, due to the inability to reduce texture memory.
2. This means that the “Opportunistic” option for streaming enabled does very little to reduce memory usage, as it can only remove mesh and part memory usage.

Overall, as I understand it, this interaction means that many of the quoted benefits of streaming enabled are only temporary when the user enters the game, and eventually memory usage will become out of control.

[Abcreator]

This is likely intentional, Roblox caches all assets retrieved either via ContentProvider or loading via a decal, etc. I don’t believe this has ever not been the-case (correct me if I’m wrong), so it may be worth changing this to be a feature-request?

@missmudmaam

When a texture is no longer in use on the client (referred to by any texture instances or in any other way), it should be removed from the client’s memory.

Just want to clarify that this should not apply to assets preloaded via ContentProvider:PreloadAsync function, assets loaded via that function should always be ready at a moment’s notice in memory.

[mrmudman]

I agree that this could possibly be intended behaviour for decals and other such textures, but I feel that the behaviour relating to composite textures (character clothing) is not intentional. I also agree that anything that is loaded through the content provider functions should always be ready, as this as been explicitly requested to be preloaded. If this does turn out to be intentional behaviour, then I will look at submitting a feature request to allow control of how textures are loaded in memory.

[teapotsandpie]

This is intentional, ContentProvider and it’s various counterparts cache textures, meshes, etc, for a certain period of time to allow content to be reloaded in the event you say… disconnect, or join back after going out of the game briefly.

It’s overall a massive time saver and bandwidth saver technique that’s been used by Roblox for years now.

[mrmudman]

I understand that contentprovider caching for a period of time is intentional, but the memory introduced in this manner can only be actually freed by leaving and rejoining the game. I feel that the actions of another player being able to permanantly introduce memory onto your client is unintended, as well as the consequences that this permanant caching has on streaming enabed.
Just want to reinforce that all of this data is stored on the client, not the game server, which means that it will not persist past the closure of the executable.

[TigerRabbit2]

Thank you for reporting this issue, and in particular the level of detail and depth with which you describe it is great and shows a lot of dedication!

That said, the actual workings of Roblox is a little bit more complicated and hence the conclusion that is drawn is not entirely accurate.

As mentioned in some other posts, this increase in memory is largely due to caching. But this should not result in memory leaks, so let me explain in more detail.

Roblox uses multiple caching systems for different assets (e.g. meshes and textures) as well as different levels of caching (e.g. drive cache, vs memory cache.) This is to provide an overall better experience by reducing load time and/or CPU time. So although your test case shows the memory doesn’t lower immediately after freeing the resource, this is only because the amount of memory the application uses isn’t near what the application considers a high enough memory usage to start emptying those caches out. (And especially on machines with lots of memory it can end up consuming quite a bit of memory)

As you load more assets to the point where memory consumption becomes higher, the system will decide at some point that it needs to free some memory in order to make place for the other assets and when that happens the system evicts assets from the cache. This can happen for different asset types and for the different levels of cache independently.

All that said, the system isn’t perfect and we do have the occasional out of memory crash. We are actively working to improve the systems involved with this to provide a better experience overall. Some complications are things like there are some experiences that reference assets through scripts, which makes it hard for us to determine if an asset it actually used, but we are working on that problem.

Finally there was a comment by @Abcreator “Just want to clarify that this should not apply to assets preloaded via ContentProvider:PreloadAsync function, assets loaded via that function should always be ready at a moment’s notice in memory.” This is more of a hint to the system where we perform best effort. The reason is that if we were to adhere to this rule, this cannot always work and lots of low end devices would go out of memory. For example some experiences preload say 1000MB of data, while the system only has 500MB of free memory available… so at that point we try to fidelity and/or evict resources that aren’t actively used.

In general one of the main ideas in Roblox is that creators don’t have to worry about memory budgets and device performance ranges as the system will do its best effort to scale the fidelity of an experience to the device’s capabilities. The reason for this is that it is actually quite a bit of work to do proper memory and performance budgetting for a couple platforms (e.g. consoles), but it is near impossible for a large range of devices and experiences. AAA games put a lot of man power into this and Roblox prevents you from having to deal with most of that burden.

I hope this answers some of the questions/comments on this thread. Thanks again for reporting this, I wish more posts had this level of detail!

[missmudmaam]

Thank you for your comprehensive reply!
This has helped me understand a lot more about exactly how the Roblox caching system works with regards to memory usage.
I appreciate your explanation and your time in helping me, TY