Developer Console: Update for Luau Memory Use Tracking

[WheretIB]

Hi developers,

Our in-game Developer Console displays a lot of data that can be used to improve performance of your experiences. And if you’ve ever used the Memory section, you would have seen the amount of memory used by different Roblox system components, but Luau script memory was only being collected into a single row.

In a recent update, we have improved the information available about Luau scripts and memory that is in use by different Luau threads to help track down issues - such as memory leaks.

We have added two new category groups: PlaceScriptMemory and CoreScriptMemory. The values for each row are displayed in megabytes and you can click on a row to see a recent history graph for that value.

Keep in mind that scripts are running on Client and on Server, so use the tabs at the top to select which memory statistic you want to view.

By default, when Luau thread is created to run a script, allocations performed by that thread will be assigned a tag that is based on the script name. By using the new debug library functions, it is possible to control the associated memory tag value:

• debug.setmemorycategory(string) will assign a custom tag name to the thread
• debug.resetmemorycategory() will reset the tag to the auto-assigned value

You can assign a custom memory category to the thread once at the start or use the combination of two functions to test memory allocations of a specific part of code.

If your script spawns additional Luau threads, they will inherit the memory tag value.

Two final things to note:

To keep the overhead of memory tags extremely low, there are implementation-defined limits for the amount of script and user memory tags. Because of this, you might see a single row sharing more than one script/tag name.

Keep in mind that a Luau thread has an associated memory tag and not the specific script instance. If your script calls a function defined in ModuleScript, allocations made there will be tracked by the calling script thread.

[Vmena]

I’ve been waiting for roblox to release something like this. It will be very beneficial in being able to track script memory consumption and get to the source of memory leaks. Thank you!

@WheretIB

there are implementation-defined limits for the amount of script and user memory tags

How high is this limit? Will it be something that the average user is running into or is it reasonably high where it shouldn’t be an issue?

[Noble_Draconian]

If I have a script that calls a function inside of a modulescript via coroutine.wrap(f)(), and inside said function I call debug.setmemorycategory(string), will that thread show up seperately in the console from the main script? Or is this not possible?

If it’s not possible, then this feature essentially becomes un-usable to me. Most of my games are made mostly of modules, with 1-2 script instances initializing the modules. If this is the case, then it should really be changed to support custom tagging with threads.

EDIT : It seems this is possible.

[Hexcede]

Based on how I interpreted the post this is what I believe happens:

debug.setmemorycategory("Top")

coroutine.wrap(function() -- This thread has the Top category since it was spawned in a thread with that category
	debug.setmemorycategory("Bottom") -- This only effects this thread
	
	coroutine.wrap(f)() -- This inherits Bottom
end)()

[0_0aztec (nick)]

Any chance this update fixed this?

GetTotalMemoryUsageMb() not correct. Calculating UntrackedMemory Usage

How is the total client memory usage at the top of the F9 console calculated? I need to calculate/be able to get the Memory usage of the category at the bottom of the F9 memory analyzer, “UntrackedMemory” [image] Certain exploits add to this memory category and this would work as a detection method against these, as it normally sits at 0 without injecting. There is not an Enum.DeveloperMemoryTag.Untracked so I can’t directly reference this value. I, therefore, have to get the total memory u…

[RuizuKun_Dev (Kitsune)]

This should be shown under a different category such as “Custom Developer Memory Tracking” not bunched up with the defaults, it is currently hard to find our own Memory tracker and differentiate them from the defaults so it should not be under “PlaceScriptMemory”

---

For now I guess we can prefix our string with a symbol so it will be shown on top of the others, however I have no idea what characters are acceptable due to the lack of documentation so

[Raspy_Pi]

Agreed. Currently it would be near impossible to use this practically in Phantom Forces.
Since all our weapon statistics are stored in module scripts it already clutters up the stats which makes it very painful to go through as seen below.

[WheretIB]

@Vmena

How high is this limit? Will it be something that the average user is running into or is it reasonably high where it shouldn’t be an issue?

I think it will actually be likely to run into the limit for a game of average size.
Currently we have around a hundred reserved for auto-assigned entries and around a hundred for user entries. There share the same pool, but we can adjust the percentage in the future.

But it is a soft limit, multiple tags will be placed in a single entry on overflow.

[WheretIB]

@RuizuKun_Dev

This should be shown under a different category such as “Custom Developer Memory Tracking” not bunched up with the defaults, it is currently hard to find our own Memory tracker and differentiate them from the defaults so it should not be under “PlaceScriptMemory”

We will take that into consideration.
We hope that ‘Search’ filter at the top right corner might be of some use for the time being.

@RuizuKun_Dev

I have no idea what characters are acceptable due to the lack of documentation so

Sorry for that, documentation update is being prepared.
I could say right now that we don’t limit characters for user tags, except for embedded zeros (\0)

[Ruuuusty]

Are there any plans on revisiting the ways that production and debug environments can be managed / separated? Seems like debugging features like this are considerably held back by the fact that they have to be compatible with runtime production performance needs.

[DataBrain (Amber)]

@WheretIB

• debug.setmemorycategory(string) will assign a custom tag name to the thread
• debug.resetmemorycategory() will reset the tag to the auto-assigned value

Awesome! As someone who never uses LocalScripts for debug purposes (I use ModuleScripts placed in ReplicatedStorage that return nil for all of my client scripts), I’m glad you thought of ways to make this work for everyone.

Wait, how do I get this to work for threads spawned using BindableEvent:Fire(callback) (i.e. FastSpawn)? All my custom threads show a memory usage of 0 when I try the following:

local ev_fastSpawn = Instance.new('BindableEvent')
ev_fastSpawn.Event:Connect(function(cb) cb() end)
-- ...
debug.setmemorycategory('$CLIENT_FRAMEWORK$' .. moduleScript.Name)
ev_fastSpawn:Fire(function()
	require(moduleScript)
end)

local ev_fastSpawn = Instance.new('BindableEvent')
ev_fastSpawn.Event:Connect(function(cb) cb() end)
-- ...
ev_fastSpawn:Fire(function()
	debug.setmemorycategory('$CLIENT_FRAMEWORK$' .. moduleScript.Name)
	require(moduleScript)
end)

Instead all of the memory usage seems to bundle with the EntryPoint LocalScript for the most part, which also is categorized with another random/unrelated module?

[mutex_lock (mutex)]

Just ran a quick test and it appears to be working correctly.

[zeuxcg]

@Ruuuusty

Are there any plans on revisiting the ways that production and debug environments can be managed / separated? Seems like debugging features like this are considerably held back by the fact that they have to be compatible with runtime production performance needs.

No and yes.

It’s absolutely a feature that this tracking is always enabled and as such carries some cost, meaning that we are limited in how we can collect this information. This is important because memory categorization is something that is often only usable on production at scale - imagine working on a 200 player game and having a need to analyze the memory usage on the server that is already running close to peak capacity. Internally we are also going to start using this system to report memory breakdown for the Roblox app when it runs out of memory, and something like this could eventually be integrated into our analytics systems for developers.

A system that doesn’t run in production is nowhere near as useful as a system that does. As such we will always prioritize systems that can be always on.

That said, you shouldn’t expect this to be the only system that helps understand the memory breakdown in the future. We have experimented with alternative ways to gather and present heap consumption, which is much more detailed than this, doesn’t fit into developer console, requires a lot more time and/or memory to gather and process the diagnostics, but ultimately can help developers track down complex memory problems. At some point Studio will likely gain this functionality even if you won’t be able to easily use it on production games. The same applies to profiling - we have plans to look into better code profiling tools, and while they will not replace existing tools like dev console script timing and microprofile, they will complement these.

[RuizuKun_Dev (Kitsune)]

@WheretIB

There share the same pool, but we can adjust the percentage in the future.

But it is a soft limit, multiple tags will be placed in a single entry on overflow.

I don’t like this at all it makes debugging harder, we should be able to see each entry separately, it defeats the purpose of being able to view each Script’s memory usage

---

We should also be able to sort which script is using the most memory so we can tell which Script has a memory leak or is using too much memory

[Baileyeatspizza]

since this was added I’ve been unable to log the entire contents in memory via the log button
as the output will just throw this message

Maybe you could change the log button to log in sections or something so it doesn’t exceed the limit

[Doomtiscally]

There are alot of scripts about this, and some in the modle catalog but you could always find script packs with them.

[VerdommeManDevAcc]

Will there be functionality added so that you can retrieve the current memory usage for a category in a script?

[focasds]

Is there any way to get the memory usage of an specific tag? Like I want to be able to get the memory usage of the script or thread in my code. Idk if this is possible…

[PysephDEV (Pyseph)]

You can sort by memory by clicking on the ‘Value MB’ (once for least-most, twice for most-least).

[RuizuKun_Dev (Kitsune)]

Thanks for the info but honestly who would have known it’s not intuitive at all as it doesn’t look like a button.

Roblox please also address this issue