I believe we are having an EXTREMELY similar situation. Our game upon starting will duplicate the entire map (a massive 25x25 baseplates) into ServerStorage, to allow future map regeneration. Our servers will max out on memory and crash after roughly 1-2 hours of uptime without StreamingEnabled, and roughly 15 minutes to an hour with StreamingEnabled. Curious to hear if you have StreamingEnabled or not.
2 Likes
Hi ! Sorry I donât know how I completely missed your reply.
I have it disabled despite multiple attempts to enable it for my game. I feel like we lack too much control as of yet with Streaming for it to be viable. It has always been causing a bunch of issues when trying to have it enabled and as a result isnât really worth the effort, I canât remember specific examples as to why itâs not viable for our use case but I know that after spending a lot of time toward having it working, I came to that conclusion.
Glad to hear that having streaming doesnât help either, at least Iâm not missing on something.
This issue is infuriating me, it forces me and my team to pivot to new kind of games because we are completely stuck right now.
The server should NOT cache the maps that are unloaded, and should NOT load the physics octree when all the assets are in server storage at all ! Please Roblox, give us better memory management tools so that I can at least empty the cache after each map unloads.
3 Likes
Yeah, the added memory (1.6 GB for us) that streaming enabled uses actually makes the problem worse. Weâve resorted to just disabling the map regeneration as server uptime wasnât even long enough for the map to ever regen. UntrackedMemory spikes as high as 1.5 GB after a full map regen (does not go back down, leads me to believe its caching!), and just cloning it into ServerStorage eats up 600 MB by itself. Huge problem for our game as it completely blocks all future map-related updates. Super unfortunate as besides the memory problems Streaming Enabled helps performance GREATLY compared to our own rendering system. Hopes to better memory management!
4 Likes
Yeah that sounds like the same issue. Roblox is caching terrain and parts for maps when itâs obvious that caching should not happen for such large assemblies. We literally do not care if it takes more time to load because itâs not cached, thatâs the whole point of having a loading screen. Caching size isnât an issue on standalone games, but since Roblox is limiting us with their memory size and non-existing memory management tools, itâs shooting itself in the foot, and us at the same time.
Moreover, we canât even load terrain on client, so there is absolutely no workaround other than copying the BaseParts of the maps themselves client-side, but that introduces a bunch of complexity because now the server doesnât have access to all of them which means delegating a bunch of other stuff to client as well which introduces security issues and adds more load on lower ended devicesâŚ
Seriously we need something quick, I had to abandon the idea of any kind of maps updates for my game as well, and Iâm moving on to another project solely because of this very restrictive and impactful deprecated limitation.
2 Likes
As far as I know, the caching is lazy allocation so it gets freed up only as you reach the limit. They donât particularly bother about having âfreeâ memory on Roblox until itâs needed, same reason why if you switch devices your client will store as many textures into memory as possible.
I find this approach combined with no developer freedom sucks at game scale, and creators have no way to handle device and server crashes from loading in too much data at once.
We run a game with high-res retro building style so we clock over the 100k part count and have had concerns with these limitations before. In our case, StreamingEnabled actually just made our overall server usage worse, without providing us much freedom in controlling what gets loaded/deloaded. We only turned it on because if we donât then our game isnât playable on mobile because the devices crash.
There are other kind of memory leaks you need to look out for also, just as an example, if you donât call player.Character:Destroy() and player.Character = nil before respawning a character it will cause a memory leak. And itâs impossible to have a weak reference to an instance, even if the container is weak, so they need to be freed up properly.
Annoyingly these issues happen to be arbitrarily nuanced because youâre never going to fully know the extent of how your game is handled by Roblox, because the profiler is vague and they wonât give us any documentation.
Havenât played your game but going to have a guess at some possible fixes.
Consider never destroying maps, and setting aside a tag for parts of the map which need to regenerate. And if they get damaged then mark that, and at the end of a game just regenerate the damaged sections. You would not need to clone models (unless they are movers) till they get damaged.
Otherwise, have you considered encoding your map into a custom rbxm/json format? Then further compressing it by group matching duplicate meshparts, models, and unions. Would remove the data from ServerStorage entirely, and allow you to store it even smaller than if it were a model. And it would allow you to load it dynamically.
3 Likes
I second this. Our game wants to expand in many different ways but canât because we are limited to the 6.25 GB that ROBLOX allocates to our game.
2 Likes
Please do consider giving us better memory tools. Debugging memory leaks is incredibly difficult. We basically have to resort to trial and error. Disabling some code, and testing it again.
There should be better memory tooling.
The current tools make it incredibly difficult to debug memory. (Or to find the courage/patience to start doing so 
)
1 Like
I know right. My game is constantly crashing because the server reaches the 6.25 GB limit. It is frustrating because we want our game to be realistic compared to other games on the platform, but since our game keeps crashing, we have to hold off on adding new and realistic assets that make our game more immersive for the platform. Our servers last 1.5-2 hours before crashing, and it significantly hurts retention and makes players complain.
If they want more games on their platform to look, function, and feel like AAA games, they should really consider increasing it because most games canât go that far before reaching that limit.
I think ROBLOX should give us the 12 GBs that they allocate for 600+ servers, and give those servers more (what games actually have 600+ server sizes though?)
4 Likes
Great post which I fully agree with. Memory is such a challenging topic to deal with as a developer, since each memory issue is unique to each game. Itâs not like scripting where you can watch a YouTube tutorial or read some Roblox documentation for help. Even with the amount of resources on the Developer Forum, it can be frustrating to deal with memory issues, especially for people like ourselves who aim to create huge maps utilising the latest Roblox technology.
I would definitely be grateful for better memory management tools or adjustments to Roblox servers.
3 Likes
For those who are interested, I think I have found a solution to the map storage issue. Since the engine seems to add in whatâs stored in ServerStorage to the PhysicsPart memory category, I found a way around that.
Create a folder under game. You canât do this normally so youâll have to use the command line do it. Basically something like this:
game.Workspace.Folder.Parent = game
You can place anything you want in there and it doesnât replicate to the client. Furthermore, since itâs outside of the engineâs purview (itâs not defined in the engine), it just sits there. Think of it being something like a persistent scratchpad. I have tested it and it seem to be working. When maps are placed in there, they do not add anything to the serverâs memory from what I can tell.
2 Likes
You mean this for in studio dev storage? I personally would not rely on this to never get obliterated by accident. This is likely not a use case they test regularly. Other alternatives are at less risk of data loss.
1 Like
I have tested this myself in a live game. The data does not go away and is part of the persistent data thatâs on the server. Most importantly, the data is accessible from script. To address your concerns, it is parented under game. However, the engine will not do anything with it because it exists outside of the engineâs purview. I doubt that Roblox will nuke it since we are using it for data storage. With that being said, itâs always a good idea to keep a backup copy of anything thatâs stored in the cloud.
1 Like
Do you need to parent an existing folder to it at runtime? Having one sitting parented to game doesnât seem to stick during runtime.
1 Like
I parented the folder to game and I was able to get data from that folder and place it in the workspace. This was done on a live server.
1 Like
Other games rely on this behavior and the folder will never be deleted.
(though it may be a bad practice)
Hm. Creating the folder, parenting to game, and then adding the assets during runtime seems to still increase physicsmemory usage. Iâd try putting it in there in studio and publishing, but the folder nor the contents persist once ingame, I have to make it again.
You donât add the assets during runtime. You add them while the game is in Studio which is what I did. The data persists since the folder is outside the purview of the engine and will never be deleted as @SubtotalAnt8185 mentioned. I published the game to a live server in this configuration and I was able to retrieve data from the folder during runtime.
To make it very clear, a Roblox admin has previously publically stated that parenting anything other than a service directly to the DataModel is unintended behaviour, and results can not be guaranteed.
1 Like
Do you have a link to that statement?
From what I can see, the folder no longer exists during runtime, only in studio playtests does it persist to runtime.