Description: Servers have a chance to completely hard-crash (no crash dmps created unless debugger attached) and this happens in live games and within Roblox Studio. Been happening for almost a year now and we’ve been trying to hunt it down (thought it was a while true loop within our code that was not delaying) or a potential memory leak but it happened in brand new servers. Reproduced it in Roblox Studio by accident one day but it instantly closed the window without any crash dmp or log so I decided to try again with WinDbg’s debugger attached to grab the crash dmp so Roblox Studio doesn’t abruptly close.
At the moment, I’m unsure what exactly is causing the crash (Maybe our parallel Luau code or the way we handle welds and characters?) but I’m guessing its something very specific we do within Roblox’s engine. Any potential work-around or tips would be nice so we can mitigate the issue until it’s fixed.
Video of reproducing the crash with WinDbg in Roblox Studio: https://www.youtube.com/watch?v=HtvMagBg0EQ (Note: has “loud” audio, might wanna mute, crash start happens at 1:08)
System Information should be irrelevant as it happens in live games and studio play-testing, but I’m using an Intel i9-13900KF and RTX 4090.
Crash DMPs & logs located in attached private message. No real reproduction files besides the crash DMPs and logs provided as I’m unsure how to fully replicate the crash completely.
After the server crashes, does the client behavior mimic what’s described here? (Everyone appears frozen, can walk around). There seems to be a lot more frequent server crashes popping up in the past few weeks.
had similar issues as OP, basically everyone appears frozen and then their client closes instantly but no crash message appears afterward. do see an error output of ‘corrupted asset data’ but no idea whether is this correlated or not
It seems to happens after a good amount of attempts, are you sure there isn’t a memory leak, or something isn’t being handled properly and it just stacks up until it crashes?
I’ve had a similar issue and my ram usage spiked, might be related or not.
Thank you. Please fix this immediately. Our game is losing thousands of players simply by bot accounts joining and crashing the server immediately upon arrival. I logged all activity and it is out of my control.
Any update to this issue, has the fix been rolled out yet or is there an ETA if not? We did a mitigation to task.synchronize before accessing AssemblyLinearVelocity as suggested in this thread earlier to stop crashes, however, that comes with the caveat of needing to switch from parallel to serial in the middle of our calculations which causes some performance dips. Appreciate you guys looking into this issue, I also know a few other experiences which share this issue and they appreciate this as well.
EDIT: Removed the mitigation so it’s fully parallel again, I haven’t received any server crash reports or heard about servers crashing as of yet so it appears the fix worked, thanks!
We actually just rolled out a fix for the crashing servers. We are still trying to triage crashes on Mobile devices, but if your problematic parallel code was being executed on the server it should be working now. Would appreciate it if you could update us on how it goes.
Do you have any further tips on how to reduce server-sided memory? Our servers are only last 1.5-2 hours before crashing, and have tried everything in our power to reduce it, but to no avail.
It really depends on what your game is doing, but if there isn’t any continuous memory allocation happening within your servers there might be a memory leak bug. Can you post details on how this is happening?
I have messaged the bug report group as I do not have the regular trust level to post in it yet. Focia19 will be approving it soon since they asked me to provide a bit more detail before approving.
