First reported Dec 3, my users are reporting server crashes in one of the places of my game when the player count is low.
The game is a building sandbox, and this place is the most popular server with a lot of builds in it, so it may be memory related.
They report that the crash happens most frequently when only one player is in the server, infrequently with two players, once with three players, and have not reported it with four or more. Also proportional to player count is the place save period: the time between calls of game:SavePlaceAsync(). The timing of the crash makes it appear that the crash is associated with a call of this API.
The SavePlaceAsync API also appears to have become more expensive lately. It may be a continuation of this issue; see: Studio/ingame freezing on save
Here is an example where three players are booted 14 seconds after the SavePlaceAsync call:
Hey there! It looks like your RCCs are hitting the memory limit, and opting to gracefully shut down and disconnect all players instead of crashing. I don’t have any visibility into how your game is designed/implemented, but I do see a pretty high instance count (over 1 million in some RCC instances) - is this a number you’d expect?
Awesome, I appreciate you checking on that for me.
It looks like the problem server has 931,654 instances. It has been built in a lot; it has 87,795 objects in it, mostly structure objects that players have built. This averages to about 10 instances per object which is about what I would expect, yes.
Is 1,000,000 instances generally where I can expect to hit the memory limit?
Is the problem compounded by the SavePlaceAsync call (serialization doubles memory usage or something like this)?
Is there any way to see if there are any particularly problematic instance types?
What about lua memory, can you see if this a significant contributor?
