I haven’t had any reports since, so that seems to have fixed it. I’ll follow up if I start seeing it again, but thanks! Please pass my gratitude on to the networking team, they’re doing great things.
EDIT: I did get one report, but it was from someone in a VIP server that may not have updated.
Starting to see this again, I’m guessing the flag might’ve been turned on again?
Same, getting several reports of random disconnects with error 279 or 277.
We did not re-enable the flag, however when we disabled the flag we did so in a way that may not have been propagated to mobile clients, I will make another tweak that will definitely be propagated even to the long tail. Do you know if the new reports were from mobile or Xbox users?
I’m not totally sure, but this seems fairly likely. It was much less pronounced, so I figured that there were tweaks made and the flag was re-enabled. If it was only mobile, that’d probably produce the same effect. I don’t have Xbox enabled for my game yet, either. Thanks!
I’ve been seeing a lot of disconnections from my game again. I’m still using StreamingEnabled with the default settings.
Here’s the place link again: [UPDATES!] Welcome to Farmtown! - Roblox
It sounds like it might be the same issue.
We have not re-enabled the original offending feature, and we are not seeing other indicators of the original failure in our metrics. What error messages are the users seeing when they are disconnected?
Looking at Farmtown in particular, I do see some out-of-memory crashes attributed to your game which might manifest as connection failures. Can you try going in to a few instances of your game and inspecting the server memory in the developer console, to see if anything stands out?
Oh yeah. I’m looking at it now and I’m seeing “datastore/cache” taking up 3470MB. That doesn’t seem quite right.
As an update, we are investigating from our end. We’ve looked at the Data Store client code and it shouldn’t allocate more than 8MB cache, so 3470MB indicates something is going wrong. I’m reaching out to some other teams to figure out how we can do some more detailed debugging to understand what is happening.
In the meantime, is this issue continuing to happen for your game?
Hi mr_smellyman,
Upon further investigation, we found that if many DataStores are initialized and used by the DataStoreService, it’s possible to get past the 8MB limit mentioned before. Do you know if you create multiple DataStore objects using DataStoreService:GetDataStore in your game and use them frequently?
Yes, I do indeed. There’s one for each player. Is there a way to ensure the memory will get released once a player leaves?
It appears to monotonically increase every time I write a new data version, which happens every 30 seconds, even if the datastore writes are successful. That doesn’t seem quite right, hope it helps.
Is there any reason for you to create one for each player? If possible, I recommend you to switch to using a few / single DataStore with different keys for different players.
Not particularly, but there are plenty of code samples on the Developer Hub that use a datastore per player as well. That suggests to me that this is intended to be a valid use case.
Hey, just wondering if there was a fix in the pipeline. It’s not especially game-breaking, but it does mean my servers have a limited lifespan. The memory usage increases by about 1MB per player every save cycle, which happens every 60s in my game. I don’t think I’m doing anything especially nutty every time I save. It writes their current data to a new key and updates another key to point to that one, which seems to be a fairly common use case.
A ticket has been filed for this issue. Since this issue results in servers crashing, we consider this a high priority to fix. However, since most people are still on break, we do not want to push any significant fixes until sometime in January at the earliest.
Oh, right, I forgot you guys had code freeze. Enjoy your time off!
This has been occurring in my game which has StreamingEnabled as it’s a large map, could this be looked into? It’s impacting my game heavily.
Game: [BOUNTIES!] City of London - Roblox
Players get a Error 277 on disconnect
Can you also check few instances of your game and find out the “datastore/cache” usage in the server memory?
The usage of datastore/cache is only around 2.4, is that high?