Currently, as of 12/11/23, the “migrate to latest update” feature does not work properly. When triggering an update migration, engineers have stated the following happens:
Servers will be shut down within ~6 minutes
New players joining the game cannot join old servers
However, this does not seem to be the case. This past Friday (the 9th), we released an update to our game that fixed various bugs in our game. We then triggered an update migration. However, as of today (the 11th), we’re still getting old errors within the past hour in our analytics dashboard, that are physically not possible anymore. We know this because the stack traces for the errors reference code modules that were deleted in our update.
This is problematic, because it decreases the utility of the error dashboard since it’s hard to tell if errors are still truly occurring in the current build of the game.
Expected behavior
It is expected that “migrate to latest update” would guarantee that all old servers would be shut down within a reasonable timeframe (6 minutes to an hour).
We’ve taken a look at the game update and observed that the expected behavior is honored. All game instances have been on running on version 76 since 3am PST, December 9th, 2023.To confirm, I am looking at Super-Skyward-Towers (7200003285) - Is the place version 76 the expected version after the update?
Yes, version 76 is the latest version of the place. However, we’re still getting error analytics from an old version of the game (even in the past hour). We know with 100% certainty these errors are no longer possible, since the stack traces reference modules that have been physically deleted.
Is this an analytics bug, i.e. are error events being incorrectly reported several days late?
MostSectionClears is not a valid member of Folder "Leaderboards"
Script 'Players.<Player>.PlayerScripts.Modules.Rounds.Classic', Line 254
would be an example of one that’s occured in the past day. The “Classic” module physically does not exist in the game anymore, so the only possible way that error is occuring is if old servers are still running, or if analytics is misreporting data.
We seem to be experiencing this in our game Tropical Resort Tycoon 2 , there’s an error that does not seem to go away even though we’ve migrated multiple times and the module in question no longer exists at all.
The error rate also pretty much exactly correlates with the one error per frame that was being thrown before it was fixed. We get ~5.2m errors in a 24h span, which corresponds to exactly one error source staying alive (60x60x60x24)
cc @friedrich1717 the error is this one if you would like to take a look:
ReplicatedStorage.Packages._Index.flow_core@0.3.16.core.World:258: Entity does not exist
Hi:
From what I see is that the error is from the placeVersion = 45. And it is the same version as it was from Jan 12.
What is the last version do you think should not have this error anymore? From what we see internally this version still populates this error and do you think it should not be there anymore?
I just confirmed that all this errors are from version 45. There might be some old servers that are still running on old versions. You can try to use migrate all servers to force stop players playing on older versions and it should help stopping this error populated again.
@friedrich1717 Filipti stated that they did use the migration feature:
I’m not sure what data you folks are looking at, but is it possible your data is wrong? Multiple people have been experiencing the same bug with migration.
Our internal log shows the version. We are working on supporting showing the version on the UI in Creator Dashboard as well. It should reflect the version accurately. Once we display the version and you still see some wrong versions running, please let us know and we can take a deeper look
We did not do any extra migrations since the previous migrations we did did not seem to catch that erroring server, and a migration always affects player count negatively. But yes, it seems like the error is gone and that no servers with that version are active anymore
Hi:
We verified that the game servers should have migrated all the players. the errors you see from an old version should be due to a wrong timestamp ingestion on our error reporting. We have put a fix to prevent this happening again. If you see issues like this again please let us know. Thanks.
Ahhhh, so the errors were actually from an earlier time, but due to a timestamp issue, they were being shown as having occured later than they actually were?
Glad you folks were able to address that! I’ll mark your reply as the solution. If this issue comes up again I’ll file a new report.