Upcoming Changes to Data Stores Versioning

Awesome work, looking forward to the new API!

2 Likes

This is the issue with Roblox blindly taking a % instead of it being pay for usage. All of these features have costs that at the end of the day developers are paying. It’s money Roblox is choosing to put towards infrastructure instead of DevEx. If they chose to optimize their infrastructure and let developers pay for usage instead of a fixed %, those savings could be passed on to developers but they have no incentive to do it when nobody has a problem with them taking a %. Nobody asked for this feature but every developer still has to pay for it because Roblox management said so

4 Likes

Same really goes for how many players are in your game at once. Developers with dozens of thousands of average active players pay the same % as a small developer with ten average active players even though the difference in cost for those servers is astronomical.

What if developers don’t have the budget for a “pay for usage” model?

Was there a reason to limit to once per hour VS just have a cap on total versions with a first in first out and the 30 day expiry?

I can see this negatively affecting any games that used it as an effective save history and restore system.

Other cloud service providers they offer free tiers so people can easily start developing on their platform then when a project becomes successful both the developer and the service provider benefit. This makes it free to start developing or if you only want small games with a few players, you only pay when your game becomes successful

https://azure.microsoft.com/en-us/pricing/free-services/

1 Like

Good question! This is something we did consider. After running a few focus groups, we decided it was more useful to guarantee having versions available over a wide range of time.

We didn’t want to force creators to significantly reduce their write frequency in order to not “use up” all available versions in one play session. Even if you only save once when a user leaves your experience, a user could have many play sessions in a single day and effectively erase all of their previous saves from the previous 29 days.

1 Like

I am honestly a little confused by this post considering I’m not the brightest with datastore coding, what are the ways this might break my datastore script stuff?

Are we eventually going to have a setting to automatically wipe previous versions and keep the most recent one? I personally don’t use previous versions so it might make sense some developers may want an option to only keep a key’s current data.

Thanks for the context. Did the focus group include any creative / building games?

These types of games where an hour of progress from a player building their home/boat/vehicle is a big gap if that player wanted to rewind / restore.

For this use of the versions api where you are trying to provide frequent restore points for the current session the change will force a rewrite to store under seperate keys and cause more and mostly uneeded datastore requests.

It would be good to hear if the dev team is open to feedback on the effect this change has for this use case or would consider a longer grace period?

Unless we missed a prior post about it being changed this is relatively short notice.

These changes will not break existing usage of Data Stores. GetAsync, IncrementAsync, RemoveAsync, SetAsync, UpdateAsync, ListKeysAsync, etc. will continue to work exactly the same way.

The main difference is the number of versions available through ListVersionsAsync, GetVersionAsync, and RemoveVersionAsync. There will now be fewer versions covering a wider timespan.

Please let us know if you still have any questions or concerns.

1 Like

We don’t have definite plans to do so with these changes, but we’re investigating different ways to bring you all more control over the versioning behavior in each individual experience. If an option to keep only the current version seems useful to you, please open a feature request!

1 Like

We appreciate your feedback. Yes, we did discuss with creators representing a wide range of experience genres, including building games.

We understand that these changes won’t perfectly meet everyone’s needs, and we do believe this is the best solution with minimal impact to the majority of experiences.

As I mentioned in another reply, we’re looking for ways we can give more, intuitive control over version behavior. If you have suggestions for how we could better meet your use case, please open up a feature request and we can discuss there.

1 Like

How can I achieve roll-back using data stores versioning? Let’s say I publish an update and it erases people’s data, how do I roll-back to a safe position in time?

Currently (for each key) you need to use ListVersionsAsync to identify the correct version to restore, GetVersionAsync to retrieve that version, and SetAsync to overwrite the current data using the previous version. In the case of accidental deletion, the “current data” would be a special version marked “deleted”. We have a code example in the documentation. GetVersionAtTimeAsync can replace ListVersionsAsync and GetVersionAsync when it’s released later this year.

At this time, there is no automatic process to restore all keys in a Data Store to a particular time.

5 Likes

@CoffeeBurg3r a common problem is this:

  1. Data fails to load (either DS error or application error).
  2. Player notices wrong data and rejoins. This can happen a few times depending on the error.
  3. Player reports error to developer
  4. Developer looks through datastore entries and server errors to debug.

If you limit a backup per hour the developer might lose crucial debugging data. Have you considered keeping backups for the first week and then merging the backups to one per day or something similar.

I understand that’s this is a complex issue and there are no solutions to solve everything. But in my experience this is a very common situation.

PS: Are there plans to create DS visualization tools? Also, are there plans to cap DS usage?

Thanks, @Redridge. We appreciate the feedback and understand the scenario. This is something we’ll keep in mind as we think about how to expand customization of this system.

Keep an eye on the Creator Roadmap and here on the DevForum for more info on future Data Stores improvements!

This would be insanely helpful. In a project that I was previously signed onto, we had a tri-attack of issues: exploiting (old codebase with zero safeguards that had to be updated), data corruption (unexpected writes that could not be processed) and versioning (nothing in place to restructure data formats from bad form to a more usable form). All of these issues required considerable effort to do global rollbacks for and supporting old systems for a period of time before removing them and simply consigning the data of all players who didn’t log in during the grace period to oblivion.

Fetching the nearest version to a timestamp and doing global rollbacks is a considerable chore right now. GetVersionAtTimeAsync would be a massive boon to this effort alone by simplifying the process to retreive the best available backup but still requires some manual maintenance and preservation of rollback systems in the game code. I would love to be able to mark an entire store as “needs rollback” and then the next get/update for any key on that marked store would get a previous version rather than the current. Would totally cut out all the intricacies of rolling back.

2 Likes

As the year is nearly ending, what is the update on GetVersionAtTimeAsync?

Our target is still to release GetVersionAtTimeAsync this year. I’ll update this thread when it’s launched. Thanks for your patience!