Announcing DataStore v2.0 - Automatic Versioning, Data Tagging, & Listing!

[Reapimus]

It may not do so yet, but at least the feature is there with the intention of automatically handling them in the future (at least, I’d imagine that’s the whole point of it existing in the first place).

[gillern]

Are there any plans to increase datastore limits, mainly for get requests? In my game players can build “machines”, each machine has a GUID key used to save it to datastores, then I have an array of metadata for each machine inside the player’s data

The list below is populated with the metadata from the player’s data file. The metadata only has the name and creator ID of the machine. When a player clicks on one of the buttons, the game makes a get request to download the machine’s content. Current datastore limitations means players can only safely click on 10 buttons per minute at most, and when browsing the library looking for new machines that limit gets hit very fast

[DanTDM_Rocks999 (Dan)]

Depending on your game, I would say that 6 seconds is a reasonable wait time between different machines. I don’t think someone would just move on to another machine that quickly to require more than 10 requests a minute. But then again, I have no idea how these machines properly work and if it would be better for someone to move on after 1-5 seconds.

Edit: I forgot about there being other players, so my idea probably doesn’t work.

[Crcoli737]

Yay! I’ve been waiting for a good change to the DSS for a long time now!

[Abcreator]

Great feature yet the fact that OrderedDataStores can’t use this kind of make this useless to me since my projects used OrderedDataStores.

OrderedDataStore does not support v2.0 features such as versioning, so calling this function on an OrderedDataStore key will overwrite the current value and make previous versions inaccessible.

[Grargror]

@dragonknightflies

We are still investigating the details to make GDPR compliance seamless for you. We’ll reveal more once the detailed feature plan is determined. In the meantime, feel free to let us know if you have any suggestions!

It’s great to hear that the Developer Services Team is looking into making it easier for developers to handle GDPR requests!

Here are a couple of ideas regarding developer-facing APIs for automating those requests:

Idea 1: Allow Developers to Enable Deletion of Data Tagged With a Single UserId

• Given
  • The DataStore v2.0 API now allows developers to tag data with a table of UserIds.
  • There is now a DataStoreOptions instance.
• Assumptions:
  • Data which is tagged with a table containing a single UserId is most likely associated only with that UserId.
• Proposed API:
  • Add an additional option to DataStoreOptions which allows developers to enable automatic GDPR deletion of data tagged with a table containing a single UserId.
• Additional Thoughts:
  • Pros:
    • By allowing developers to opt-in to deletion of data associated with one UserId, Roblox allows developers to maintain ownership of the data while also providing a method to automate a portion of GDPR requests.
    • From a developer perspective, this option is easy to implement.
  • Cons:
    • This option still requires developers to manually handle deletion of data which is associated with multiple users.

Idea 2: Provide a Way to Register a Callback for GDPR Requests

• Given
  • Developers currently manually handle GDPR requests.
• Assumptions:
  • In order to handle GDPR requests, developers theoretically know how an experience’s data is organized.
  • Developers likely already have code for handling GDPR requests.
• Proposed API:
  • Allow developers to register a callback for an experience, which Roblox would automatically call for GDPR requests. A list of UserIds would be provided to the callback, which would handle the logic to delete the data associated with those UserIds from the experience’s DataStores. This is similar to creating an AWS Lambda function. If a callback is not registered for an experience, Roblox can notify the developer to handle the request manually.
• Additional Thoughts:
  • Pros:
    • Allows developers to fully manage GDPR deletion logic.
  • Cons:
    • Requires more up-front setup by developers in comparison to Idea 1.

[GolgiToad]

Wow, I am overwhelmed! I just programmed my first DataStore (v1?) a few hours ago, discovered DataStore2 module an hour ago when I ran into save limits almost immediately, and now DataStore v2.

I’ll be honest, I have no idea what’s what, but this is only 8 days old, so I guess I’m starting here!

[nonserials (brenden)]

finally, better datastores. this is gonna be nice

[sleitnick]

It doesn’t seem like pageSize works for ListKeysAsync. No matter what number I put for pageSize, I always get just one item per page. Anyone else experience this?

[Extuls]

Just tested and this is happening for me as well.

[crowsegg]

How many keys are there in the datastore ? This can happen if there are very few keys in the datastore. Can you try with a datastore with 100+ keys ?

Also the API will always return less than pageSize number of entries, it does not always return the pageSize entries.

[sleitnick]

Yes, it worked better with more data. What’s the reason for this behavior with small sets of data? If I have just 5 keys, it’s a bit odd having to do 5 different fetches for those keys, rather than just one fetch.

[crowsegg]

The system is optimized to scale for large datastores. We will look into optimizing for smaller sets of data.

[sleitnick]

It would be cool if there was a way to fetch what scopes existed for each named DataStore too. As far as I can see, that’s the last remaining “unknown” in terms of finding what exists.

My particular use-case is extending my DataStore visualizer plugin. I can list out the DataStores that exist in a game, but cannot show the user what scopes are included in each store.

[crowsegg]

We have made Scopes prefix of the object key. AllScopes option was added specifically for the purpose of iterating through all keys in a datastore in all scopes.
In this documentation page

Create on Roblox

AllScopes section contains sample code and sample output.

[rogeriodec_games]

Can anyone answer this?

Will versioning consume the 4MB DataStore quota?

Once DataStore Versioning was implemented recently, I wonder if the additional data saved in each version will consume the 4MB DataStore quota? If so, this could be dangerous, as a change to a large table using SetAsync will duplicate this table in each version of the DataStore and take up more and more space within the available 30 days.

[dragonknightflies]

No, versions don’t consume the quota. You still have 4MB size limit for the latest version of an object.

[rogeriodec_games]

Thanks. Could you also confirm my question below, that is, if the space limit in the DataStore is 4 Mb per key?

DataStore Space Limit Questions

Just to confirm the current DataStore space limitation. I have two dictionaries that will store game data from these DS: -- General DS for all players - single scope local MyGameStore = DataStoreService:GetDataStore("MyGame", "MyGame") -- Individual DS for each player local PlayerKey = "Player_".. Player.UserId PlayerStore = DataStoreService:GetDataStore("Player", PlayerKey) In the case above: Will MyGameStore have an individual 4MB limit? Will PlayerStore have a 4MB limit for EACH PLAYE…

[dragonknightflies]

Hi, the older versions does not account for the 4MB limit of a key. So you won’t get errors as long as your latest version is less than 4MB