Announcing DataStore v2.0 - Automatic Versioning, Data Tagging, & Listing!

Is the userIds field guaranteed to not be used for automatic deletion in the future?
e.g. If you have a system where people can list users they want to allow into their house, and one of the allowed users requested erasure, Roblox would never wipe the whole record?

1 Like

I would prefer that either it will be automated or that we will have to option to make it automatically erased. I don’t see a reason for it to not be forced since to me that seems to be the whole purpose of the feature - to automate the erasure process.

Giving developers the option to not erase data that was requested by the user to be deleted doesn’t make sense to me.

1 Like

If you have a table storing profile information on Bob, and Bob adds Alice onto a permissions list associated with his profile, the record would/may be tagged with both Bob and Alice in the array. If Alice made a request for erasure, and it automatically wiped all records tagged for her - this would end up wiping Bob’s profile - which is something obviously I would want to avoid.

Really I would want some routine which is called when a erasure request is made so I can just modify the permission list to remove Alice from Bob’s permission list.

I suppose we don’t know if/or any implementation details yet, but maybe it will be something covered by the “Cloud Scripts” on the roadmap as sounds like the sort of thing a Cloud Script should handle?

3 Likes

Is there a reason for all of the instances?

DataStoreKey, DataStoreKeyInfo, DataStoreInfo, DataStoreOptions, DataStoreSetOptions, DataStoreObjectVersionInfo, and DataStoreKeyVersionInfo could all either be dictionaries or have their own data types, like the *Params types.

Using instances for these makes code very verbose, to enable these new settings you have to create a DataStoreOptions instance, call a method on that instance with a dictionary, and then use that as a parameter to GetDataStore.

All of the Instance properties seem to be unused on these instances, which is very confusing.

I would prefer dictionaries or new data types be used instead of these instance, to make code less verbose and to make them less confusing (do these Instances really need to store all of the information associated with an instance? Name, Parent, Children, etc.).

15 Likes

This is a very awesome and helpful update. I thank everyone who contributed to this update. The roblox team is very hard working and are great at what they do. Again, I can’t thank you enough roblox, always making your platform better and better.
-Cats767_99

2 Likes

I feel this would fit better as a separate reply. I’ve been using the post I’m replying to as a way of tracking the way I’m tinkering with the new API.

tl;dr of what I want to say is: if you want to use (or test) ListDataStoresAsync, do not use the iterPageItems code sample on the Developer Hub (found in Pages instance documentation). If you’re like me and plucked that function off to use by virtue of listing APIs returning Pages objects, you will encounter this same issue where not all your DataStores will get listed.

I got some follow up communication on all of my testing and ListDataStoresAsync does not work with the coroutine that iterPageItems creates (as of right now, not sure if there’ll be any addressing of this down the line). Write your own iterator or for loop generator. I was given a code sample and then later wrote my own which successfully iterated all my experience’s DataStores.

In contrast with the test code I ran in the post I’m replying to, my new code only needs to use a single ListDataStoresAsync call instead of 60 which didn’t even catch my full list of DataStores. That code caught only 135 DataStores while my new code catches all ~320 DataStores that have ever been in use across the 4+ years this experience has been developed for.

Below I’ve provided a better code sample which will allow you to see every single DataStore your experience has ever used.

--- ListDataStoresAsync test
-- @author colbert2677

local DataStoreService = game:GetService("DataStoreService")

local startClock = os.clock()

local pages = DataStoreService:ListDataStoresAsync()
local dataStoreNames = {}

while not pages.IsFinished do
	local page = pages:GetCurrentPage()
	for _, value in ipairs(page) do
		table.insert(dataStoreNames, value.DataStoreName)
	end

	if not pages.IsFinished then
		pages:AdvanceToNextPageAsync()
	end
end

print(#dataStoreNames, dataStoreNames)
print(string.format("Took %f to iterate DataStores", os.clock() - startClock))

(~4.316013 seconds to catalogue ~320 DataStores)

Check your console for a dropdown beside a number and open it out. You’ll get a list of strings of the names of DataStores. You can also iterate through the table if you’d like.

8 Likes

Does this have automatic transactions? It was really annoying to constantly have old data replaced as no transactions were in place, so I had to create my own system for that. Are they built in now?

5 Likes

ProfileService on a new DataStore 2.0 feels a little like a dream about to come true, doesn’t it :star_struck:

2 Likes

ProfileService v2 is already here :sunglasses::

16 Likes

OMG Thank you soo much I can’t wait to test out the new features :D.

1 Like

You’re incredible :heart: Absolutely pumped to get this in my games

1 Like

I think these new DataStore features are great, however for GDPR requests, it would be nice if there was some kind of automated system that could remove all data with a specific tag if a user issues a GDPR request- having developers manually erase data can be a bit messy.

6 Likes

I mean, there is a way to make a save slot system, where users can choose to restart their save slot or continue without restarting.

For example:
User joins experience >> asked to load in or restart save files >> asks user to confirm action >> and boom!

1 Like

Thats an excellent idea! But for this to work it would need to be a must for all games. Takes a bit of a learning curve, but its totally doable and adjustable just like FE.

Also I`m hyped for ProfileService update, thanks loleris!

1 Like

It looks like you can associate a UserId with a key when using SetAsync now in this version of DataStores for this exact reason. It even says so and recommends doing so on the devhub article for DataStore v2.

Edit: There’s also similar support for this in UpdateAsync as well.

1 Like

However, the keys aren’t deleted automatically when a GDPR request is issued.

4 Likes

Yesyesyesyesyesyesyesyes! I have been waiting for this update for months. I need to know when this update will come out. The feature I want the most, though, is saving instances. That was a feature of the legacy Data Persistence, so I really want that to be added.

2 Likes

This is nice and all, but it seems super wasteful for things like session locking, where only one value in a big table is being changed, which would create thousands of useless versions for active players. Is there anything I can do to avoid straining the systems?

1 Like

Versioning isn’t useless with session-locking :face_with_raised_eyebrow:
ProfileService added features to check older versions of data, and also overwrite the newest version with them, if there’s any little accident on your part, it’s super helpful.

This isn’t a problem, in fact I was able to get a single key to have 7k+ versions while cleaning older versions was disabled, and that didn’t make datastore requests any slower. My :UpdateAsync calls still took around 0.25-0.75~ seconds.

No, older versions already get cleaned up after 30 days, Roblox expects a bunch of versions to be created at all times, and that’s fine, they knew what they were getting themselves into, and they commited to it.
+ This doesn’t affect speed, efficiency, or anything else for you.

2 Likes

It may not do so yet, but at least the feature is there with the intention of automatically handling them in the future (at least, I’d imagine that’s the whole point of it existing in the first place).

1 Like