Announcing DataStore v2.0 - Automatic Versioning, Data Tagging, & Listing!

You can typically call both immediately. UpdateAsync will internally retry itself until the transform function returns either with data to be written to the key or nil to cancel the write. Get and Set operations normally shouldn’t take too long unless other logic is processing data and it yields; and any time spent actually writing to the data service would be on Roblox’s side.

Can follow up in PMs if you’d like. Just had a brief scare reading that post and thinking that GetAsync might’ve been returning the wrong things again, lol. The most likely cause for a GetAsync returning nil is either running into the cache, improper processing or something on Roblox’s side. If you’ve been good for now then you shouldn’t have anything to worry about; and if not, worth a bug report writeup.

DataStore V2 is new and they’ve apparently done some migrations and versioning on our behalf but I wouldn’t put it pass a migration to potentially miss on data, not sure what their process there was. Always worth a post if you encounter any new issues with this new API. :slight_smile:

I haven’t gotten around to playing with it yet, but I will! Soon:tm:.

2 Likes

If the API will support the feature natively, then I don’t see any point in using Datastore2 over the new API.

3 Likes


Does IncrementAsync have this UserId support too?

4 Likes

I know @dragonknightflies has responded to a bunch of my feature requests regarding datastores and to finally see this out is amazing. I actually have no real words. Automatic Versioning and Datastore and Key Listing are incredibly powerful and help to make datastores feasible while maintaining a level of safety than before. Data Tagging is one of the most useful things I can possibly think of and I am incredibly surprised that Roblox decided to add it in. Not many people know about it but, it helps a lot when indexing large sets of data and being able to break that initial set into tags is incredible.

The only gripe that I can think of is that datastores prove to not be well-suited during the developmental phase. I think that temporary datastores that can be instanced within studio for developmental testing would prove to be equally as powerful to Developers and lessen the amount of data-prone errors that Developers face since it provides a testing layer per-se that does not affect all users or rather, on production.

As a developer, we do have alternative libraries that offer something similar by emulating the functionality that datastores have but, so far it has not been added as native which is a bit restricting when you don’t want to push database changes to production w/o it possibly going wrong.

6 Likes

Agreed, I’ve created workarounds for these data types but it would be convenient to get straight to the point.

3 Likes

Documentation for various datastore v2 APIs are missing from the devhub at the moment : Some datastore v2 APIs are missing from documentation

3 Likes

The docs have a useless table in it:
Screenshot_20210812_233317
It just says there is no limit, which means this limit table is useless.

4 Likes

(post withdrawn by author, will be automatically deleted in 1 hour unless flagged)

2 Likes

This is great.

If our games have millions of entries, will we have to write our own migration logic that distributes of processes migrations over multiple servers?

Will there be any easy way to migrate data without taking down the game or anything like that?

Super excited to see what comes from this.

11 Likes

Really, really excited to see what people with do with this, new implements, I mean, there’s a lot!
This will actually really change the DataStore scene a lot, and also make certain things obselete, but yeah I mean, this is great.

I’m excited to see what happens after wards with querying features for example, that would be really really useful.

Imo, yes, this is way better than using the berezaa method, it’s better for GDPR and also way quicker, there’s no issue with limits like the berezaa method, 30 days imo is probably more than enough for most people, data issues either occur because of you or some really bad problem on Roblox’s side, and those are quite rare

Also bye bye DataStore2.

3 Likes

Is this some sort of dream? No words can express my excitement to see this image:

I needed this badly due to my tendency to break things after experimentation.

@loleris I would like to see ProfileService updated to use Datastore 2.0

4 Likes

I’m happy but sad at the same time xD I will have to update my DataBase v6 and include DataStore v2.

What’s the use of an open source module when you don’t have the latest updates, right? xD
:grinning_face_with_smiling_eyes:

3 Likes

It seems there’s a typo on the line right after it too where SetMetaData() should be SetMetadata().

2 Likes

This is easily my favourite Roblox update for a long time and fixes a number of my major pain points for the entire platform. I hope this functionality will be integrated into existing DataStore wrappers people have developed!

3 Likes

Another question: What is the point of setting Metadata? Can anyone explain that to me? Why shouldn’t I just store the metadata as normal data, where I have more memory to work with? Is there an advantage of storing it in such a way?

3 Likes

Supporting migration code can be a pain point for long-term projects. I have hundreds of lines of legacy deserialization code that I still need to upkeep; Most of it is over 5 years old.

I felt clever implementing an extremely compact save format for inventory data that encodes using the inventory’s length, but now I need to pass the length of the legacy global inventory when deserializing characters so that I can properly skip the old equipment data.

I’m okay with keeping old code like this around for a while, years even, but it’s a huge relief to know that I don’t need to support it forever.

13 Likes

Because in the future we’ll be getting features to query by properties inside metadata. You’re fine doing it inside your normal data partition, meta data should be for small things.

5 Likes

I don’t think that’s the best option, but it has to consider what kind of script it is. As roblox continues to update things, older scripts end up not working or some other reason. I had a game from 2015 and everything doesn’t work anymore, but before it did. Just update the script occasionally just to keep it working.

Off note: I am a huge fan of your game, keep up the great development! Nice meeting you, Tomarty!

4 Likes

What benefits does this have vs DataStore 2 made by @Kampfkarren . I made the switch to DataStore 2 last Saturday in my game, and do not plan to switch back unless there is a good reason for me to use this. I can also access versions just fine with DataStore2, and it says there forever and not just temporarily.

2 Likes

Is there an ETA on binary string support for DataStores? string.pack was added, but we still can’t use packed strings for saves without expensive base64 / ascii85 conversions. JSON is great for beginners, but it’s unnecessarily slow, and is honestly horrible to have set as a default; DataStores only ever needed to support strings, because developers can easily use JSONEncode / JSONDecode. I have the same criticism for MessagingService.

I’m hoping to switch to using binary strings for save data someday (instead of bit buffers), so that I can efficiently skip through data and find specific information without needing to deserialize the entire thing. If I need to redirect them to another server, I could just skip to their active character’s data, read its current zone, and send them on their way. Players can have potentially hundreds of individual characters with stats, items, quest progress, perks, etc., so deserializing between intermediate formats can take milliseconds of execution time from an already busy server. With a packed string format it’s very fast.

I’m also planning a house system with furniture customization, and string.pack is the obvious solution for storing these object. “{X:-61.20931480293841,Y:-13.1213440189129,Z:21.10298308140293}” has horrible efficiency compared to string.pack("fff", x, y, z) which uses only 12 bytes. Compact formats like this will reduce network overheads, reduce server overheads, and reduce data storage costs. It should be supported properly.

26 Likes