Batchstore Module (DatastoreService Module)

Batchstore

Hello guys, I decided to make a small little module dedicated to large datasets; this module allows you to split data into “batches” that you can retrieve data from using easy simple functions like GetAsync, SetAsync and RemoveAsync; this uses regular Roblox DatastoreService just FYI.

An example of how this works would be listed in this diagram, you’d have complete control of a key in a chunk! :smile:

What this module is built for is massive data loading, in order to load data faster and having to go through less keys but recieving the same type of data back through the batches; this module has some pros and cons though; the pros is that you can store a bunch of data and avoid giant rate-limits when going through a big list of data keys, the cons are that loading times depend on the amount of Batches (you are allowed 150,000 characters by default and 200,000 is the maximum); for every patch 1 second is added to the delay (first batch ignores the limit)

Documentation

In order to save and get data, you can follow this example:

local BatchStore = require(path).new("NewData") -- NewData can be anything

local serverData = {
  ServerWeather = "Rainy";
  ServerSeason = "Winter";
  -- etc..
}

BatchStore:SetAsync(game.JobId, serverData)

print(BatchStore:GetAsync(game.JobId)) --> This would return the data we inserted earlier

BatchStore:RemoveAsync(game.JobId)

print(BatchStore:GetAsync(game.JobId)) --> This would return nil because we removed it

Self-explanatory module, only intermediates should really use this and I only made this for fun; if you want to use it you should probably tweak it due to the nature of me making it solely for fun.

You are allowed to make public changes by making a pull request! :smile:

Releases

If you want to get the module: Releases · aisarwinning/batchstore · GitHub
If you want to want to look at the github page: GitHub - aisarwinning/batchstore: Roblox Datastore Batch Splitter API

Please like the post it inspires me to do more stuff for open source!

Also check out my other posts: [Open Source] Simple Combat System (with blocking and stun)

3 Likes

I’ll check out this module when I get home, In the past tried to make a similar system where I could load chunks of data to avoid the 4MB limit per key.

Are the batches limited to 3 or scales?

Good module overall

The batches are un-limited, the character limit although is 150,000 (you can change this but don’t cross 200,000 characters)