That might be what the “Batch Processing” feature they mention in the OP will entail once it releases.
1 Like
For current DataStores which are always JSON strings, there’s not much that can be done, if you were to take every byte of a buffer as if it was a character, you would get some non-UTF8 strings and control bytes, which are no good.
Though, if Roblox added a new DataStore standard that allows for developers to define their own file formats rather than everything being JSON, in this scenario Buffers would be immensely useful, and I think they might be aware of this potential usecase:
1 Like
Okay is there a way to properly measure the size of data for datastores in roblox? Because my current method is:
local function GetDataSize(Data)
return #HTTPService:JSONEncode(Data) -- In bytes
end
I dont know if this is 100% accurate, but as far as I know there is no other way to do this.
My game uses 1 datastore key per player as a general storage for multiple saves, and the limit is 4 MB, and this is how I currently prevent users from hitting said limit
2 Likes
Just thought I’d reply to this in case anyone was thinking the same thing I was, you cannot simply convert the buffer into a string in order to save it that way reducing the overhead, as instead of simply saving the individual bytes as an array, Roblox saves many of the characters as simply their unicode endpoints in string form.
To demo this, I have this code sample:
local testing = buffer.create(4)
buffer.writeu8(testing, 0, 63)
print(buffer.tostring(testing))
game:GetService("DataStoreService"):GetDataStore("Testing"):SetAsync("StringBuffer", buffer.tostring(testing))
Which when ran in Studio’s command bar produces this output from the print statement.
Implying that it was able to encode the buffer as a string, but when I view the DataStore Manager to view the actual value (absolutely love this new tool by the way, thank you so much for it), we instead see this.
While the question mark was encoded as a single byte, the other empty bytes are now 6x larger, taking six characters to store a single byte, meaning that for the vast majority of data, it will actually get about 6 times larger. As long as your data is using a limited subset of characters, you can get around the extra overhead (such as if you were using Base64), but at that point you just end up with exactly what Roblox does anyway (and this is where the 3:4 ratio comes from).
With that said, at least on paper, this actually sounds like a pretty good change, effectively being able to make 4x the amount of get requests, and 2x the amount of set requests as before. As some people have pointed out however, not all DataStore requests that servers make are related to player data so maybe some extra capacity could be given on a per-server level, but honestly, if you have two people in the server you already have the same capacity for get requests as you would with the previous limits, and anymore players is just free requests. I could potentially see problems with set requests, as you don’t end up breaking even until you hit 6 players per server, but still, I don’t see why this would be a problem for anyone. In the end, I think my only problem with this is the overhead that buffers have when saving to DataStores, as these new limits seem to be encouraging developers to use buffers to decrease their DataStore sizes, and yet this overhead doesn’t let us take full advantage of it.
2 Likes
Aha; that’s wonderful to hear! 
Thanks for letting me know, PhreakZip! 
2 Likes
You prove that you are just replying for the sake of replying, we are asking for a feature with use-cases, yet you need to negatively impact that discussion by sharing downsides which are not affecting the obvious use-case.
1 Like
Will ProfileStore still work after these updates?
1 Like
I don’t see why not. Here is my API quota usage with ProfileService:
The datastore limits shouldn’t be an issue either unless you are saving extremely large amounts of data per player.
2 Likes
Are you able to give some more info on how this would work?
1 Like
But Minecraft worlds are more than a single chunk, storing something like Minecraft world data would be better stored on a players computer because of Roblox stores it all for free the storage space would easily become too much.
Although local stored data should technically be possible if you have a player use an app to store the data and send it to the game via http
1 Like
my point is that it’s a worst case scenario for a singular chunk to take up more than even 1MB, with the right structure it would likely take up barely any space so each world could likely fit a lot of content in for the player before it becomes problematic.
1 Like
Also another huge concern i have with this games that provide users with different save files to play , might be at risk since Life Time Players Doesn’t take into account multiple save data.
Some noteable examples for game that do this are
Arcane Odyssey
Voxel Blade
Deepwoken
and many more rpg games that follow suit.
2 Likes
Unless those games are being incredibly inefficient with how they are storing data, it’s highly unlikely they would ever have a player commonly using more than 1MB of data across all save files combined.
5 Likes
Could we see the introduction to “force” viewing the number of keys and size of Datastores if a game does have more than 100 Datastores registered?
This seems like quite a important thing I’d love to be able to view but due to having more than 100 datastores registered due to old formats makes it fully impossible as far as I’m aware to view these metrics if I o desire for specific datastores that are registered 
2 Likes
Please let us delete entire Data Stores. I accidentally made so many that I can’t even delete and now I’m being robbed out of important metrics.
5 Likes
It is incredibly upsetting to see this thread spammed with misinformative garbage and armchair expertism. ITT Roblox developers severely underestimate how much a megabyte is. God forbid you guys knew that when DataStores were introduced to the platform, they started with only a 256KB limit.
Forget 1MB, I’m barely just under 50% of the ORIGINAL limit. This is a net positive but people are freaking out too much over what they think is a problem instead of what actually is one. For build games I’ve already recommended actually designing your systems, doing stress tests, checking your dashboards and your data structure. Everyone else has zero room to complain.
In my previous post I posted a blurred out view of one of my top players’ raw data in Visual Studio Code but there is still people missing the point that this is a net positive for everyone, so instead I’ll list out every single bit of data I’m storing per player TIMES TWO (due to test environment requirements) to provide a sense of scale.
The important part to keep in mind is that I store all this and don’t cross 100KB. It has a mix of modern, bad and questionable practices.
- A few unused keys from older iterations of the game (2017-2020)
- Ban data (boolean + string reason)
- PVP status (bans, wins, losses, kills, deaths, combat rank)
- Patch notes (hashed string so it only automatically shows once per update)
- Spawn location name
- Material grinding endgame stage unlocks (character classes)
- Shop purchases (includes data per shop category for limiting some purchases by day/week)
- Active subscriptions
- Unlocked lobby teleport locations
- Unlocked emojis (for noticeboard posting)
- Secondary currencies (to date we have 39)
- Primary currencies (to date we have 2)
- All pets unlocked
- All titles unlocked
- General gameplay stats (bosses killed, current EXP, highest combos)
- Companions (deployable combat NPCs with their own levels, unlocked skills, progression)
- Achievements (we inject all achievements into this table and mark them as complete or not and their current progress instead of making it only obtained achievements; right now we have over 200)
- Chat titles (we have around 80 that can be obtained publicly, around 90 in total)
- Account-bound boosts that apply to all characters
- An extensive record of the purchases a player makes in both Robux and game currency
- All system mails sent to the player which include rewards when claimed
- Storyline progression (main story and side quests)
- Journal (unlockable information on every enemy mob, over 100 of them, that have anywhere from 1-5 entries to track progress on)
What I didn’t mention in my other post is that we also have separate keys per character class, which means even more data accessed. We have 8 playable characters right now and there’s no barrier to obtain any of them. The per-character data is significantly smaller which allows us to do things this way.
Every player can reasonably be assumed to have all of the following data saved SIXTEEN TIMES OVER (8 characters on live, 8 characters in test build):
- Skill progression (unlocked skills, unlocked skill enhancements, plus 3 skill chains that player access through number keys)
- Character customisation due to being able to mix and match outfits (a table that has keys for all 15 parts of an R15 character, that are then further broken down into an identifier for which outfit that limb takes from plus a primary, secondary, tertiary and quaternary colouring)
- The above TIMES TEN because players can have a maximum of 10 saved outfits. Which means that for 8 characters on both live and test build, that is 160 SAVED OUTFITS.
- Current stats (leveling points put into attack, HP, defense, etc., plus current character experience)
- All weapons a player has obtained (includes the weapon’s stats, enchantments, native skills)
It’s okay to intuitively limit your players too. Most games have the same kind of player-facing limits to respect the game’s data storage capabilities and I’m not talking about Roblox games when I say this.
Just some extra thoughts I wanted to share in addition to my previous posts here.
tl;dr: The people who need to worry aren’t in this thread, they’ve been reached out to. The rest of you complaining are just spamming until you show a proper overview of your observability dashboard exceeding limits, or thinking about problems you don’t actually have and won’t run into.
6 Likes
local data storage would actually be so amazing and a wonderful way to solve the main problem sandbox games have with these changes.
2 Likes
sorry but you downplaying this very hard. either your misinformed or you lack experience working on games that do use the massive storage roblox provided in their Advantage.
people do use massive amounts of data , either for different save files , building Saves , inventory/Bank slots to save items , certain analytics , Certain history of actions , etc , and a lot of other mechanics that will add an addition layer to the games data specially when some of the systems were designed with intent of not having this.
and yes there will be a few cases of badly optimized stuff , but really if you using this much you know what your doing.
just because you use tiny bits of data doesn’t mean everyone is.
yes this won’t effect a lot of people , but it will effect some people and they like to be heard too.
but it will effect me.
don’t speak for everyone and don’t try to silence the people who do. just because you don’t use that much data
and i’m not Advocating for removal of this it makes sense for them to do it but i wished they did increase limit from 1MB * Life Time Player to either 2MB * Life Time Players and maybe if they feel generous maybe 3MB but thats overkill.
maybe instead of calling me Misinformation you tried and listened.
4 Likes
then why my game use 21.5kb for 105 keys ?
1 Like
can you add the ability to see the storage taken by each key separately ?
1 Like