I am currently working on a placement system game and I was wondering was is the best way to handle saving data that exceeds the maximum size for a single datastore entry (260000). What i plan on doing is first checking if the table is under the maximum size by check the length of the serialized string returned by the json encode function. However, the problem I am having is segmenting the table so it can be saved in different datastores. For example, if I have a entry that is 320,000 characters I would have to split it into two entries (260,000+60,000). I know it would be very hard for limit to be exceeded (because i ran some tests and the only way it will be exceeded is if the player places thousands of items on there area). Anyways I was wondering what would be the best way of handling saving data in such scenarios, and I really don’t want to set a maximum number of items in which persons are allowed to have on their plot.
- Pretty sure a lot of building games that save have a max number of parts
- Why not just do something like save the object type and CFrame?
bloxburg doesnt, and that’s all I am saving along with the colors, and to cut down on the size I have a color dictionary so I only save the key and not the actual color3 value
After encoding to JSON and testing string length if it exceeds the max length then split the list at every 259996 positions to have room to add a few characters at each. Then loop through each element and append a string continue pattern like this.
"....Assume that this is a really long string that is too big for a datastore to f<end>"
Then make loading function so that if it sees the continue statement and it increments the datastore key by 1 and continues loading the data to append to the original. This will let your code know that there is more to load. Just remember not to add a “” (or whatever continuation pattern you used) if the string fits
"it, so now this is a new datastore save that the load function will recognize exists."
Correct me if im wrong but don’t most games that store lots of data often use “BitBuffter” to accomplish this? Phantom forces, theme park tycoon, and im sure bloxburg judging by what is being saved. Stravant has a module for this. Alternatively you could look at @TheNexusAvenger tutorial on the module itself. Stravant's BitBuffer Module - Compact Storage Of Data [Tutorial] (It was created in 2015 so it could very well be outdated, however, it should still give you a rough idea on how it works. Just be cautious.)
Im confused on why this is taking up so much space? What are you saving? All you should be saving is the cframe in numbers, the color and the name of the item.
To combat this why not make a system where:
The player’s UID corresponds to a data store which holds references to a number, which will describe the number of datastore chunks owned by the player. This number then is pasted into an algorithm with the user’s UID to just create a unique key, could just be by saying UID…“DATASTORE”…DATAID. So that way each user can have multiple DataStore Keys storing chunks of their data.
When the data is all loaded in, it is combined into one table, and can then be split again when you need to save it.
Of course this is a lot easier said than done and would require a way to work out how much can fit into one “chunk” of data.
i know but they can run out of space if thousands of item are placed. so i am just ensuring thats does not happen
thank u very much this seems like a very promising method. 
Thousands of items can be placed? Thats alot…