Saving User created content exceeds the 64kb limit

Help and Feedback Scripting Support
, ,
#1

So I am making a building game and I have added a system to save your creations.
For right now I have tried decreasing the size of the size of the data by rounding the position etc.
But I still only manage to save let’s say about 1800 parts at max per token.
I was thinking about using multiple datastores for a single build but I am not sure if it is a good idea because of the limit on how many times you can save/load datastores.

Is there any good way to save user generated content more effectively or to get more storage capacity?

Game link: Stair Building - Roblox

1 Like
#2

Do you mean a sort of thing that loads a map made by other players, kind of like flood escape 2 map testing?

#3

No, it is more like a building game. A bit like build a boat but this is a completely sandbox game.

#6

I am not really sure if that is what you are looking for but I think you should take a look at this thread:

1 Like
#7

Currently I am saving all the block data in an array. CFrame, Block name, material, color, size, extra properties.

#8

What I’d do is try to compress the data as much as possible. For example use prefabs instead of storing raw data. Don’t store strings but numbers representing strings that can be converted later on. Instead of storing “Wood” for the material store for example the number 1, etc.

For prefabs, if there’s multiple objects with the same properties such as colour, material, etc where the only difference is the size or position, you can store them as one entry and have a unique ID to represent it.

Example of before and after:
Before (247 characters):

player1Data = {
    objects = {
        { cframe = .., size = .., color = { 255, 255, 255}, material = "wood" },
        { cframe = .., size = .., color = { 255, 255, 255}, material = "wood" },
        { cframe = .., size = .., color = { 255, 255, 255}, material = "wood" },
        { cframe = .., size = .., color = { 255, 255, 255}, material = "wood" }
    }
}

After (192 characters):

player1Data = {
    prefabs = {
        { color = { 255, 255, 255 }, materialID = 1 }
    },
    objects = {
        { cframe = .., size = .., prefabID = 1 },
        { cframe = .., size = .., prefabID = 1 },
        { cframe = .., size = .., prefabID = 1 },
        { cframe = .., size = .., prefabID = 1 }
    }
}

And these are just simple improvements, you can do much more than this by for example compressing the R, G and B values of the colour into one integer (using some math tricks); Or simply shortening the variable names as Roblox stores all the data in raw JSON.

#9

I had a similar concern for saving data for player’s houses in my game. I ended up using a compression algorithm that would take the Encoded JSON string of the array and compressing it.

Here’s the post I used to get started:

It can make your strings around 3x shorter if they have a lot of repetitive patterns

#10

color and material can be changed individually… also the color values are already shortened.

#11

They can be changed individually, but if there’s duplicates then you shouldn’t store each of them. For duplicates you create an entry and link it. So you’re storing one number shared across several objects instead of the material/color data for each of them. It’s a waste of storage space if you store duplicates.

#12

the problem is that I have at least 30 or more color choices and every single roblox material as a choice.

#13

That’s fine, if you have a set amount of colour choices it’d be even easier because you can have an ID for each of them and store those instead. Then upon loading the data you pull the values the IDs represent from a table.

#14

good idea. So I would just save a single digit instead of like 12

#15

Yeah, and instead of storing a string of characters for the materials you store a number as well. You can do this for pretty much every roblox property that doesn’t require a custom input. (Enumerated properties)

#16

I would have to check though if it is the new save system or the old as some people have their builds saved already

#17

When they join check if they got data stored in the new DataStore, if not check the old DataStore and if there’s a match in there convert their data and store it in the new one.

1 Like
#18

I would still use the same datastore just change what is saved

#19

Then just skip the moving part and convert it, maybe add a single boolean flag (isNew = true) to new entries and if an entry doesn’t have that flag, you convert it and add it. Or maybe a version identifier like version = 1.

#20

yeah. Do true and false values get saved as a single bite?

#21

I’m pretty sure it all gets stored as raw JSON, so the word “true” or “false” is saved as text. I could be wrong, but that’s why I edited my message to give you the second example with a version identifier. It’s one character instead of 4-5 for the value.

I am not sure how MongoDB handles the data. It might store it as one bit, because MongoDB uses BSON (Binary JSON) but that’s out of my scope of knowledge on their services. (MongoDB or a variation of it is afaik what Roblox uses for DataStores.)