Help with efficiently storing 2k+ stats

Help and Feedback Scripting Support
#1

So I’m working or will be working on a rpg game that will consist of 2k+ stats. Basically players in my game in order to earn skills will have to discover the method of training to obtain skills. Like for example there might be a waterfall and you can earn like 13 different skills from set waterfall. Cutting it in half or stopping it directly in the middle or even absorbing the water would be theoretical ways of getting a skill. Now the issue I have is being able to confidently save all of this data. Because each skill will have to save the training method, the amount of times you train on it, and the individual level of the skill.

I thought about saving it to an externa website because I don’t know how well roblox could handle saving all that data. I also thought of doing like a darksouls type thing where players have to hit a save point before they leave and that will save all the data, because then I could take the time the request would need to process everything. I also thought of keeping separate data stores for individual types of skills. Like a data store for just water skills, but that seems highly inefficient. So what do you think I could do to keep requests down to a minimum while ensuring everything is saved correctly.

Note please do not recommend data store2, it won’t work for what I need.

1 Like
#2

There are a few things I can recommend.

First thing is instead of saving the name of everything, like “stat_1 = waterfall”, use IDs and numerical indexes when possible. When saving data, everything gets saved in JSON, so when using numerically indexed arrays, the indexes are omitted from the final JSON array, so this would shorten the amount of data that has to be saved.

So for example in Lua, your table might look something like { [1] = “waterfall” }, but in JSON that table gets converted to [“waterfall”], so the square brackets surrounding the number, the number itself, and the equal signs are omitted from the final JSON string which will cumulatively reduce the number of strings in the final JSON string.

For the ID part, in a module somewhere, assign each skill an ID number to be saved instead of saving the skill name itself.

In a module you’d do something like

{
skillName = “Waterfall”;
id = 1
};
{
skillName = “mining”;
id = 2
}

etc for each skill, then when you go to save, you’d do something like

{ 3, 1, 2 }

the number in the first place would be the ID of the first skill learned, second number = second skill learned and so on.

Final thing would be compression, use a lossless compression method to attempt to further shorten the amount of data that needs to be saved. There are several implementations in Lua on GitHub and on this forum.

PS sorry for awful formatting I am on mobile atm

#3

You really don’t need to be worried about request overload. The amount of data you’re storing won’t really be something to worry about. Remember that in Roblox Datastore each key can save up to 4 MB, which is a great amount.

All you need to do is control how often the data is saved. Roblox Datastore has a limit on how many times you can get and set data per minute. It counts as one time toward the GET budget when you retrieve data from one key, and one time toward the SET budget when you set data for one key. You will see that the size of your data has no impact on your ability to save.

I would recommend that you autosave the data for all players once per minute. And one time the moment the player leaves the server. This could easily be achieved by something like this:

task.spawn(function()
while task.wait(60) do -- Will run every minute
for _,player in pairs(game.Players:GetPlayers()) do
savePlayerData(player) -- Implement this yourself
end
end
end)

game.Players.PlayerRemoving:Connect(function(player) -- Activates the function when the player leaves the server
savePlayerData(player) -- Implement this yourself
end)