You can always compress the data to save up on space.
3 Likes
For inventory it’ll be ItemID,Amount,AmountStolen
but there are tons of features like customisation and lists of lose items left lying around your house like in skyrim, I’d have to record ItemID,Orientation,Poisition,MapLocation.
As well as all perk trees, stats and more. I will use simple datastores until I need to scale above 260 KB, but if you look at skyrim’s save files it ads up to around 26 MB of data, since we are recording the same sorts of things that’s what it could be. I will be compressing as much as I can though and despawning loot in chests after a certain amount of ingame time.
2 Likes
Hia. As someone who is constantly dancing on the edge of reasonable use of datastores, I’m here today to tell you that there are no data limits for DataStores!!
The only limit is a individual value character limit. You can have no more than 260,000 characters in a single datastore value. Your data is automatically converted into a JSON string when you save to datastores and you can use HttpService.JSONEncode to see how many characters your files would have.
23 Likes
Yes but splitting it into 40 datastore values would use up a lot of the allowance for GetAsync and SetAsync, both at 60 + (10 * players) per min respectively.
If you’re using 260,000 for a single save file… I’m sorry but you are doing something wrong.
The most I’ve ever used for my game is around 200,000 when someone placed over several thousand potted plants on their base and I had to save the position and rotation of each plant.
This was before I learned about compressing floats from something like 2.4999999999999999984 to 2.5. If you are using any floats (non-integer numbers) in your data files you should compress them with something like this:
-- converts a long and messy float into a neat and short STRING
-- there's probably a better way to do this though
local function formatfloat(float)
local str = tostring(float)
local pos = str:find("%.")
if pos then
local pre = str:sub(1,pos-1)
local suf = str:sub(pos+1, pos+2)
if pre:len() > 0 and suf:len() > 0 then
local sufval = tonumber(suf)
local preval = tonumber(pre)
if sufval and preval then
local inc = 1
if str:sub(1,1) == "-" then
inc = -1
end
suf = "0"
if (sufval >= 40 and sufval <= 60) or (sufval >= 4 and sufval <= 6) then
suf = "5"
elseif sufval >= 9 or sufval >= 90 then
preval = preval + inc
end
return (tostring(preval) .. "." .. suf)
end
end
end
return float
endedit: this function rounds to the nearest .5
18 Likes
For the average item it will take this much data [999,99,99] instead of [“ItemID”:999,“Amount”:99,“Stolen”:99] so each inventory record would take approx 11 bytes, 12 if you include the “,”. And so say you had 1000 items in your inventory, that’s 12kb, and you had 200 chests recently looted item 50 items remaining in each, that’s another 120kb (200 x 50 x 12b). now we have character customisation, 1kb max ish, skills, perks, stats total around 2kb.
Now I’m at 135 KB.
Now for quests, format being [QuestID,QuestPos] or [999,99] or 8 + 1 for comma = 9 B per quest,
after a long play time you may have done 200 quests so that’ll be 1.8 KB = 136.8 KB
Now for player houses, let’s say the base house details take 1.2 KB for the main part of, = 138 KB
Now I have to account for each lose item, [ID,Amount,X,Y,Z,LocationID] or [999,99,999.9,999.9,9,999] = 26 B + 1 for comma = 27 B, some players with a ton of stuff lying around (200) = 5.4 KB = 142.2 KB
That’s less than I thought, but that’s just for the game base, later on there will be tons more which could push it into the 1 Mb - 3 Mb area
EDIT: 1 - 3 mb is actually not that bad, as that’s not many shards
1 Like
You could also use string.format for this. e.g
print(string.format("%.4f", math.pi))
9 Likes
There is absolutely no reason for you to be saving key names for every single entry to the table. You already know what each entry is and you are just wasting space.
[“ItemID”:999,“Amount”:99,“Stolen”:99] -> [999,99,99] already saves you a massive amount of space.
Edit: There’s also no reason for you to be recording information about items the player doesn’t have. Save space by not putting in a table record for any items where the amount and stolen are 0
2 Likes
Yes, that’s what I said, that I was already compressing it down to [999,99,99] as an array instead of a dictionary
Edit: I’m not recording anything they don’t have, it’s an array of [id,amount,stolen],[id,amount,stolen],[id,amount,stolen] etc
Oh gotcha, my bad. I think if you limit the amount of items someone can have in their inventory (reasonable for an mmo to do) you shouldn’t come anywhere near hitting the character limit. If you do, you can just split the base-building stuff into its own datastore.
1 Like
I’m actually adopting your ordereddatastore method for saving to make sure it works properly too, as losing 100 hours of playtime data with items you paid for would he a nightmare for players
1 Like
Another trick is to get rid of redundant item ids by nesting item attributes tables in a single table per item. My keys look like this:
- Item 1
- {Amount0, Stolen0}
- {Amount1, Stolen1}
- {Amount2, Stolen2}
- Item 2
- {Amount0, Stolen0}
etc.
But yeah I’m also questioning what kinds of stuff you’re trying to save that you need 40 keys for.
1 Like
I have to save player data over multiple keys in Farming among Friends but I think only a few users ever had so much data that it was really necessary. I never thought about floats being saved as crazy decimals so I’m sure that hurt the size of the data. I also just stored plain JSON instead of some compressed format and that also doesn’t help.
@berezaa @TheGamer101
This is unrelated to the OP but since you guys posted here, I’ll throw it out there. Can’t you just use math.abs(num) to get rid of floating point errors? I did that for the values of NumberValue objects.
1 Like
Appears not:
local http = game:GetService("HttpService")
local str = http:JSONEncode({0.1+0.2})
print(str)
-- [0.300000000000000044408920985006]
local str = http:JSONEncode({math.abs(0.1+0.2)})
print(str)
-- [0.300000000000000044408920985006]
1 Like
you can convert to string and truncate after 1 decimal place
I know, but I was directly referring to the proposed solution of math.abs… which doesn’t work.
What about splitting up individual types of data into different keys/scopes/stores?
Inventory could have its own store, with each item having its own key (if necessary, if not, just keep all under one key - and perhaps limit the total number of items available in one’s inventory?)
Quests could have it’s own datastore. One key could be used for the quests a player is currently doing, and another could be used for the quests already completed (solely the IDs - and for current quests possibly the progress or whatever suits)
Player House data can have its own store, saving only the necessary information.
I don’t know how recommended it is to use separate stores, it may be frowned upon, I don’t know. Though, especially if you do need a lot of data, it may be a viable solution perhaps?
1 Like
There is indeed a limit for requests. This is why rather than having a load of separate stores you can put them as a list.
— Below is an example of roughly how I would store the information
local Inventory = “”
for i = 1,#Items do
local NewString = tostring(itemID) … “:”
NewString = NewString … tostring(itemcount) … “:”
NewString = NewString … tostring(AmountStolen)
Inventory = Inventory .. Newstring .. ","
end
---- Below is how to get the data from the string
local ItemData = string.split(String,“,”)
for i = 1,#ItemData do
local Data = string.split(ItemData[i],“;”)
local ItemID = Data[1]
local ItemCount = Data[2]
local AmountStolen = Data[3]
— Put the information where it needs to be
end
1 Like