So i was working on my datastore, but i have to save tons of data, and i was wondering if it is possible/optimized to store data in seeds.
Lets just say i have a table:
local tool = {
NameID = 23, -- a name in id
Damage = 10,
Range = 9,
}
The tool will be randomy generated, so the Damage, range, name will become different every time. (Note: i am saving way more data than this, this is just an example)
So lets say we have an inventory:
Inventory = {
tool, -- the other varable i just made
{
NameID = 12,
Damage = 56,
Range = 1,
},
}
So instead of doing this, i would like to save it in numbers like this:
Inventory = {
17529365,1237128375187 -- the seeds
}
Is this optimized/good? And if it is, how can i achieve this?
If not, please give me other ideas of how to save tons of data!
Wait, isn’t this text compression? I am not sure how i can use this in this case. Would you explain to me how? (but thanks for the link, it should be usefull to me later)
(nm looking into the first link you sended)
Also i go to sleep in a few minutes so you know i why i won’t reply.
If I understand you correctly, you’re are saying that SeedA generates an item with some properties, since that seed always generates the same item, you can save the seed instead of a list of the items attributes. This is clever but you are going to have an incredibly rough time if you ever need to change the random function the generates the items from the seeds.
If you want to change the random generation later, its going to be almost impossible to make sure that everyone’s existing items stay the same. The only way I can think of doing it is keeping every old generator version’s code and including which version was used with every item. I don’t think that’s worth the memory saving.
What the other commenters are talking about is encoding your data as text and then (optionally) running it through a text compression algorithm to save storage.
Your idea of deterministically generating the data from a seed is totally sound and could save a lot of storage, assuming you have lots of generated data. Someone pointed out one potential pitfall with this: if you change the code that generates things, players’ saved items will change too!
So if you have a sword generator, you can use it like this:
function generateSword(seed)
local r = Random.new(seed)
local sword = {
seed = seed,
power = r:NextNumber(5, 20)
... many more properties
}
return sword, seed
end
local r = Random.new()
function givePlayerSomeSwords(player)
for _ = 1, 3 do
local sword, seed = generateSword(r:NextInteger())
--store in the players' runtime inventory
end
end
function loadPlayerInventory(player)
local compressedInventory = getPlayerCompressedInventory(player)
local inventory = {swords={}}--reconstruct runtime inventory from compressed data
for _, seed in ipairs(compressedInventory.swords)
table.insert(inventory.swords, generateSword(seed))
end
return inventory
end
function savePlayerInventory(player)
local inventory = getPlayerInventory(player)
local compressedInventory = {swords={}}
for _, item in ipairs(inventory.swords)
table.insert(compressedInventory.swords, item.seed)
end
return compressedInventory
end
I am Working on that TextCompressen modulescript but for some reason it doen’t work. I have no idea why, can you send the modulescript and how to enable it.
local Module = {}
function Module.GetCode(text)
local dictionary, length = {}, 0
for i = 32, 127 do
if i ~= 34 and i ~= 92 then
local c = string.char(i)
dictionary[c], dictionary[length] = length, c
length = length + 1
end
end
local escapemap = {}
for i = 1, 34 do
i = ({34, 92, 127})[i-31] or i
local c, e = string.char(i), string.char(i + 31)
escapemap[c], escapemap[e] = e, c
end
local function escape(s)
return (s:gsub("[%c\"\\]", function(c)
return "\127"..escapemap[c]
end))
end
local function unescape(s)
return (s:gsub("\127(.)", function(c)
return escapemap[c]
end))
end
local function copy(t)
local new = {}
for k, v in pairs(t) do
new[k] = v
end
return new
end
local function tobase93(n)
local value = ""
repeat
local remainder = n%93
value = dictionary[remainder]..value
n = (n - remainder)/93
until n == 0
return value
end
local function tobase10(value)
local n = 0
for i = 1, #value do
n = n + 93^(i-1)*dictionary[value:sub(-i, -i)]
end
return n
end
local function compress(text)
local dictionary = copy(dictionary)
local key, sequence, size = "", {}, #dictionary
local width, spans, span = 1, {}, 0
local function listkey(key)
local value = tobase93(dictionary[key])
if #value > width then
width, span, spans[width] = #value, 0, span
end
sequence[#sequence+1] = (" "):rep(width - #value)..value
span = span + 1
end
text = escape(text)
for i = 1, #text do
local c = text:sub(i, i)
local new = key..c
if dictionary[new] then
key = new
else
listkey(key)
key, size = c, size+1
dictionary[new], dictionary[size] = size, new
end
end
listkey(key)
spans[width] = span
return table.concat(spans, ",").."|"..table.concat(sequence)
end
local function decompress(text)
local dictionary = copy(dictionary)
local sequence, spans, content = {}, text:match("(.-)|(.*)")
local groups, start = {}, 1
for span in spans:gmatch("%d+") do
local width = #groups+1
groups[width] = content:sub(start, start + span*width - 1)
start = start + span*width
end
local previous;
for width = 1, #groups do
for value in groups[width]:gmatch(('.'):rep(width)) do
local entry = dictionary[tobase10(value)]
if previous then
if entry then
sequence[#sequence+1] = entry
dictionary[#dictionary+1] = previous..entry:sub(1, 1)
else
entry = previous..previous:sub(1, 1)
sequence[#sequence+1] = entry
dictionary[#dictionary+1] = entry
end
else
sequence[1] = entry
end
previous = entry
end
end
return unescape(table.concat(sequence))
end
return {compress = compress, decompress = decompress}
end
return Module
You’re lacking require().
Why would you optimize the data if it makes such little difference?
Data limit for a single data store value is 4MB meaning you can store 4,000,000 standard characters per value.
Let’s say a JSON encoded table for a single tool is so huge that it takes 1,000 bytes.
Then, you’ll still be able to store almost 4,000 tools.
Unless you necessarily need such extreme quantities, I wouldn’t recommend overcomplicating the task.
What you are doing in generate sword, is sending the seed again back, so it can be used again. But is there a way, you can get a seed, depending on the stats in the table? So you can turn every single table into a seed, and then return it into a table (or string)?
Also you forgot typing do after the for loop.
for _, item in ipairs(inventory.swords) do -- you forgot to type the "do"
What you’re looking for is modular arithmetic. If all of the max values are the same (I.e. NameID can be max 100, Damage can be max 100, Range can be max 100) You can do this:
Ideally you would encode and decode the output with base93.
But if the maximum values are different you can’t do this.
Please note that you probably shouldn’t/can’t use this for everything. For example vector3’s don’t have a max. And if you set the max value too high (E.G. 10000000) it’d be boatloads of times worse than just plain ol’ text encoding.
This is probably what you’re looking for, but not what you should use.
