DataSerializer
IF YOU FIND A BUG, PLEASE LET ME KNOW BELOW
A package for compressing/decompressing data with support for DataStore.
Downloading
Wally (recommended)
Versions 1.*.* assure backwards compatibility
DataSerializer = "ohimesan/dataserializer@1.1.7"
Creator Marketplace
Benchmark
All tests taken with Base64 encoding and max compression (Zlib level 9)
--------------------------------
Testing "API-Dump"
Serialized in: 4254ms, 520 chunks
Deserialized in: 50ms
Original size: 1167706
Serialized size: 321833
Comparison: 28% of original value
--------------------------------
Testing "PlayerData1"
Serialized in: 14ms, 5 chunks
Deserialized in: 1ms
Original size: 6937
Serialized size: 4364
Comparison: 63% of original value
--------------------------------
Testing "PlayerData2"
Serialized in: 35ms, 14 chunks
Deserialized in: 1ms
Original size: 16870
Serialized size: 8359
Comparison: 50% of original value
--------------------------------
Testing "PlayerData3"
Serialized in: 2ms, 2 chunks
Deserialized in: 1ms
Original size: 1450
Serialized size: 1039
Comparison: 72% of original value
--------------------------------
Tests made: 4
Average percentage: 53%
How does it work?
Expand
Bitbuffer
The BitBuffer module with some additions is used to serialize values
VarInts
VarInts are variable-length integers. They are good because as the number increases, the length also does. Making short numbers use less amount of data.
Reutilizing
This is the most important part, reutilizing values. Any kind of value reference.
There are a lot of dictionaries with repeating indexes, and it’s good to shorten it to ~2 bytes.
Documentation
Expand
These are all the values currently supported
export type SupportedType =
nil
| boolean
| number
| string
| { [SupportedType]: SupportedType }
| BrickColor
| Color3
| CFrame
| Vector3
| Vector2
| UDim2
| UDim
| Ray
| Rect
| Region3
| Enum
| NumberRange
| NumberSequence
| ColorSequence
Serializes a data and returns the chunks
input is the input, can be any value in SupportedType.
zlibLevel is the level of compression, the higher the lightest, but slowest. Use 0 for disabling, 9 for max compression. Defaults to 6.
useBase64 is whether the data will be encoded into valid characters or not. Base64 makes the result ~33% heavier, but does the job.
Keep it enabled if you are storing the output in a DataStore, as it only accepts valid characters. Defaults to true.
Deserializes the previously serialized data
input is the previously serialized data
usesBase64 is whether the input is using Base64 or not.
Example code
local DataSerializer = require(DataSerializer)
local playerData = {
Admin = true,
Values = {
Coins = 2000000,
Diamonds = 7000,
Skills = {
"Thorns",
"Flying",
"X-Ray"
}
}
}
-- Returns an array of strings, that you can save in DataStore
-- to deserialize later
local encoded = DataSerializer.Serialize(playerData)
DataStore:SetAsync("Player_1", encoded)
local DataSerializer = require(DataSerializer)
local encoded = DataStore:GetAsync("Player_1")
local playerData = DataSerializer.Deserialize(encoded)
Update log
1.1.4
Improved Base64 performance
I switched the encoding/decoding functions to
fast_base64.lua · GitHub
