I'm so EXCITED about my compression

My game homebuilder constantly ran into this error before I was using LZW compression, multiple people would get this issue daily.
error while saving: 105: Serialized value converted byte size exceeds max size 64*1024 bytes.
After I implemented LZW, there hasn’t been a single instance of that error being thrown. I log all saving errors on my discord server using webhook.

Nice! I wonder if you could pack it down even smaller by running RLE first.


You just gave me an Idea, I know how I could optimize this WAY WAY more, for RLE
If I broke up my Vec3 positions into individual lists where the position on that list kept correspondence with (for instance) the Y and Z lists, I could WAY shorten the amount of space that positions take up simply by grouping items which are “snapped” together

So this is a 2x3 of walls

instead of saving like it currently does which would look like this

[1] = {["X"] = 2,["Y"] = 0,["Z"] = 1,},
[2] = {["X"] = 2,["Y"] = 0,["Z"] = 2,},
[3] = {["X"] = 2,["Y"] = 0,["Z"] = 3,},
[4] = {["X"] = 2,["Y"] = 1,["Z"] = 1,},
[5] = {["X"] = 2,["Y"] = 1,["Z"] = 2,},
[6] = {["X"] = 2,["Y"] = 1,["Z"] = 3,},

I could save it like this

["X"] = "2*6",
["Y"] = "0*3 1*3",
["Z"] = "(1,3)*2",

Right now I’m not doing anything special for the Vec3s (aside from rounding them to the nearest 1/10th), mostly because the multi-dimensional approach wouldn’t help that much since I can’t predict which axis players are going to prefer(although I do have it set up so it could detect the best compression and method and save it like that) but hmmm yes

Thanks for the idea!


Would there be any interest in us exposing some compression utilities, like deflate and/or lz4? We have some seriously fast compressors available in the engine and we use them for various tasks, like compressing file data, network traffic, etc.


Yes please

1 Like

Always :slight_smile:

1 Like

I’m in.

1 Like

That’d be amazingly helpful to a lot of us!

What game?

It’s a sequel to

But I’ve decided to do it over from the ground up and do pretty much everything better. The new game should be out before the end of January.

i was looking around on google’s youtube channel as they have REALLY awesome videos, saw this and it reminded me of ya :stuck_out_tongue:


An update on this since I’ve launched my game. A player made THIS plot

It’s ~100,000 part count total

Saved plot after 1.15 seconds.[57737|260000 - 22.21%]

22.21% of the maximum save size in a datastore key


Soooo, it’s been a year, and this post has a lot of likes… any chance of looking into doing this still?


I too would like such a feature since I’m making a game that could use it (and never attempted to make my own compression sytem before…)


In the meantime, the general text compression I used (on top of my own custom stuff for this specific use case) is available here Text compression


I can’t find the post where this was mentored, but basically someone wanted to make a working camera with Viewport Frames, however the compression needed to store such data to a database and retrieve them would need to be something like this.

1 Like

Bumping this again, is there any plans for this in the future?

Would be pretty useful, currently we use a custom implimentation of LZW and string.pack for reducing network load, however compression like this could allow for more possibilities.


Hey is there any update on this? I would love to have some compression utilities that I can use on my game.

1 Like

That is really amazing and this would benefit so many developers

1 Like

Bumping this, I really want to have fast decompress and compression methods built-in for chunk serialization and deserialization for my voxel projects, it would really help a lot since speed is crucial for real-time rendering. Hopefully this has not been forgotten.