I'm so EXCITED about my compression

Nice! I wonder if you could pack it down even smaller by running RLE first.

:open_mouth:

You just gave me an Idea, I know how I could optimize this WAY WAY more, for RLE
If I broke up my Vec3 positions into individual lists where the position on that list kept correspondence with (for instance) the Y and Z lists, I could WAY shorten the amount of space that positions take up simply by grouping items which are “snapped” together

So this is a 2x3 of walls

instead of saving like it currently does which would look like this

[1] = {["X"] = 2,["Y"] = 0,["Z"] = 1,},
[2] = {["X"] = 2,["Y"] = 0,["Z"] = 2,},
[3] = {["X"] = 2,["Y"] = 0,["Z"] = 3,},
[4] = {["X"] = 2,["Y"] = 1,["Z"] = 1,},
[5] = {["X"] = 2,["Y"] = 1,["Z"] = 2,},
[6] = {["X"] = 2,["Y"] = 1,["Z"] = 3,},

I could save it like this

["X"] = "2*6",
["Y"] = "0*3 1*3",
["Z"] = "(1,3)*2",

Right now I’m not doing anything special for the Vec3s (aside from rounding them to the nearest 1/10th), mostly because the multi-dimensional approach wouldn’t help that much since I can’t predict which axis players are going to prefer(although I do have it set up so it could detect the best compression and method and save it like that) but hmmm yes

Thanks for the idea!

4 Likes

Would there be any interest in us exposing some compression utilities, like deflate and/or lz4? We have some seriously fast compressors available in the engine and we use them for various tasks, like compressing file data, network traffic, etc.

178 Likes

Yes please

1 Like

Always :slight_smile:

1 Like

I’m in.

1 Like

That’d be amazingly helpful to a lot of us!

What game?

It’s a sequel to

But I’ve decided to do it over from the ground up and do pretty much everything better. The new game should be out before the end of January.

i was looking around on google’s youtube channel as they have REALLY awesome videos, saw this and it reminded me of ya :stuck_out_tongue:

8 Likes

An update on this since I’ve launched my game. A player made THIS plot


It’s ~100,000 part count total

Saved plot after 1.15 seconds.[57737|260000 - 22.21%]

22.21% of the maximum save size in a datastore key

11 Likes

Soooo, it’s been a year, and this post has a lot of likes… any chance of looking into doing this still?

30 Likes

I too would like such a feature since I’m making a game that could use it (and never attempted to make my own compression sytem before…)

2 Likes

In the meantime, the general text compression I used (on top of my own custom stuff for this specific use case) is available here Text compression

3 Likes

I can’t find the post where this was mentored, but basically someone wanted to make a working camera with Viewport Frames, however the compression needed to store such data to a database and retrieve them would need to be something like this.

1 Like

Bumping this again, is there any plans for this in the future?

Would be pretty useful, currently we use a custom implimentation of LZW and string.pack for reducing network load, however compression like this could allow for more possibilities.

13 Likes

Hey is there any update on this? I would love to have some compression utilities that I can use on my game.

1 Like

That is really amazing and this would benefit so many developers

1 Like

Bumping this, I really want to have fast decompress and compression methods built-in for chunk serialization and deserialization for my voxel projects, it would really help a lot since speed is crucial for real-time rendering. Hopefully this has not been forgotten.

4 Likes

Absolutely! I would love having access to data compression functions!

Re-inventing the wheel every time is so tedious. :frowning:
Please expose those functions!

1 Like