As a Roblox developer, it is currently hacky to utilise zstd compression within the engine. You can currently acheive this by calling JSONEncode with a buffer and retrieve it from the returned JSON string, however, this is hacky because it relies on the implementation detail here with regards to how its formatted (also it encodes as a Base64 string).
The method for which is as follows:
Identify the compressed stream in the JSON string, using JSONDecode doesn’t work because it decompresses the buffer.
De-encode the Base64 string
Instead, it would be nice to simply compress the buffer directly without needing to JSONEncode it and doing this hacky workaround.
If Roblox is able to address this issue, it would improve my development experience because I could compress and decompress buffers without needing to rely on a HttpService method that itself is an implementation detail.
Well its good thats not what I’m asking for. JSONEncode supports buffers as an argument (this isn’t documented anywhere), it sometimes compresses them as well!
This is likely a side effect of other Roblox methods that internally call the same JSON encoder, as you can see, it still works regardless, the z part of zbase64 indicates it was compressed.
im very interested in this could you explain more on how you are compressing a buffer into something even smaller i presume? that could be really helpful to know for networking
Is this service safe to use with datastores? Could the implementation change and potentially break saved compressed data? This service would benefit greatly from an announcement if it’s built with datastore use in mind because this is a huge and historically very difficult to accomplish use-case. I don’t see this use-case documented at all which is a big issue for someone looking to use it. It’s very important to confirm or flag whether this is safe for use with datastores.
Why was only zstd chosen as an algorithm, a lot of formats on the internet use zlib, and Roblox’s own file format uses lz4 (though I’ve seen some files with zstd).
