What do you want to achieve?
I would like to be able to encode this table into JSON.
What is the issue?
HttpService will frequently give the “Can’t convert to JSON” error while attempting to encode.
What solutions have you tried so far?
I’ve reduced the size of the table, though I believe it may be an issue with the fact that my compression system relies on using every single character (0-255) in ASCII for compression purposes. My game’s terrain generation requires a lot of storage to save properly so it has to be able to compress everything down very well.
If it is because of certain ASCII characters, could someone please list them so that I can avoid them, and are there better ways to compress data? If it is not, does anyone know the actual reason?
Each line of “unreadable nonsense” represents XYZ position (within a chunk) and material + occupancy. X and Y are the chunk coordinates, while Objects contains all objects within the chunk.
EDIT: Objects only contains name, position, and rotation, not actual instances.
I’m sure there is a better way to go around this w/o needing to encode the data before serialisation, as far as I can tell your encoding entries in the json table, instead why not encode the json string returned from EncodeJSON?
Would be more performant imo since you’re eliminating a bunch of different JSON elements & various other factors you can’t achieve through this implementation.
As far as I can tell, roblox’s JSON only accepts valid codepoint characters, and what I believe you might be trying to parse is byte characters? Though I might be totally wrong on this and my interpretation might be flawed
The data has to be encoded or else it uses too much space. Each chunk contains ~2 million cells which need to have 5 different values attached to them, the X, Y, Z, material, and occupancy values. This means it will take an absolute minimum of 5 bytes per cell, meaning 10 megabytes to contain a fully edited chunk (which is nearly impossible to actually do in-game but I’d like to prepare for the worst). The external database I use can only hold a maximum of 16 MB per chunk, so it’s cutting it rather close.
I guess I could encode it after serialization but that seems like it wouldn’t be as efficient since it’s manipulating a huge string rather than a table but I could try if there’s no other way. I also would like to avoid it mainly because I’ve never really edited a string in that way and would prefer a table if possible since strings were not really made for that purpose.
As for byte characters, yes, I am using them if I understood correctly. The characters are generated from binary converted to raw numbers and then converted to ASCII characters with string.char().
I guess I could encode it after serialization but that seems like it wouldn’t be as efficient since it’s manipulating a huge string rather than a table
This would spike CPU usage, but packing it up like this would much better improve the odds of your data as the more repeated data there is, the shorter the string itself.
to avoid it mainly because I’ve never really edited a string in that way and would prefer a table if possible since strings were not really made for that purpose.
Strings were made to contain codepoints, at the moment your story bytes into strings which isn’t exactly the usage of a string but is acceptable.
The characters are generated from binary converted to raw numbers and then converted to ASCII characters with string.char()
Sure, i’d go through this to see at what point what specific character causes the JSON method to fail.
HttpService:JSONEncode() will only successfully encode the original ASCII characters (0-127). It will throw an error for everything else on up (128-255). You technically only lose one bit there, but that may end up requiring more bytes in the end, and not all characters are legal, so some will be escaped in the resultant string.
