How does it work?
Table To Buffer works by making a “Serializer”, which is just a list of instructions that runs some operations like reading a string, or reading a number, they all automatically offset to make it as painless as possible, but all of that is not exposed to you, you’re only given a simple builder with descriptive methods to make a serializer for your table.
Table to buffer has two ways of making a serializer.
Automatic
- Your table will automatically be “analyzed” and a Serializer will be created for it. The serializer this creates is very unoptimized, for example, all numbers it encounters it will assume them to be double precision floats, this is good for one-off operations, but for saving data, it is bad.
Manually
- You will manually make a serializer for it by using the builder pattern, which allows you to define the keys that the table has, and give it a data type. The order in which you place the calls matters, as that is the order the serializer will read and write to the buffer.
“ByteNet exists”
Yes, but ByteNet is meant to be for networking purposes. Perhaps you’re storing a lot of information on a DataStore, then, this project may help you save some of that storage to be able to cram even MORE into it before resorting to making another DataStore
Table to buffer will also, when possible, compress the strings it receives using Rochet2/lualzw: A relatively fast LZW compression algorithm in pure lua (github.com). However, strings may not be compressed if it is not beneficial (i.e: String size is bigger). The current implementation of the strings allows me to add more than one compression algorithm, so in the future perhaps LZ4 or other compression algorithms can be added, and given as an optional parameter when defining the string.
Table as Buffer
This is mostly an experiment, as such, it is not meant to be used in production, so if you choose to try this module, avoid using it other than playing around with it.
What is the idea behind this “Table as Buffer”?
The concept is simple. You will give it a starter table, and it will automatically make a serializer for it.
The function will then return a proxy table, which reads and writes from a backing buffer. This would aid in protecting from some exploiting functions like getgc(), which can be used to look into what you have created on your game (tables, strings, threads…).
The buffer would prevent them from finding any table with the strings as keys, thus rendering getgc() partially useless as buffer as table prevents it, since the buffer is the one with the information and not a table. They would have to understand the binary format of the buffer it to try something with it even, and then they risk completely breaking the game due to reading and writing from it incorrectly, it also aids that the string is compressed (and we can switch the compression algorithm quite easily). If the concept makes sense, and works in practice.
Looks and sounds cool, how to use?
The project has been documented with moonwave → Table To Buffer | TableToBuffer
Project Source Code → SecondNewtonLaw/TableToBuffer (github.com)
This project was also going to encompass some instances, such as CFrame, but I limited it to only being primitives due to the jump in complexity it would have had. (If you look at the old commits, it was going to have instances, but back then it was named something else)
This is my first ever package, so it may have some errors or questionable design, so if you have any constructive criticism, go ahead.
Additional Credits:
- Rochet2/lualzw: A relatively fast LZW compression algorithm in pure lua (github.com) - Compression used for strings.
