Tables and buffers [TaB] is a module made for easy conversion of primitive data from tables into buffers. It is often useful in case of sending data over client-server boundary and storing a lot of information, where memory size matters.
Benefits
less memory occupied on average, good for huge data size
portable and easy to send over client-server boundary
can represent nested and mixed tables of basic data
Limitations
max string length and table size is 4095 elements/characters
I recognize this is not the fastest solution, nor the most memory efficient, but it strikes a good balance. If you intend to use this, notify me if you want bugs fixed or profiling.
Possible features
adding support for non-primitive data types
more optimizations
Documentation
encode(tbl_in: {any}, optimized: boolean): buffer
Convert table tbl_in into buffer.
If optimized is true, less memory will be used for headers of tables with same elements.
decode(buf_in: buffer): {any}
Convert buffer buf_in back into table.
Configuration
ERROR_MODE: boolean
You can enable errors instead of warnings by enabling this attribute. In case it is false, the module will try to continue execution, but may still error.
Download
You can get the updated module from the creator marketplace here.
I have made a new version which now has some new features, but I feel like I should explain how the conversion actually happens first to make the features understandable.
Explanation
So each item in a table gets 2 bytes (or 16 bits) for its data type information and its size (or length). Previously 1 byte was used by data type and 1 byte by size information. But there is no way this module will reach 256 data types, so now it’s taking up only 0.5 of a byte by default for a data type! Therefore, each element can be of size (or length) up to 4096, instead of measely 256. You can configure the bits needed for data type information (don’t go lower than 3) to adjust the balance between data variety and data size.
Features
bit size of element data type can now be configured
by default 4095 is the maximum element size/length
module now supports booleans
module also supports instance references (represented as strings, so long names lead to more bytes needed for encoding)
P.S. So far I have just uploaded the module versions to the asset store, but I may change to something like git if new versions keep coming. I am adding updates because I am going to use this in my development and I am also sharing it with you for free. You can write your suggestions in the comments and support this module by sharing and liking it.