T - A lightweight library for compression and type-enforcement

Resources Community Resources
#1

Recently, I’ve been finding myself intrigued in data compression and I wanted to make a library that could compress data alongside preserving their types with much flexibility and many types and also give detailed error messages. I’ve cooked up a small library for this and the use of it is very simple.

Primitive Types:
T.uint8-T.uint64 (with 1 byte steps, uint48+ start to have precision issues at large values)
T.int8-T.int64 (with 1 byte steps, however, int48+ have precision issues due to roblox representing numbers as floats,)
T.float8, T.float16, T.float32, T.float64
T.string_datastore_safe, T.string_ascii, T.string_utf8
T.string (alias for string_datastore_safe as it is the most restrictive)
T.flag (a boolean, coerces truthy and non-truthy types, nil -> false)

T.any (cannot be serialized)
T.primitive (can be serialized but only supports the primitive types)

All of these are of type: typer.

These are the primitive types and the building blocks of the entire system, you can combine these primitives alongside the ‘extender’ functions which are:

T.optional(typer) -> typer
T.many(...typer) -> typer
T.array(typer) -> typer
T.fixed_size_array(typer, size : number) -> typer
T.dynamic_array(typer) -> typer -- there is an additional bit of overhead per element in this type of structure.
T.struct({
  [any]: typer
}) -> typer

Thats cool and all, but how do we actually enforce these types or compress them?

T.pack(typer, value : any) -> buffer_data, --will error if value cannot be packed, 
T.to_sendable(buffer_data) -> (flag_buffer : buffer, raw_buffer : buffer, extra_args : {any}) -- this is for remotes, flag_buffer is densely populated while raw_buffer is sparsely populated. extra_args for unserializable items.
T.to_string(buffer_data) -> string

T.from_string(string) -> buffer_data
T.from_sendable(flag_buffer : buffer, raw_buffer : buffer, extra_args : {any}) -> buffer_data
T.interpret(typer, buffer_data) : any

A small example of compression:

local petIdModification = T.struct({
    PetId = T.string,
    ShouldDelete = T.flag,
    ShouldMakeShiny = T.flag,
    ShouldEquip = T.flag,
})
local packet = petIdModification({PetId = "xxxxx-xxxx-xxxx-xxxxx", ShouldDelete = true})
PetModifyRemote:FireServer(T.to_sendable(packet)) -- or to_string also works in this case.

Use cases:
Two use cases come to mind, and they’re to do with either remote or datastore usage. Data may be compressed for both to reduce payload size. Also, unreliable remotes may benefit greatly from compression due to their byte limit.

The module is fully typed, your types are preserved.

Here is the module: https://create.roblox.com/store/asset/101646425935658/T

3 Likes