serde | Schema-Based Serialization & Deserialization
serde is a serialization/deserialization library that uses schemas to convert structured data into compact binary buffers and back. Its buffer-based design enables fast processing and minimal output size, making it ideal for networking, DataStores, and any scenario where efficient data representation is critical. Although not exactly built for speed, serde aims to support whatever you throw at it.
serde currently supports all base creatable roblox and luau datatypes!
Use
serde is schema-based, meaning you describe how your data is structured through various data-types, and the library will perform actions according to your schema (think of it like instructions on how to put together and take out data!)
As forementioned, serde supports all base creatable datatypes with a few exceptions. See here:
Exceptions & Caveats
Instance- To prevent a dangling pointer of sorts while referencing anInstance,serdewill only serialize the properties of the object. You can specify which properties you wish to be serialized / deserialized.Object- This datatype is not creatable.Content- TheObjectproperty is not supported, however,Contentcan still be used, just not if it uses theObjectproperty. If it does,Content.nonewill be used in place of it.DockWidgetPluginGui- Property access on this instance seems to be disabled.OverlapParams- Instances in the instances array are not serialized; and so it will be empty.RaycastParams- Ditto.RaycastResult- TheInstanceproperty will be ignored. Due toRaycastResults not being creatable, a fake struct resembling one will be returned instead.RBXScriptConnection: The Instance is not creatable, and so a struct resembling the instance will be returned instead.RBXScriptSignal: This datatype is not creatable.Secret: Ditto.SharedTable: Can have any content; Some of which may not be serializable / deserializable, and therefore it has been excluded.
All data-types can be found under the main module. Each data-type has its own Converter<T> interface, where T is the datatype. A Converter<T> is used to serialize and deserialize data. To actually serialize and deserialize, the module exports a serialize and deserialize function which respectively do as their name says. Hereâs an example of serializing a string:
local serde = require(path.to.serde)
-- `serde.string` is a `Converter<string>`, meaning it serializes the `string` datatype
local serialized = serde.serialize(serde.string, "Hello, World!") -- `buffer`
local deserialized = serde.deserialize(serde.string, serialized) -- "Hello, World!"
serde has also built-in a few complex data-types ({T}, {[K]: V}, structs, tuples, etc) for ease of use. These complex functions, given the layout of the object (that be an array, dictionary, etc), return a newly made Converter resembling the data. This is how it looks in action:
Important
Using astructis almost always smaller and more efficient that using adictionary. This is because the keys are known beforehand and so they do not need to be stored in the buffer (unlike in adictionary). For this reason, only the value of each key is stored, making astructthe same as atuple)
Arrays, Dictionaries, Structs, & Tuples
Arrays
local serde = require(path.to.serde)
local string = serde.string
local array = serde.array
-- `array` takes a `Converter<T>` and returns a new `Converter<{T}>`
local arrayOfStrings = array(string)
local serialized = serde.serialize(arrayOfStrings, {"Hello, ", "World!"})
local deserialized = serde.deserialize(arrayOfStrings, serialized)
-- ^^^ `{"Hello, ", "World!"}`
Key-Value pairs (dictionaries)
local serde = require(path.to.serde)
local string = serde.string
local number = serde.number
local dictionary = serde.array
-- `dictionary` takes a `Converter<K>` & `Converter<V>`
-- and returns a new `Converter<{[K]: V}>`
local newDict = dictionary(string, number) -- `{[string]: number}`
local myDict = {
["hello"] = 1,
["world"] = 2,
}
local serialized = serde.serialize(newDict, myDict)
local deserialized = serde.deserialize(newDict, serialized)
-- ^^^ `{["hello"]: 1, ["world"]: 2}`
Structs
local serde = require(path.to.serde)
local struct = serde.struct
local string = serde.string
local bool = serde.bool
local number = serde.number
-- `struct` takes a dictionary of fields and their types
local userSchema = struct({
name = serde.string,
age = serde.number,
isCool = serde.bool
})
local user = {
name = "ROBLOX",
age = 19,
isCool = true
}
local serialized = serde.serialize(userSchema, user)
local deserialized = serde.deserialize(userSchema, serialized)
-- ^^^ The same as our `user` variable
Tuples
local tuple = serde.tuple
local string = serde.string
local bool = serde.bool
local number = serde.number
-- Pass as many `Converter`s as you like, of any type!
local myColorSchema = tuple(serde.number, serde.number, serde.number)
local serialized = serde.serialize(myColorSchema, {
255, 127, 65 -- You can also pass a tuple as an array if it is easier. *
})
-- That means that the following code is also valid
--local serialized = serde.serialize(myColorSchema, 255, 127, 65)
local a, b, c = serde.deserialize(myColorSchema, serialized)
print(a, b, c) -- (255, 127, 65)
* - As alluded, you can also pass an array of the values instead of the tuple itself. This makes it possible to have tuple values in dictionaries and structs:
local tuple = serde.tuple
local struct = serde.struct
local string = serde.string
local number = serde.number
-- Pass as many `Converter`s as you like, of any type!
local footInches = tuple(serde.number, serde.number)
local userSchema = struct({
name = serde.string,
height = footInches
})
local serialized = serde.serialize(userSchema, {
name = "ROBLOX",
height = {4, 11} -- 4'11'', haha he's a shortie!
})
local deserialized = serde.deserialize(userSchema, serialized)
print(deserialized) -- `height` will be reserialized as an array,
-- meaning it will be {4, 11}!
For details on how to customise serde to serialize and deserialize your own custom converters, give the following a read:
Implementing your own Converter
Implementing your own Converter
It will greatly help if you are well-versed in how buffers work before reading this section.
serde doesnât limit you to its inbuilt data-types, but also allows you to register your own via the custom method. Hereâs an example use case: Currently all numbers in serde infer what type of integer they should be, and take up 2 or more bytes (the first byte tells the deserializer what type of integer it is, and the rest of the bytes are the integer itself). In this scenario, we assume you want your number to efficiently be stored as a u8 (0 - 255).
A Converter<T> consists of 3 functions: read, write, and getLengthOf. What read and write do is self-explanatory, and getLengthOf returns the length in bytes a value would be if it were allocated. Simply, all that needs to be passed is a dictionary with 3 these functions:
local serde = require(path.to.serde)
local custom = serde.custom
local u8 = custom({
read = function()
end,
write = function()
end,
getLengthOf = function()
end,
})
Since the length of a u8 in bytes is already known (each u8 is 1 byte), the getLengthOf function can be completed by returning 1:
-- ...
getLengthOf = function(v: number)
return 1
end,
-- ...
As for read and write, it will be slightly more complicated. As our u8 is being written from scratch and doesnât depend on any of the in-built converters, we must allocate and write memory ourself using the CursorController, which is passed as the first argument to each function. Letâs start with the write function:
-- ...
-- You may have to explicitly define the
-- type due to Roblox's typechecking not being the best...
write = function(cursor: serde.CursorController, v: number)
end,
-- ...
The cursor is an in-built utility that controls where in the buffer memory is being written. At this current moment, the cursor is conveniently placed exactly where your data needs to start being written. The first thing you do in any function is allocate memory.
Always allocate memory first. If you retrieve the buffer THEN allocate, the buffer will not be updated. Allocate all the memory needed, then start writing.
CursorControl implements the allocateBytes function, and the functions⌠function is self explanatory:
-- ...
-- You may have to explicitly define the
-- type due to Roblox's typechecking not being the best...
write = function(cursor: serde.CursorController, v: number)
cursor.allocateBytes(1)
end,
-- ...
Now all the memory needed has been allocated, all that is left here is writing. Retrieve the buffer via getBuffer, and write a u8 value with buffer.writeu8. The offset of the value is the current position of the cursor (which you can get with getCursor:
-- ...
-- You may have to explicitly define the
-- type due to Roblox's typechecking not being the best...
write = function(cursor: serde.CursorController, v: number)
cursor.allocateBytes(1)
local buff = cursor.getBuffer()
-- Write `v` into `buff` with an offset of the cursor position
buffer.writeu8(buff, cursor.getCursor(), v)
end,
-- ...
However, weâre not done yet! The cursor is still at its original position, and we should set it up for the next piece of data to be allocated! Since 1 byte was allocated, move the cursor forward once with incrementCursor (This means, effectively, for every byte you allocate, remember to move the cursor!):
-- ...
-- You may have to explicitly define the
-- type due to Roblox's typechecking not being the best...
write = function(cursor: serde.CursorController, v: number)
cursor.allocateBytes(1)
local buff = cursor.getBuffer()
-- Write `v` into `buff` with an offset of the cursor position
buffer.writeu8(buff, cursor.getCursor(), v)
cursor.incrementCursor(1) -- Move forward 1 byte
end,
-- ...
At last, the write function is done. Thatâs only the serialization however! This data still needs to be deserialized with the read function. Thankfully since our memory is already layed out, the function will just read the buffer at the cursors location, move the cursor forward for the next data to be serialized, and return our extracted value:
read = function()
-- Get the buffer. No memory needs to be allocated so this is the first thing we do
local buff = cursor.getBuffer()
-- Read the u8 value with an offset of the cursor position
local value = buffer.readu8(buff, cursor.getCursor())
cursor.incrementCursor(1) -- Move the cursor forward 1 byte
return value
end,
The full implementation for the u8 data-type (without comments) should now look like this:
local u8 = custom({
read = function(cursor: serde.CursorController)
local buff = cursor.getBuffer()
local value = buffer.readu8(buff, cursor.getCursor())
cursor.incrementCursor(1)
return value
end,
write = function(cursor: serde.CursorController, value: number)
cursor.allocateBytes(1)
local buff = cursor.getBuffer()
buffer.writeu8(buff, cursor.getCursor(), value)
cursor.incrementCursor(1)
return
end,
})
Here, u8 is now a usable Converter<number>, letâs try it out:
local schema = serde.struct({
name = serde.string,
age = u8
})
local data = {
name = "ROBLOX",
age = 58
}
local serialized = serde.serialize(
schema,
data
)
local deserialized = serde.deserialize(schema, serialized)
âŚAnd it should work - deserialized should have the correct value of 58 (meaning reading and writing works). This is an optimised version of serde.number!
Why use serde over Squash, BufferEncoder or any other library?
Though some alternatives like BufferEncoder may perform marginally faster in microbenchmarks, the difference is insignificant in practical use. For example, in one test with 10,000 iterations, serde took approximately 1.19e-6 seconds per call, while BufferEncoder achieved 5.47e-7. These are microsecond-level differences; too small to meaningfully affect real-world performance (itâs still 836351 calls per second!).
What makes serde different is its high-level, schema-driven design. It abstracts away any low-level concerns like whether a number is signed or unsigned, and handles serialization logic for you. While it may trade a few extra bytes (e.g. adding a type identifier to distinguish between different numeric types), this small cost results in a much cleaner and more maintainable API for the end developer. On top of that, the system is flexible enough to support complex data structures and most Luau types out of the box.
In short, serde gives up a tiny bit of size and speed for a big gain in developer experience. Itâs perfect when you a serialization system without worrying about the nitty-gritty of binary encoding.
Download the latest .rbxm from here: serde.rbxm
