BufferEncoder is a table to buffer serializer that is designed to be as performant as possible while being able to serialize any type of table you want with all datatypes realistically used in tables.
It automatically serializes the table for you without having to define a schema for how the table is serialized - unlike most other serializers publicly available right now - and has very simple API to use.
Features
- Very optimized and space-efficient
- Supports any type of table (array, dictionary, mixed), cyclic tables, and any value for keys in dictionaries
- Supports encoding every datatype that you’d realistically put in a table
- Can encode custom values into 2 bytes if registered beforehand using encoder.enums.register() (eg. userdata from newproxy())
- Can deduplicate repeated tables, strings, numbers, vectors, and enumitems to reduce data size
- Has syncing from server to client for networking purposes
- Has simple encryption that relies on psuedo-random number generation
- Fully typed
Performance
It performs very fast compared to all serializers I’ve found publicly available
It may in some cases be faster than JSONEncode and JSONDecode, depending on the structure of your data
Benchmark
i was only able to find one module that works similar to mine, benchmarking with ones that use schemas isn’t really viable since they dont work the same as mine 
Comparison between BufferEncoder, MessagePack, and HTTPService (native is enabled for both modules)
Code
--!optimize 2
--!native
local HTTP = game:GetService("HttpService")
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local benchmarkmodules = ReplicatedStorage.benchmarkmodules
local BufferEncoder = require(ReplicatedStorage.BufferEncoder:Clone())
local MessagePack = require(benchmarkmodules.msgpack:Clone())
local c = 50
local tab, tab2, tab3, tab4, tab5, tab6 = {}, {}, {}, {}, {}, {}
for i = 1, c do tab[i] = {i, tostring(i), string.rep("a", i), true, false, nil, 4, "ok", true, 5436} end
for i = 1, c do tab2[i] = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" end
for i = 1, c do tab3[i] = i ^ 2 end
for i = 1, c do tab4[i] = {a = 'b', b = 'c', c = 'd', e = {a = 1, b = 2, c = "ok"}} end
for i = 1, c do
tab5[i] = {math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000), math.random(1, 1000)}
end
for i = 1, c do tab6[tostring(i)] = tostring(i) end
local tabs = {
tab,
tab2,
tab3,
tab4,
tab5,
tab6
}
return {
ParameterGenerator = function()
return BufferEncoder, MessagePack, HTTP, tabs
end,
Functions = {
["BufferEncoder"] = function(Profiler, BufferEncoder, MessagePack, HTTP, t)
local r = BufferEncoder.write(t, nil, nil, nil, nil, Profiler)
Profiler.Begin("Read")
BufferEncoder.read(r)
Profiler.End()
end,
["JSONEncode & Decode"] = function(Profiler, BufferEncoder, MessagePack, HTTP, t)
Profiler.Begin("Write")
local r = HTTP:JSONEncode(t)
Profiler.End()
Profiler.Begin("Read")
HTTP:JSONDecode(r)
Profiler.End()
end,
["MessagePack"] = function(Profiler, BufferEncoder, MessagePack, HTTP, t)
Profiler.Begin("Write")
local r = MessagePack.encode(t)
Profiler.End()
Profiler.Begin("Read")
MessagePack.decode(r)
Profiler.End()
end,
},
}
Installation
You can install it on Wally or from the github page, Links are at the top of the post.
Basic Example
API can be found in the github repository
local BufferEncoder = require(path_to_encoder)
local parts = {}
local Properties = {'Name', 'ClassName', 'CFrame', 'Color', 'Transparency', 'Material', 'Anchored'}
for _, part in workspace:GetDescendants() do
if part:IsA("BasePart") then
local t = {}
for _, prop in Properties do t[prop] = part[prop] end
table.insert(parts, t)
end
end
local buff = BufferEncoder.write(parts)
-- you could then save that buffer or send it in a remote or whatever you want
Credit
-
@xKiiyoko for the icon!
- @DevSynaptix for the CFrame quaternion encoding code
