BufferConverter2 | Blazingly Fast Schema-Based Buffer Serialization

0.08s to serialize 10k Instances TOTAL! :exploding_head:

Minor Update:

  • Fixed some incorrect types for the SchemaData

  • Made JSON serializing faster by using a lookup table and concat instead of interpolated strings

Update:

  • Fixed CFrames not working correctly for size types other than f32 and f16

  • Fixed unions not interpreting structs correctly

2 Likes

you could also use:
–!native and --!optimize 2

hey am I be able to use a function that calculates which number primitive is the best based on the value?

local Floor = math.floor
local function ChoosePrimitive(Value: number): "u8" | "i8" | "u16" | "i16" | "u32" | "i32" | "f16" | "f32" | "f64"
	local IsInteger = Floor(Value) == Value
	if IsInteger then
		if Value >= 0 and Value <= 255 then
			return "u8"
		elseif Value >= -128 and Value <= 127 then
			return "i8"
		elseif Value >= 0 and Value <= 65535 then
			return "u16"
		elseif Value >= 32768 and Value <= 32767 then
			return "i16"
		elseif Value >= 0 and Value <= 4294967295 then
			return "u32"
		elseif Value >= -2147483648 and Value <= 2147483647 then
			return "i32"
		end
		
		return "f64"
	else
		if Value >= -65504 and Value <= 65504 then
			return "f16"
		elseif Value >= -16777216 and Value <= 16777216 then
			return "f32"
		end
		
		return "f64"
	end
end

i have a table in which the values arent always the same primitive and i gotta save it to data stores, so i would like to save as much space as possible

1 Like

I am unsure if you are able to because this module requires the schema to be consistent for serializing and deserializing, however if you can figure that out then yeah this’ll work

However, I am currently working on a way to serialize schemas so you can store them alongside the data itself (this hopefully should still be quite memory efficient as each schema isn’t too large, however I would still recommend just using 1 schema for every player’s data in the datastore)

1 Like