BufferConverter2 | Blazingly Fast Schema-Based Buffer Serialization

Resources Community Resources
, , ,
#62

Package Version 110

  • Increased the amount of special CFrames to 2177, consequently the overhead for special-included CFrames has been increased to 2 bytes.
#63

Alright thanks, now when comparing the size between fully serializing and referencing, is it in bytes? and if so, how many seconds would it take to serialize, between the two?

I’m gonna play around with this, and see if i can implement it into my plugin.

1 Like
#64

Its in bytes, and references are faster than full instances.

1 Like
#65

Hello, I just remembered that I was actually working on a big update for this module, however I am unable to release it due to my laptop being broken. Here are the things you can expect:

.schematize(str: string) constructor for schemas

  • This is something that ive been working on to enable schema caching and storage/transmission of schemas, it uses the BufferConverter2 Schema Language (BSL) to convert the string into a specialized SchemaData. Heres example usage: 
    local schema = BufferConverter2.schematize [[
	struct(
		Foo string(u8),
		Bar s_array(number(u8), 10),
		Baz struct(
			Bop number(f16)
		)
	)
]]

New schemas for Roblox userdata datatypes

  • Currently there are no SchemaData available for serializing datatypes such as OverlapParams, RaycastParams, Rays, Region3s, Rects, Fonts, etc. These are all available in the new update
1 Like
#66

Package Version 114:

  • Added BufferConverter2.schematize() which has computed values, operators and more, using the BufferConverter2 Schema Language (BSL)!
    Screenshot 2025-03-15 144944
    Screenshot 2025-03-15 1449441919×568 94.6 KB
  • Added SchemaData for OverlapParams, RaycastParams, Region3s, Rays, PathWaypoints and Fonts
#67

If the module gains enough traction maybe I’ll also make a syntax guide :thinking:

#68

Have you fixed cframe issue yet :sob: Also could you update module version on the main post please its still old package and i cant update since you havent published the package it seems

#69

What CFrame issue??? It works fine for me :cry:

#70

When i am a sending data too frequently from multiple clients its like module is overloaded and cframe every now and then turns nil. This also happens with small amounts of data too

#71

I think i am using old version so it maybe a bug in old version but in new one you maybe have infact fixed it

#72

Are you available to update module on this devforum page rn or no

#73

It could just be something with how the f16 read/writer I have works, try doing data.cframe(“f32”, “f32”) for your schemas

Also yeah ill update the download rq holdon

#74

The download link has been updated, try the new one to see if the issue is fixed

1 Like
#75

I would love to see auto schema feature that takes input data and outputs a functional schema for BufferConverter2. This would be especially helpful for already established games looking to integrate this into their games. I came across this, really liked it, and then realized how many schemas I’d have to manually build to make it work in my games.

1 Like
#76

Hi, there is currently the infer schema which is basically any, tho ill probably make a plugin which can give you an automatically generated schema from a piece of sample data

#77

I did make a module a while ago that serves a similar purpose as this one but doesn’t require schemas and works with any table structure.

Sorry for the shameless plug but I haven’t really seen much attention to mine so I decided to link mine here. This one is still great but the trouble of switching over to schemas for an already established codebase is pretty annoying.

1 Like
#78

Hi, this is what the first BufferConverter did,the reason why the second one uses schemas is because not using schemas is wayy slower and can be undesirable for alot of people :confused:

Also it is harder to customize the data to fit your needs, e.g if your number is only 20-50 theres no need to serialize it as an f32

Although I guess the convenience makes the downsides worth it for people, I don’t judge for plugging since i do it too sometimes :V

#79

Mine is optimized to be as fast as possible for as many use cases as possible, and its performance is very close to httpservice’s JSONEncode and JSONDecode, which are very fast already.

I have some benchmarks for it that i sent in the post for bufferencoder, but I can make more.

1 Like
#80

Though yeah, customizing how your data is saved is impossible without forking if its done automatic for you, but most of the time you probably don’t need to do that and solutions like your module exist if the need exists

#81

I mean fair enough that’s pretty fast
image

Notice how bufferconverter1 timed out :skull_and_crossbones:
(note this test was done with a rewrite of bufferconverter2)