Hello guys i made this cool buffer module just wanted to showcase and know if you guys have any features that would be cool to add.
github: GitHub - El-Luckyon/Modules: My module collection
so basically what this module does it lets you define custom data types that can be used to store values in a buffer
there is 2 types of data-types, static and dynamic.
static : data is fixed during compile time
dynamic : data size is cannot be determined until runtime
well seems simply enough doesn’t it? but the biggest issue i faced was typechecking and intellisense compatablity. to achive this i use something called samples so a sample is basically a luau data type
this sample is what ur actual data type represents it can be anything like 1, 2, or " " or vector3.new(0, 0, 0) so when you pass datatype into a bufferObject ur passing in this sample and the bufferobject uses uses a generic type that is equal to the sample of the given data type. well now comes the problem of finding the data type by using a sample. for simple data types like number and strings you can directly store them as map keys but for complex data types like cframe this is not possible so we use hashing to store a hash relating to this sample so the hash of vector3 is something along the lines of X + Y + Z prefixed with a string representing the type so if ur data type has a sample Vector3.new(1, 5, 3) the hash would be “vec9” currently this hashing is done for like a few types only
like cframes vec3 color3s brickcolors etc. so as long as each of ur datatype has a unique sample provided there should be no collisions or anything. well running a hash function everytime we encode and decode is a bit wasteful and since the datatypes of buffer object is fixed, we compile a meta data when the object is created. this meta data is either direct pointer to the hash (if there is only one data type) or table containting key/indices pointing to thier respective hashes (if ur buffer object has multiple data types).
Well the buffer objects are of 3 types which is also determined during the creation of the meta data
STATIC : contains only static datatype(s)
DYNAMIC : contains only dynamic datatypes(s)
COMBINED : contains both static and dynamic datatype(s)
this 3 type system gives the most optimal encoding and decoding speeds since this way no unesscary logic is run during encoding/decoding.
well as for the encoding and decoding functions they are just 2 monolithic 100 line functions
that checks the type of the bufferobject then encodes each of its data types using thier respecitve methods as for dynamic data types we just store uint16 storing it size during the encoding (the size is determined by a method provided by the user) so the decoder can use it to decode it.
here is the syntax/usage of this module
local function readvec3(buff : buffer, startIndex : number, sizeOfValue : number) : Vector3
return Vector3.new(buffer.readf32(buff, startIndex), buffer.readf32(buff, startIndex + 4), buffer.readf32(buff, startIndex + 8))
end
local function writevec3(buff : buffer, startIndex : number, valueToWrite : Vector3, sizeOfValue : number)
buffer.writef32(buff, startIndex, valueToWrite.X)
buffer.writef32(buff, startIndex + 4, valueToWrite.Y)
buffer.writef32(buff, startIndex + 8, valueToWrite.Z)
end
local str = BufferModule.dType("", string.len, buffer.writestring, buffer.readstring)
local f64 = BufferModule.sType(640, 8, buffer.writef64, buffer.readf64)
local u8 = BufferModule.sType(16, 1, buffer.writeu8, buffer.readu8)
local vec3 = BufferModule.sType(Vector3.new(), 12, writevec3, readvec3)
local st = os.clock()
local testBuffer = BufferModule.buff(
{
Name = str,
Description = str,
Age = u8,
Coins = f64,
Gems = f64,
JoinTime = f64,
Position = vec3,
}
)
print("Compiled bufferObject in: ", os.clock() - st)
local data = {
Name = "Luckyezz",
Description = "Very berry cool guy",
Age = 21,
Coins = 2039499112,
Gems = 2949343,
JoinTime = os.clock(),
Position = Vector3.new(99, 69, 99.875),
}
st = os.clock()
local buff = BufferModule.encode(testBuffer, data)
print("Encoding completed in: ", os.clock() - st)
st = os.clock()
local json = HttpService:JSONEncode(data)
print("Json encoding completed in: ", os.clock() - st)
print("Buffer output: ", buffer.tostring(buff))
print("Json output: ", json)
print("Size in Buffer:", buffer.len(buff))
print("Size as Json: ", string.len(json))
st = os.clock()
local decoded = BufferModule.decode(testBuffer, buff)
print("Decoding completed in: ", os.clock() - st)
print("Decoded output: ", decoded)
decoded.
st = os.clock()
local jsonDecoded = HttpService:JSONDecode(json)
print("Json decoding completed in: ", os.clock() - st)
print("Json decoded output: ", jsonDecoded)
very simple to use nothing too fancy for most of the data types you can just pass in the buffer.read/write method
as you can see i benchmarked somewhat
its faster than ur normal json encoding and decoding by almost 10x
its also much less smaller in size since you don’t store the keys inside each buffer like json
output:
as you can see json can’t compile stuff like vector3s while this module can
also the intellisense i was talking abt the return of the decode function matches the type of the encode function. also the typechecking is done during u passing the params into the encode function
so basically this module takes like 3 functions from you and when you call another function it uses ur own functions to cookup some wierd looking number.
thank you
(my first post sorry if it was bad or messy or whatever)