CBuffer - Bit-Level Buffer Control

Resources Community Resources
#1

As the title suggests, this module provides an object called a CBuffer which exposes API allowing for bit-level control of a buffer’s data. You can read or write any number of bits you like (between 1 and 32 inclusive) without the boundaries of bytes getting in your way. This object is especially useful for network data compression algorithms that incorporate bit-packing.

The CBuffer has a cursor position and reading or writing to the CBuffer will advance the cursor forward unless a start position is manually supplied to those methods. Trying to read or write things out of bounds of the buffer will throw an error.

The read and write operations are limited to a maximum of 32 bit numbers, as roblox doesn’t provide libraries to perform bitwise operations on numbers larger than that. A hacky workaround is possible with intermediate buffers, but I imagine not many people will be needing to read or write something like a 45-bit number.

Constructors:
CBuffer.new(n : number) -> CBuffer
creates a new empty CBuffer that is n bytes long

CBuffer.fromBuffer(b : buffer) -> CBuffer
Creates a new CBuffer that uses b as its internal buffer.

Properties:
cursor : number - The cursor position in bits. A position of zero points to the first bit in the buffer.
rawBuffer : buffer - the actual buffer object being written to.
sizeInBits : number - the number of bits in the buffer. Is always a multiple of 8.

Methods:
writeUnsignedBits(data : number, n : number, position : number?) -> ()
Copies the first n bits from data into a space in the buffer beginning at position. If no position is provided, it begins at the cursor position and advances the cursor forward n bits. data is expected to be an unsigned integer.

readUnsignedBits(n : number, position : number?) -> number
Reads the next n bits beginning at position and returns them as an unsigned integer. If no position is provided, it begins at the cursor position and advances the cursor forward n bits.

setCursor(cursorPos : number) -> ()
Places the cursor at the specified bit index in the buffer’s memory. The cursor may range from 0 to CBuffer.sizeInBits - 1, i.e. a cursor position of zero points to the first bit in the buffer.

The lua file can be downloaded here:
cbuffer.lua (3.8 KB)

8 Likes
#2

Thanks, this might come handy.

#3

This is really useful! Is there any chance you could add 64 bit float support though? Userids require more than 32 bits to represent.

#4

Just give players an ID when they join and map userids onto that. Far smarter if networking is enough of a concern to warrant this in the first place.

You’d only need 7 bits for a 100 players.

#5

@thebigkannye is right, the main use case of the CBuffer is for reducing network usage by passing data over in buffers instead of raw values. It’s more efficient to create a session ID for players that has a much smaller range, where the IDs of players who have left can be reused.

If you really wanted to store a 64-bit in a CBuffer though, you could just use roblox’s built in u64 write function on a normal buffer, then pass that buffer into CBuffer.fromBuffer(), set the cursor position to 64 (assuming the userID is written at position 0), and write additional data from there.

#6

My use case is storing it in a datastore (This is the reason that i need to use the buffer instance with a bit level support to begin with, to compactly serialize huge amounts of data). I already map the userids, but i still need to store them at some step in the process.

I’ll use the hacky method pseudokev mentioned, a built in method would just be best for everyone.

#7

Ahh, I assume your plan is to serialize the binary into something else then right? You could split the userid into the smallest factor pair or use some other data compression technique.

And yeah, built in methods are always great. But that’s really up to Roblox as it’d be far preferable to have engine level bitwise support for 64 bits rather than implement it in luau.

#8

Can you give example on how to use this?