Is there a better way to use operations (mainly bit32) on buffers?

[Ghosteined (Actually_Someone)]

I need to use bitwise operations (like AND or other) and bit shifts on a large buffer I have but I couldn’t find anything ressembling that in the buffer library, so I made a little function for this (another one for bit shift that uses buffer.copy but it’s pretty much the same code).

The problem is that my game calls this function many many times (>5000 per thread) every frame (and I cannot remove or optimize the calls made) so I’d like to know if there is a better/faster alternative to doing that (especially since it uses loops).

local function processBuffer(b: buffer, bitOffset: number, op: (number) -> (number))
	-- The last iteration may not be in a batch of 32
	local bLen = buffer.len(b)
	local iterations, lastIterationLength = bLen, bLen % 4

	local bOutput = buffer.create(bLen)
	for i=4, iterations, 4 do
		local position = bitOffset + (i - 4) * 8

		buffer.writeu32(bOutput, position, op(
			buffer.readu32(b, position)
		))
	end

	-- Handle the last iteration
	local position = bitOffset + bLen - lastIterationLength
	local size = lastIterationLength

	buffer.writebits(bOutput, position, size, op(
		buffer.readbits(b, position, size)
	))

	-- Returning
	return bOutput
end

[Ghosteined (Actually_Someone)]

Just to precise I don’t totally understand buffers and this function might not work properly (I can’t understand if offset is in bytes or bits)

[pullman45]

@Ghosteined

(I can’t understand if offset is in bytes or bits)

The offset is in bytes:

You can test it for yourself. If you want fast bitwise operations then yes you’d want to use the bit32 API.

[Ghosteined (Actually_Someone)]

Thank you for this, I have now made these utility functions (which are unfortunately too slow for my use case but I’ll accept it).

-- Utility functions
local function processBuffers(b1: buffer, b2: buffer, op: (number, number) -> (number))
	-- Variables
	local bLen = math.min(buffer.len(b1), buffer.len(b2))
	local b3 = buffer.create(bLen)
	
	local lastByte = bLen - 4
	
	-- Looping through the buffer and applying the operation
	for i = 0, lastByte, 4 do
		local num1, num2 = buffer.readu32(b1, i), buffer.readu32(b2, i)
		
		buffer.writeu32(b3, i, op(num1, num2))
	end
	
	-- Writing the last iteration as the last iteration may not be in a batch of 4 bytes
	local num1, num2 = buffer.readu32(b1, lastByte), buffer.readu32(b2, lastByte)

	buffer.writeu32(b3, lastByte, op(num1, num2))
	
	-- Returning
	return b3
end

local function getShiftedBuffer(b: buffer, shift: number)
	-- Variables
	local bLen = buffer.len(b)
	local bOutput = buffer.create(bLen)
	
	local bitLen = bLen * 8
	local firstBit = math.max(shift, 0)
	local lastBit = math.min(bitLen, bitLen + shift) - 32
	
	-- Applying the shift to the new buffer
	for i = firstBit, lastBit, 4 do
		local num = buffer.readbits(b, i - shift, 32)
		
		buffer.writebits(bOutput, i, 32, num)
	end
	
	-- Copying the last iteration as the last iteration may not be in a batch of 4 bytes
	local num = buffer.readbits(b, lastBit - shift, 32)
	
	buffer.writebits(bOutput, lastBit, 32, num)
	
	-- Returning
	return bOutput
end

local function print2dBuffer(b: buffer, width: number)
	local bLen = buffer.len(b) * 8
	local lines = math.ceil(bLen / width) - 1
	local lastLineWidth = bLen % width
	
	local str = "\n"
	for y=0, lines do
		if y == lines then
			width = lastLineWidth + 8
		end
		
		for x=0, width - 1 do
			str ..= tostring(buffer.readbits(b, y * width + x, 1))
		end
		
		str ..= "\n"
	end
	
	print(str)
end

[pullman45]

Seems like one of your main bottlenecks is string concatenation (for your print2D function at least). You can confirm that this is true using the microprofiler. Basically, every time you concatenate two strings together, the Lua VM constructs an entirely new string which eats up performance.

Using table.concat should speed up your code a decent amount.

[Ghosteined (Actually_Someone)]

Thanks, however the print2D function was not the one slowing down my game as it was only for visualization during testing, but I didn’t know about the concatenation issue as I’m pretty new to luau optimization.