Compressing CFrames, reducing Bandwidth/Network Usage for high remote usage

I am currently rehauling an old framework, and an issue I used to run in was: very high bandwidth/network usage. I knew this was from the remotes I’d send very often (specific character cframe replication, effects, whatnot), and in general, the remotes in my games passed a lot of data.

I optimized what I could of course, but I have seen bigger games with smarter developers compress their data and reduce this network usage which when measured had a big reduction on network usage.

Sadly in my framework all this data is 100% necessary, and it’s just something im afraid will happen again once I rehaul more aspects. ( I have intentionally been ignoring networking/remotes while searching for an answer. )

40-50 players would be golden, but probably unplayable with current data passage.
(havent tested that though)

Anyone dealt with this? If so, how have you fixed/improved this?

4 Likes

I mean there are a few ways of compressing a CFrame if you know for sure that the CFrames are causing most of the bandwidth to be eaten up.

First and probably the easiest way would be to serialize the cframe and use a compression algorithm to compress it. You would just do something like local cf = table.concat(cf:GetComponents(), ','), and then pass this string to a lossless compression algorithm, then pass it to the server and have the server decompress.

The issue with doing this though would be that it could be slow to compress and decompress this every time a remote is fired. Also, some lightweight compression algorithms (such as lz4) can sometimes cause the amount of data to be sent to increase instead of decrease.

Another way could be to send a position and orientation instead of a whole CFrame since orientations only have three axes of rotation, thus 6 numbers (3 for pos, 3 for rot) instead of 12.

Another way could be to send only two of the rotational vectors since we know for sure that a CFrame’s rotational components must be orthogonal. The third rotation column is just column1:Cross(column2).

Another tip to take into account is that we can round each component to, say, the thousandth instead of sending a raw decimal.

We can also get rid of zeroes if the number is between -1 and 1 (exclusive).

This is what I was able to come up with.

local function round(...: number)
	local packed = { ... }
	
	for index, number in packed do
		local stringified = tostring(math.round(number * 1000) / 1000)

		if stringified:sub(1, 1) == '0' then
			stringified = stringified:sub(2)
		elseif stringified:sub(1, 2) == '-0' then
			stringified = stringified:sub(1,1) .. stringified:sub(3)
		end

		packed[index] = stringified
	end

	return if #packed > 1 then packed else unpack(packed)
end

local function toNumber(str: string)
	if str == '' or str == '-' then
		return 0
	end

	return tonumber(str)
end

local function makeVector3s(...: string)
	local vectors = {}

	local packed = { ... }

	for i = 1, #packed, 3 do
		local next1, next2, next3 = packed[i], packed[i + 1], packed[i + 2]

		table.insert(vectors, vector3(toNumber(next1), toNumber(next2), toNumber(next3)))
	end

	return unpack(vectors)
end

local function getCompressedCFrame(cf: CFrame) -- serializes and compresses a CFrame
	local pos = cf.Position
	local pX, pY, pZ = pos.X, pos.Y, pos.Z

	local xVec = cf.XVector
	local xX, xY, xZ = xVec.X, xVec.Y, xVec.Z

	local zVec = cf.LookVector
	local zX, zY, zZ = zVec.X, zVec.Y, zVec.Z

	return table.concat(round(pX, pY, pZ, xX, xY, xZ, zX, zY, zZ), ',') 
end

local function fromCompressedCFrame(str: string) -- takes a serialized, compressed CFrame and transforms it back to a usable CFrame
	local splitString = str:split(',')

	local pos, xVector, zVector = makeVector3s(unpack(str:split(',')))
	xVector = xVector.Unit
	zVector = zVector.Unit

	local yVector = xVector:Cross(zVector).Unit

	return CFrame.fromMatrix(
		pos,
		xVector,
		yVector,
		-zVector
	)
end

So all in all an identity CFrame would be serialized and compressed from 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1 (34 characters) to ,,,1,,,,,-1 (11 characters)

For a closer to real world example, I put a randomly oriented part’s CFrame into what I created and it gave me -4.129,.5,37.567,.707,.354,-.612,-.5,-.362,-.787 which is 48 characters .

For reference, the actual CFrame is -4.12884426, 0.5, 37.5671997, 0.707106829, -0.50000006, 0.49999997, 0.353553295, 0.862372518, 0.362372637, -0.612372458, -0.0794593096, 0.786566198 which is 147 chars. So each time it gives us about 2/3 of a reduction.


All that being said, though, compressing a CFrame so much is not very practical in my opinion, and I believe Roblox does do compression to some extent when a remote is fired. Could you maybe give some insight on how specifically you are firing each remote and what data is being sent, and how often?

3 Likes

Yes, sure, I just looked into my old framework file with the microprofiler and the results are after rapidly firing a gun, that costs me around 40-50kbs up to a 150kbs if i fire even more rapidly (or probably if its multiple people shooting pistols) on sent bandwidth

I’ll tell you the biggest ones:

every 1/30 of a second every character sends 2-3 client-calculated cframes to server, which passes it to other clients, where the clients then receive and interpolate it themselves

then, a remote for each weapon action (hit, fired, reload, then the effect remotes which pass around comestic effect data to other clients) - essentially different tables are being passed around with Color3, number, bool and string values.

the firing remotes include an origin cframe, either the full gun config or a snippet of the gun config(module), the tool - usually theres maybe one or two more arguments for anti-cheat reasons, but i think nowadays with byfron i could get rid of some checks.

It’s crazy bandwidth just passing around data/cframes/tables or effect data. The players end up lagging themselves, even while the ping is fine.

Now, while rehauling ive been thinking about server and client model for my rehauled framework but usually with weapon systems its all the same. It’s been a while since I made the framework and my knowledge has improved, but even with a newer attempt at fixing bandwidth the amount of data is still there and hasnt gone away due to how my framework is designed.

1 Like

CFrames are 20 bytes (source), so either you are sending an absolutely ridiculous amount or something else is causing the problem.

You should consider serializing and compressing large tables that have many repeating elements, as compression works best for strings with repeating patterns, or reevaluate whether what you’re sending is absolutely necessary.

Packet Profiler will definitely help you out here.

4 Likes