Compressing CFrames for Client over RemoteEvent

I’m working on an Enemy tower defense like game themed around a game I used to play a lot. I’ve recently resorted to doing all of the movement logic on the server as before, but instead now, send the CFrames to the clients to position and rotate the enemies accordingly alongside handling the animations and other visuals. Things to consider: I do NOT use Humanoids or methods of a Humanoid such as MoveTo().

Things I’ve collected from this:

  • Framerate on the client matters. (How frequent they listen to the RemoteEvents) Which appears to increase the data being received on the client. I could limit this to be as if they had 60 frames always if nothing else is an option.

I need suggestions as to how to send compressed or as little data as possible while still sending the client the necessary data to position each Enemy.
At 100 FPS the Network Receive reaches as high as 400kb/s. The goal would be to keep it no higher than 100kb/s.

I am not sure I’ll be able to settle for having the client move and predict where the Enemy will be on the server (Relying less on a RemoteEvent). This also becomes a bit more difficult and tricky without using traditional humanoids with the MoveTo() method.

1 Like

So you are currently sending the CFrame? Couldn’t you just send one number for each enemy? The number would tell how far the enemy is on the route.

And you probably won’t need a 64-bit number for one enemy. If, for example, the entire path is less than 1000 studs long and 0.05 stud accuracy is acceptable, your number representation only needs to be able to represent 1000 / 0.05 + 1 = 20001 different values. I added +1 because floor((number in [0, 1]) * integer) can be between 0 and integer which is integer + 1 different values).

The number of values that an n-bit binary number can represent is 2^n. 2^16 = 65536 so a 16-bit number is sufficient if being able to represent 20001 values is enough. You could send a buffer that contains a 16-bit unsigned integer for each enemy. Buffer size is defined in bytes (one byte = 8 bits).

Here’s some code for converting normalized positions to a buffer and a buffer back to normalized positions. I don’t have experience with buffers so I’m not sure if I wrote this correctly.

-- normalizedPositions should contain numbers in the interval [0, 1].
local function convertToBuffer(normalizedPositions: {number}): buffer
	local posBuffer: buffer = buffer.create(2 * #normalizedPositions)
	for i: number, normalizedPos: number in normalizedPositions do
		local integer: number = math.floor(normalizedPos * 20_000)
		local offsetInBuffer: number = (i - 1) * 2
		buffer.writeu16(posBuffer, offsetInBuffer, integer)
	end
	return posBuffer
end

local function convertToArrayOfNormalizedPositions(posBuffer: buffer): {number}
	local numberOfPositions: number = buffer.len(posBuffer) / 2
	local normalizedPositions: {number} = table.create(numberOfPositions)
	for i: number = 1, numberOfPositions do
		local offsetInBuffer: number = (i - 1) * 2
		local integer: number = buffer.readu16(posBuffer, offsetInBuffer)
		local normalizedPos: number = integer / 20_000
		normalizedPositions[i] = normalizedPos
	end
	return normalizedPositions
end
1 Like

I’ve heard that you should do it every 10th of a heartbeat

Perhaps? It any ideas how that’d function? Using individual numbers to determine CFrames along the path? (Also that’d imply they’d need the code on the client too?)

I haven’t tried that yet but I’m sure if I do, they’re going to look choppy.

You send them like that and tween them (or so I’ve heard). And it’s not every 10th of a heartbeat. It’s every 10 heartbeats (one would be 10 vs 0.1 heartbeats mb lol) There’s a devlog by Kdude on YouTube you can watch that gives you good information on this (part 1)

How are you currently calculating the CFrames? Do you have a list of path segment end points?

There are waypoints they travel to, but I Lerp them on the server through those CFrames. So the only way I could do the same thing on the client is to give the client the same code the server is using.

Here’s some untested example code that includes functions for calculating a new normalized position and for calculating the CFrame from a normalized position.

local pathPoints: {Vector3} = --?

local pathLength: number
for i: number = 1, #pathPoints - 1 do
	pathLength += (pathPoints[i + 1] - pathPoints[i]).Magnitude
end

local startNormalizedPositionsForSegments: {number}
local lengthSumSoFar: number = 0
for i: number = 1, #pathPoints - 1 do
	startNormalizedPositionsForSegments[i] = lengthSumSoFar / pathLength
	lengthSumSoFar += (pathPoints[i + 1] - pathPoints[i]).Magnitude
end

local function validateNormalizedPos(normalizedPos: number): ()
	if normalizedPos < 0 then
		error(`normalizedPos  too small ({normalizedPos})`)
	elseif normalizesPos > 1 then
		error(`normalizedPos  too big ({normalizedPos})`)
	end
end

local function getCFrameFromNormalizedPos(normalizedPos: number, yOffset: number): CFrame
	validateNormalizedPos(normalizedPos)
	local segmentIndex: number = #pathPoints - 1
	while startNormalizedPositionsForSegments[segmentIndex] > normalizedPos do
		segmentIndex -= 1
	end
	local segmentStartPoint: Vector3 = pathPoints[segmentIndex]
	local segmentEndPoint: Vector3 = pathPoints[segmentIndex + 1]
	local segmentStartNormalizedPos: number = startNormalizedPositionsForSegments[segmentIndex]
	local segmentEndNormalizedPos: number = if segmentIndex == #pathPoints - 1 then 1 else startNormalizedPositionsForSegments[segmentIndex + 1]
	local interpolationAlpha: number = (normalizedPos - segmentStartNormalizedPos) / (segmentEndNormalizedPos - segmentStartNormalizedPos)

	local position: Vector3 = segmentStartPoint:Lerp(segmentEndPoint, interpolationAlpha)
	position += yOffset * Vector3.yAxis

	local segmentVector: Vector3 = segmentEndPoint - segmentStartPoint
	local horizontalSegmentVector: Vector3 = Vector3.new(segmentVector.X, 0, segmentVector.Z)
	local lookVector: Vector3 = horizontalSegmentVector.Unit
	local rightVector: Vector3 = lookVector:Cross(Vector3.yAxis).Unit

	return CFrame.fromMatrix(position, rightVector, Vector3.yAxis)
end

-- if the speed of a single enemy doesn't change during its lifetime
-- then you don't need to use previous normalized positions.
-- You can just calculate the normalized position at any moment by multiplying
-- the normalized speed of the enemy by the time the enemy has existed.
local function getNewNormalizedPos(deltaTime: number, previousNormalizedPos: number, speed: number): number
	validateNormalizedPos(previousNormalizedPos)	
	local normalizedSpeed: number = speed / pathLength
	local newNormalizedPos: number = previousNormalizedPos + normalizedSpeed * deltaTime
	return math.min(newNormalizedPos, 1)
end

If you need to calculate the CFrames on both the server and the client, you can put the code in a module script in ReplicatedStorage (in which case the functions shouldn’t be in local variables but should instead be included in the table returned by the module).

So is this something I can first attempt to use in conjunction with buffers?

For the record I’m completely unfamiliar with the buffer library. I just had a friend give me a bit of a rundown and helped me out a bit. I manage to cut the 100-400kb/s down to 60-70 max. I’m not sure if we can go any lower though. maybe?

Yes, the function convertToBuffer I wrote for creating the buffer expects an array of normalized positions (numbers in the interval [0, 1]). You’d need to store the normalized positions of the enemies in an array on the server, give that array to the buffer creation function, send the buffer to the client, turn it back into an array on the client using the function convertToArrayOfNormalizedPositions and use those normalized positions for calculating the CFrames using getCFrameFromNormalizedPos.

I don’t know how your current client code figures out which enemy each CFrame belongs to but I’m pretty sure that you can figure out which enemy a spesific normalized position in the array belongs to in the same way.

Also, if the code doesn’t work as expected, you could try it with and without buffers (without buffers = sending the normalized positions array to the client). That way you could see whether the problem is in the buffer conversion or in other code.

I follow but I also do not. Are the array of normalized positions that of which make up the whole path?

No, the array of normalized positions is not supposed to contain 20001 numbers or positions if that’s what you mean. It’s supposed to contain as many numbers as there are enemies at the moment, and these are numbers in the interval [0, 1].

With normalized position I mean a number in the interval [0, 1] telling where on the path a spesific enemy is. So for example, if an enemy has just spawned, it’s normalized position is 0, if it’s in the middle of the path, it’s 0.5 and if it’s at the end (about to be removed), then the normalized position is 1. With path I mean the entire path consisting of all the segments. You’ll need to store a number like this for every enemy.

The number 20001 is just how many different values are possible for a position that is calculated from a replicated 16-bit integer using the code I wrote. The code that converts the replicated number to a CFrame will give a CFrame whose position is one of the 20001 possible positions. You are not meant to store all these possible positions anywhere.

Also, the number 20001 is unnecessarily small (it was just an example). You could more than tripple it while still using a 16-bit integer because with 16 bits you can represent 65536 different values. You’d just have to change the number 20000 in my code to 65535 (or something smaller).

If there are more than one possible path then you’ll need another number for each enemy telling which path that enemy is on. With a different path, I mean a different route, not a different segment in the same sequence of segments.

Judging by the picture, the only real data you need is the position and the Y axis of the rotation. If you use buffers, you can represent the angle as a 16 bit number.

Alright, so I’m testing what you sent.


This is great but without reading back, are you suggesting I send these values to the client instead of a CFrame?

With the help of a friend, I believe that is what I currently do. We broke down on the unnecessary values of the CFrames.

Yes, the numbers printed to the output seem to be exactly what I mean with normalized position. When you have the normalized position for each enemy stored in an array, you can either

  1. send the array of normalized positions or
  2. first convert the array of normalized positions to a buffer, send the buffer and convert it back to an array on the client.

The purpose of converting to a buffer containing 16-bit integers is just compression.

Also, now that I saw those errors I realised I should have limited the result of getNewNormalizedPos such that it can’t go over 1.

Do you think storing these numbers in a buffer would be more manageable than CFrames and perhaps would be much less of a payload to send to the client every heartbeat? (I can show you my buffering module if necessary that I use to serialize CFrames)

That’s something I can easily fix don’t worry lol

Now I’d also have to implement my movement logic on the server into this if I want it to function specially, right?