How to round without floating point precision errors?

I need to store the data of a part in a compact way. Specifically the position, orientation and size.
And since this data is quite long, I try to round it to the nearest hundredth of a stud.

My code so far:

function round(vector)
	return Vector3.new(math.floor(vector.X * 100) / 100, math.floor(vector.Y * 100) / 100, math.floor(vector.Z * 100) / 100)
end


function serialize(inst: Part, root: Part)
	local position = round(root.Position - inst.Position)
	local orientation = round(inst.Orientation)
	local size = round(inst.Size)
	
	local output = "{position.X},{position.Y},{position.Z},{orientation.X},{orientation.Y},{orientation.Z},{size.X},{size.Y},{size.Z}" -- I actually use string interpolation here but I replaced it with " so that I could have the code
	return output
end

And the code prints something like this:
-7.760000228881836,-3,4.699999809265137,0,0,0,3.569999933242798,0.9599999785423279,4.579999923706055

After some researching, I realized it doesn’t round perfectly due to floating point precision issues. So how would I fix this, if there even is a fix?

Instead of rounding to the nearest hundredth of a stud, why don’t you just store the data as integers (original data multiplied by 100 and rounded to an integer). Then, when you fetch the stored data to use it again, you just divide it by 100. Also, why are you multiplying by 100.5 instead of by 100 in the code you posted?

local function convertToIntegerVector(vector)
	return Vector3.new(math.floor(vector.X * 100), math.floor(vector.Y * 100), math.floor(vector.Z * 100))
end

local function converToNearlyOriginalVector(integerVector)
	return integerVector / 100
end

Edit:
Also, if you know the minimum and maximum values of the numbers in the vectors, you can further compress the number. Calculate the number of bits needed to represent the needed number of integers.

local numberOfBitsNeeded = math.ceil(math.log(maxInteger - minInteger + 1, 2))

Divide that by eight and round up to get the number of bytes needed (byte is a sequence of 8 bits).

local numberOfBytesNeeded = math.ceil(numberOfBitsNeeded  / 8)

Then convert the number to a string containing the correct bit sequence. The number of string characters needed is the number of bytes. I don’t know how exactly the conversions should be done, though (finding the correct characters for a number and finding the number for a character sequence). You can probably find information about that elsewhere, though.

The 100.5 thing was for testing, sorry.
But yes this worked, and I have no idea why, but thanks lol.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.