To make a long story short, every time I perform an operation on a CFrame coordinate (ie. 0 - 0.333) to a coordinate in a CFrame, it ends up returning with many more decimal points (ie. -0.33300000131130215 rather than -0.333 as I’d expect)
Is this some kind of floating point issue I won’t be able to get around? It’s really irritating since I’m doing operations with these CFrames and comparing them to predetermined values. For example, I might multiply the CFrame’s x-coord of 0.333 by three and expect 0.999, but in reality I’ll get something like 0.9990000000000003234. I can make a few workarounds for this, but I’d rather figure out if there’s something I can do to directly solve the problem first.
If you’d like to look over my code, let me know. Thanks!
That’s just the nature of floating point errors. You could round the number to the precision you need (in the case of wanting to round to the thousandth, math.round(n / 0.001) * 0.001), but I’m also inclined to ask what you’re doing that these floating point errors are problematic
I’ve already tried rounding the errors post-operation but for whatever reason they just revert. I do this to round the numbers.
local function ReturnPrecision(value, decimalPoints)
return value - value%10^(-decimalPoints)
end
The reason why I expect high precision is because I’m setting x & y boundaries from a vertex point based off an increment in a grid system I’m designing. For example, my vertex point might be at (1, 1, 0) and I’d like to increment the x & y values down by 0.333 in a loop until they both hit my boundary values of x = -1, y = -1. This is modular code so it’s being used for many different, I need to be general and can’t set these values directly.
This is my code by the way, not the full thing but this is where all the operations are. As you can see I cache the CFrame of the last point I set to perform operations on, so the error propagates no matter what I do. I can workaround this a few ways (ie. making psuedo-points which reflect expected values) but it’s just kinda ridiculous.
local pointSetCalculation = function(vertice, limb)
local pointLocations = {vertice}
local numberOfPointToSet = torsoRelativity[limb][4]
local xIncrement = ReturnPrecision(torsoRelativity[limb][2]/(math.sqrt(numberOfPointToSet) - 1), 3)
local yIncrement = ReturnPrecision(torsoRelativity[limb][3]/(math.sqrt(numberOfPointToSet) - 1), 3)
local xBoundary = ReturnPrecision(vertice.X - (math.sqrt(numberOfPointToSet) - 1) * xIncrement, 3)
local yBoundary = ReturnPrecision(vertice.Y - (math.sqrt(numberOfPointToSet) - 1) * yIncrement, 3)
print(xIncrement, yIncrement, xBoundary, yBoundary)
local lastPointSet = vertice
local totalPointsSet = 0
while numberOfPointToSet > totalPointsSet do
if lastPointSet.X > xBoundary then
lastPointSet = CFrame.new(
ReturnPrecision(lastPointSet.X - xIncrement, 3),
lastPointSet.Y,
lastPointSet.Z
)
elseif lastPointSet.X == xBoundary then
warn("SHIFTING Y TO " .. lastPointSet.Y - yIncrement)
lastPointSet = CFrame.new(
vertice.X,
ReturnPrecision(lastPointSet.Y - yIncrement, 3),
lastPointSet.Z
)
elseif (lastPointSet.X == xBoundary and lastPointSet.Y == yBoundary) then
print("Process Complete, set ".. totalPointsSet.." points on ".. limb)
end
totalPointsSet += 1
table.insert(pointLocations, lastPointSet)
end
return pointLocations
end
I just ran another test and it doesn’t matter what I do, every single time I assign a value to a CFrame it will just give me those nasty floating point errors.
Probably just going to workaround it by adjusting the boundary checks with math.clamp like so
elseif math.clamp(lastPointSet.X, xBoundary - 0.001, xBoundary + 0.001) then
In general I think where your function goes wrong is that it’s doing subtraction on value itself which is already affected by floating point imprecision (if it’s one bit off for example). I would guess that subtracting a number that’s so low would just lead to even more potential floating point issues, but frankly I don’t have enough low level programming knowledge to explain why this happens so it’s just a shot in the dark.
However rounding should always give you an exact integer representation so doing a calculation like dividing by 0.001, rounding the result (which gives you that exact representation), and then multiplying it again by 0.001 (or doing 10 ^ decimalPoints, multiplying by that number, rounding, then dividing by that number) shouldn’t give you any issues, so I’d opt to switching out your rounding function for one that does some form of rounding personally.
It’s a good idea to program leaving a small margin for error for these floating point errors
