What is the best way to implement an equality check on floating point inaccuracies?

Help and Feedback Scripting Support
#1

So I’ve been messing around with Vector2s and noticed that when I execute the following example code

local pos1 = Vector2.new(0,0) 
pos1 = pos1 + Vector2.new(.2,0)
print(pos1.X .. "," .. pos1.Y)

the output is

0.20000000298023,0

I searched the forums and found that there was no built-in fix to this issue, so I was wondering what the agreed-upon solution to these errors was?

My first thought was to round to the nearest tenth or hundredth of a digit. However, the implementation I’m thinking of in my head looks a bit inelegant:

local value = 0.20000000298023 --Pretend this is any number
local decimal = (value*100) - math.floor(value*100)
if decimal >= .5 then
      value = math.ceil(value*100)/100
else
     value = math.floor(value*100)/100
end
--Note that I haven't actually tested this code it's just to illustrate where my mind was heading with this solution.

Is there a better way to handle this?

2 Likes
I can't get this tween to work
#2

The best way is to avoid equality checks on floating points if you can help it. However, if you must, the best way to do it to my knowledge is to get a certain precision on the floats you want to compare, then compare those numbers.

local function compare(float1, float2, precision)
    precision = 10^precision
    return math.floor(float1 * precision) == math.floor(float2 * precision)
end

print(compare(0.20000000298023, 0.20000000298023, 10), "true")
8 Likes
Script won't print out "What" for some reason
#3

Another standard way to do this is to compare the absolute difference against a constant with a very small value.

local EPSILON = 1e-5 -- precision

local function float_equals(n0, n1)
    return math.abs(n1 - n0) < EPSILON
end
8 Likes