Why is 0.7%0.05 giving 0.05 when it should give 0?

Solution

Multiply both numbers by 100, or whatever power of 10 is needed to remove the decimal places.

Solution by @RamJoT

I have some code for a sliding GUI and need to check if the number entered into a text box is a multiple of 0.05 from 0 to 0.9.
I check for this by using modulus: if num % 0.05 == 0 then print(num) end
num being the input in the text box.

When I enter 0.3, 0.6 and 0.7, I am given an output of 0.05.
I also typed in a calculator 0.7%0.05, which still gave me 0.05.

I tried many other numbers such as 0.15, 0.5 etc. which still gave 0.05.

I’d love some help :smiley:

1 Like

a%b is the same as:

a - math.floor(a/b)*b.

0.7 - math.floor(0.7/0.05)*0.05

0.7/0.05 = 14, and floor rounds down but 0.7/0.05 results in an integer anyways.

0.7 - 14*0.05 = 0

Yes, the result should be 0…

1 Like

Perhaps a work around might be to multiply both numbers by 100, to remove the decimal places, and then try the modulus.

1 Like

I’ll try that. Thanks for replying :smiley:

It worked! Thanks again :smiley:

You should post your solution so that future people who may have the same problem know how to fix it, and mark @RamJoT’s post as the solution

1 Like

It isn’t actually giving 0.05. This is a floating point imprecision problem.

local n = 0.7 % 0.05
print(("%.20f"):format(n))
--> 0.04999999999999993339
2 Likes