works correctly up until the end, wondering what the reason behind this is and what the best thing to do to fix it would be.
0.2 - 0.1 = 0.9999… ?
when h starts as 3, has a slightly different 0.2 - 0.1 (ends with 8 instead of 9)
Could possibly a throttling issue, try adding a small timer after;
while h > 0 do
This could just be a floating point error. If you’re not sure what that is, this Computerphile video with Tom Scott explains it incredibly well.
This is a floating point error indeed. Happened to me too a while back too!
A way you could fix it:
for i = 20, 0, -1 do
print(i/10)
end
Try doing something like this…
local h = 2
for i = h,0,-0.1 do
print(i)
end
That surely won’t work. It is the same thing as @Patodactyl used.
Not really he used a while loop?
Doesn’t make a difference. Still the result will be affected by floating points.
why does floating point seem to only be an issue while in a loop? print(0.2 - 0.1) should be the same math, but also seems to solve as intended
To be honest I am not so sure, but my best guess would be that when the code is compiled or interpreted, it firstly gets rounded in for loops through floating points because that is faster. But during subtraction/division/multiplication etc… that doesn’t happen, this is just what I think that happens.
This happens because the format lua numbers are stored in (doubles) have a base 2 exponent. To represent numbers smaller than 1, the 53 bit significant divided by a power of two (or in other words multiplied by a negative power of two) must equal the desired number. Before you continue, trying starting with the smallest significant value (1) and dividing by two to reach 0.1. You wont get it exact.
When your script is first loaded, it finds the double precision floating point representations of each number. Since 0.1 can’t be represented with a base 2 exponent exactly, it must round to the nearest value that can be.
The reason why you are not seeing the precision errors in the print statements is because they are rounded. Use this code to see the rounding error:
local float = '%.32f'
print(float:format(0.1)) --> 0.10000000000000000555111512312578
If you were to use a decimal that when multiplied by a power of two equaled one, then it would look more like so:
local float = '%.64f'
print(float:format(0.5)) --> 0.5000000000000000000000000000000000000000000000000000000000000000
What is the solution to this problem? Instead of using equality checks, check if your value is close to your desired value by some margin of error. That margin of error depends on how many operations you have performed. The maximum rounding error from a single operation or conversion (the machine epsilon) is 2^-53. Any number between 10^−308 and 10^308 can be represented with full 15-17 bit precision but beware because they may not be precisely represented if relatively prime to 2 (in other words, are not a power of two) and over many operations the precision will decrease.
I’d recommend making an application in Lua that simulated floats. There are quite a few rules and hidden gotchas with floats and figuring them out early will save you lots of time down the road. (not to mention make you look cool in front of your colleagues)