Lua math is incorrect

[napastnikiek1 (napastnikiek)]

Hello!

This is my second problem with float numbers
My problem is just that math.round() does not work correctly. math.round() changes my numbers to integers, example: math.round(1.2,2) → 1

I subtract 0.1 from 1.5, results:

1. 1.4 --> math.round(1.4,2) --> 1
2. 1.2999999999 --> math.round(2999999999,2) --> 1
3. 1.1999999999 --> math.round(1.1999999999,2) --> 1

but if it subtracts an integer then math.round() works, example:
16 - 0.1 --> 15.9000008 --> math.round(15.9000008,2) --> 15.9

how can fix it?

[fellow_meerkat]

math.round() doesn’t have a second parameter, that 2 is never used. You can multiply by 10, round, and divide by 10. Or you can use string.format("%0.1f", number) (which rounds decimals, with the number after the dot specifying decimal places) and convert the return back to a number.

[msix29]

math.round doesn’t take a second parameter.

[napastnikiek1 (napastnikiek)]

I mean the result, and I know that math.round() rounds with a number.
Can 1.299999999999 be rounded to 1.3?

[Scottifly]

As @fellow_meerkat said, try this
x = 1.299999
math.round(x * 10) /10 = 1.3

[napastnikiek1 (napastnikiek)]

will use string.format(“%0.1f”, number)

[5Scll1 (5Scll)]

This topic has been solved
Only reply here if:

You have additional details

The solution doesn’t work for you

If you have an unrelated issue, please start a new topic instead.

[alexinite (alex)]

Ah the intricacies of floating point numbers!
For those who don’t know why this even occurs with numbers in games or programs for that matter, just computers in general: computers really aren’t the best with integers because numbers on computers have to be stored in bits (which there’s only a finite allocation for) so you get these sort of issues that look like this:

3 - 1.1 = 1.899978

This mainly happens because some bits have to be cut off to store a number in the floating point format, or else they would not fit in the 32/64 bit arrangement they’re given. This problem can usually be solved by rounding it to the nearest whole, but if you’re looking for a decimal placement rounding you could make your own round function:

local function round(x: number, depth: number): number
    depth = 10^(depth - 1)
    return math.round(x * depth) / depth
end

round(3 - 1.1, 2) --> 1.9