Modulo Operator Inaccuracy

Help and Feedback Scripting Support
#1

Hello Developers,

I am working on a math function which requires the use of the modulo operator. However, I noticed slight inaccuracies in it’s calculation. Here’s what I mean:
image
image
(This also happens with other alternatives I’ve seen)
image

Are there any alternatives or fixes for this?

NOTE: All I’m trying to do is remove numbers before the decimal point

6 Likes
#2

It’s called a floating point error, very common in all programming languages.

You can round the value to the nearest hundredth using string.format.

Code:

local x = string.format("%0.2f", 1.37%1)
3 Likes
#3

Thanks! Any other way to do this with a math function (math.floor), other than string.format?

1 Like
#4

AFAIK your original solution is the only way I know.

Usually these inaccuracies don’t matter all the much in calculations and the only problems that they cause typically involve displaying text to the user, which is what the other solution is for.

I would usually just leave it alone since it’s not a big deal but if it really matters, go ahead with either for the above.

1 Like
#5

How would I format this string that would give the same result as:

string.format(“%0.2f”, 1.37%1)*10^2
(converts decimal point to a number)

#6

In this case it does, I am making a timer which counts every hundredth of a second, it is pretty noticeable when there are extra decimals

#7

This actually did not end up working. I am still getting extra decimals.

#8

Try this (decimalPlaces is how many you want to round to, so in your case, pass in 2):

local function ClearFloatingPointError(number, decimalPlaces)
	local str = tostring(number)
	decimalPlaces = math.max(decimalPlaces, 0)

	local formatted = string.format(`%.{decimalPlaces}f`, number)
	formatted = formatted:gsub("%.?0+$", "") or 0

	return tonumber(formatted)
end
#9

Sorry, can you be more specific?

How does this timer work and what does it do exactly? + What are you using it for/why is it needed?

#10

You don’t need to know any of that, as I’ve stated, I am trying to get only the decimals of a number.

#11

if you know how many decimal spaces to expect then you can just use something like this to get rid of the floating point error

local function round(x: number, decimals: number)
    local pow10 = 10 ^ decimals
    return math.floor(x * pow10) / pow10
end
#12

As I’ve stated in the original post, this still comes with inaccuracies. Not all floating points go up, some go down by 0.000000001.
The actual solution is this:

local number = 1.37
local digits = 2
(string.format(`%0.{digits}f`, number)*10^digits)/10^digits
#13

oh i meant to use math.round, that would solve the issue. your solution converts it to a string, which isn’t really efficient if you want to do more math operations on it and sometimes might cause errors.

#14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.