Why are Luau floating-point calculations inaccurate?

TL;DR:
Luau floating-point calculations are somewhat inaccurate. Why is that?

Recently, I was creating a system where a player could upgrade the cooldown of his tools and needed to get a percentage of how upgraded the tool was. However, when printing the output of the code, I noticed that there was a floating-point error. Here’s a simplified version of my code:

local startCooldown = 1.5
local endCooldown = 0.5
local currentCooldown = 1.3

local percent = (currentCooldown - startCooldown) / (endCooldown - startCooldown)

print(percent) -- Outputs 0.19999999999999996

The output is 0.19999999999999996, when the expected value should be 0.2

This is the same code but in C++:

#include <iostream>
using namespace std;

int main()
{
    float startCooldown = 1.5;
    float endCooldown = 0.5;
    float currentCooldown = 1.3;
    
    // Equivalent of the print() function
    cout << (currentCooldown - startCooldown) / (endCooldown - startCooldown); // Outputs 0.2

    return 0;
}

Here, the output matches the expected value of 0.2.

I did some more testing:

print(0.1 + 0.2) -- Outputs 0.30000000000000004
#include <iostream>
using namespace std;

int main()
{
    cout<< 0.1 + 0.2; // Outputs 0.3

    return 0;
}

I am aware that this happens because certain decimal values can’t be perfectly represented in binary. My theory is that there are certain techniques which aim to improve accuracy that are disabled in Luau.

Does anyone know why this happens? Why did Roblox choose to disable the accuracy improvements?

This is a common characteristic with floating-point arithmetic in programming languages, it is unlikely that Roblox disabled accuracy improvements.

A workaround would be to round the numbers so it avoids the representation of an inaccurate number

print(math.round(percent * 1e6) / 1e6) -- Outputs 0.2 after rounding to 6 decimal places
1 Like

To me these post always seem like its blaming roblox (there are other things to worry about)

Perhaps its the c++ compiler automatically rounding and not showing the precision of the values?

3 Likes

Thanks! That’s a handy way to round a number to a certain amount of decimal places. I’ll definitively keep it handy.

Thanks so much for letting me know! I apologize if it seemed like I was trying to blame Roblox, as that wasn’t the intention of the post. I didn’t know that this also happened in C++, except that you needed to manually set the precision level of the output using std::cout << std::setprecisionlevel();

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.