Numerical Inaccuracies in Slope Calculations?

[d_reamweaver (Castorice)]

I’m trying to create a script that calculates the slope at a point of two functions, x^2 and ln(x), through the definition of a derivative

The real slopes at x = 3 are 6 and 1/3 respectively

However, for smaller “h” values, I’m getting less accurate results:

When h = 0.1, it produces the results 6.100000000000012 and 0.3278982282299081

When h = 0.0000001, it produces the results 6.000000087880153 and 0.3333333253863202

When h = 0.00000000000001, it produces the results 6.217248937900877 and 0.33306690738754696

Could someone explain why this happens?

local function differentiate(f, x)
	local h = 0.0000001
	return (f(x + h) - f(x)) / h
end

local function xSquared(x)
	return x^2
end

local function ln(x)
	return math.log(x)
end

print(differentiate(xSquared, 3))
print(differentiate(ln, 3))

[shibajuice]

I think it likely has to do with the floating point precision of Lua. Lua only has about 15 digits of precision, and working close to those bounds are going to make the math inaccurate. If you need to have precise digits, I think there are some workarounds that exist (look up BigNum on the forum).

Numbers | Documentation - Roblox Creator Hub

[azqjanna]

As you move the numbers being divided closer together, you are pushing the significant figures further to the right, which makes the result less accurate. Try entering 3.1 in here, then 3.01, then 3.001 and so on. The significant bits move right. Since bits are falling off the right side of the number, that has to mean the accuracy is worsening. Eventually it will be impossible to tell the number from 3.
https://www.h-schmidt.net/FloatConverter/IEEE754.html

The derivatives you want have exact answers. Are you more concerned with how to solve these local slope calculations numerically?

[meguy1]

This is called numerical differentiation. The formula you are using in the ‘differentiate’ function is the simplest way to compute a numerical slope, but there are much more accurate, “higher-order” methods.

See this Wikipedia page for the formulas, and an explanation of the floating-point errors you are seeing:

Numerical differentiation

The “Step Size” section explains the inaccuracies you are seeing.
I recommend using the formula labeled “five point midpoint approximation” for much better results.