As you move the numbers being divided closer together, you are pushing the significant figures further to the right, which makes the result less accurate. Try entering 3.1 in here, then 3.01, then 3.001 and so on. The significant bits move right. Since bits are falling off the right side of the number, that has to mean the accuracy is worsening. Eventually it will be impossible to tell the number from 3.
https://www.h-schmidt.net/FloatConverter/IEEE754.html
The derivatives you want have exact answers. Are you more concerned with how to solve these local slope calculations numerically?