Limiting Precision Issues With Vector3

Help and Feedback Scripting Support
#1

I am facing a precision issue with floating-point numbers when creating a Vector3 object. I use the string.format function to round a number to one decimal place, which gives me the desired output, such as -8.9 or 0.0. However, when I use these formatted numbers to create a Vector3 object, I still end up with values like -8.899999618530273 or 0.0 with trailing zeros.

The error seems to occur during the creation of the Vector3 itself, and I am concerned that using CFrame might result in similar issues. I am looking for alternative solutions to create a Vector3 with the desired precision without encountering these floating-point number problems.

The reason I need such precision is that I want to check if a part already exists at a certain position, and I want to limit the possible positions to avoid creating a part per millimeter. However, due to rounding errors and the limited precision of floating-point numbers, this becomes a challenge.

local worldPosition = WorldCFrame.Position

local x, y, z =
	tostring(worldPosition.X),
	tostring(worldPosition.Y),
	tostring(worldPosition.Z)

x = string.gsub(x, "(%d+%.%d)%d*", "%1")
y = string.gsub(y, "(%d+%.%d)%d*", "%1")
z = string.gsub(z, "(%d+%.%d)%d*", "%1")

print(x, y, z) -- -8.9 0.0 -1.3

worldPosition = Vector3.new(x, y, z)
print(worldPosition) -- -8.899999618530273, 0, -1.2999999523162842

local isOccupied = self.WorldPositionHistory[worldPosition]

Has anyone encountered a similar issue and found an effective solution for creating a Vector3 with precise precision without rounding errors or trailing zeros? I am open to any suggestions or ideas to resolve this issue.

Thank you in advance for your help!

1 Like
#2

This is a common problem with floating-point arithmetic. The limited precision of floating-point numbers means that some decimal values cannot be accurately represented, leading to rounding errors.

You could multiply the position values by a scaling factor then round them to the nearest integer.

local worldPosition = WorldCFrame.Position

-- defines the scaling factor (10 for 1 decimal place)
local scalingFactor = 10

-- scale the position values by the scaling factor
local x, y, z =
    math.round(worldPosition.X * scalingFactor),
    math.round(worldPosition.Y * scalingFactor),
    math.round(worldPosition.Z * scalingFactor)

print(x, y, z)

-- use the scaled and rounded values to create a Vector3 object
worldPosition = Vector3.new(x / scalingFactor, y / scalingFactor, z / scalingFactor)

print(worldPosition)

local isOccupied = self.WorldPositionHistory[worldPosition]