Vector 3 Variable Values Turning Into Decimals

Help and Feedback Scripting Support
#1

My problem is quite simple. At one point I save a Vector3 variable. I will be comparing the Z coordinate later. I print the same value that was assigned to the Z coordinate to make sure it works and it does. It prints the correct number. However, once I reference this variable in another part of the script, it receives a value that is very similar but has a very small decimal attached. For example, 54.4 became 54.400001525879. This causes problems as I then need to compare this number in an if statement and 54.4 is not equal to 54.400001… I cannot just drop the decimal because as seen in the example, I am dealing with comparing precise numbers, down to the decimal point. Where is this decimal coming from and how can I make it work as intended? Thanks. Here is where the variable is assigned and where it is referenced:
Note: Any whole numbers or numbers with a decimal of 0.5 seem to be working correctly

table.insert(notes, Vector3.new(
			(gui.ScreenGui.NoteMarker.Position.X.Scale - (1 - gui.ScreenGui.NoteBoundary.Size.X.Scale) / 2) * (1 / gui.ScreenGui.NoteBoundary.Size.X.Scale), 
			(gui.ScreenGui.NoteMarker.Position.Y.Scale - (1 - yscale) / 2) * (1 / yscale),
			math.floor(workspace.Song.TimePosition * nps) / nps))
			print(math.floor(workspace.Song.TimePosition * nps) / nps)

Note: nps stays constant most of the time
The X and Y here are irrelevant. The only value that I am comparing is the Z

Here, I cycle through the table that contains this vector3 variable and compares it to the value that was initially assigned to it. I have checked to make sure that the value that it is being compared to is the correct number.

local function GetCurrentNote()
	for i = 1, #notes do
		print(notes[i].Z.."z")
		if notes[i].Z == math.floor(workspace.Song.TimePosition * nps) / nps then
			return notes[i]
		end
	end
	print(math.floor(workspace.Song.TimePosition * nps) / nps)
	return false
end

The first print in this function is the one that contains all the decimals. The second one is fine.

1 Like
#2

I think this is a result of floating point error. What you could do is multiply it by 10, floor it, then divide it by 10 again.
Example:

local z = 54.400001525879
z = z * 10
z = math.floor(z)
z = z / 10 -- Now 54.4

Note that this might not be the best solution, but I think it’ll work.

One liner:

z = math.floor(z*10)/10
1 Like
#3

Thanks for the response. My concern with this is that in some cases I may need values with double digit decimals. Also, how come this is only necessary AFTER it gets saved as a variable and not when the value is taken directly from the source.?

#4

If you want multiple digits after the decimal point, you can do:

local wantedDigits = 2
local mult = 10^wantedDigits
z = math.floor(z*mult)/mult

I made it a variable to show what Z was(in the other post). In this case the z is an actual variable. That was just a result of my bad explanation, sorry.

#5

Unfortunately, most decimal fractions cannot be represented exactly as binary fractions. A consequence is that, in general, the decimal floating-point numbers you enter are only approximated by the binary floating-point numbers actually stored in the machine.

Quoted from the python page about this topic. Hope this helps.

1 Like
#6

Are you suggesting this goes when I am reading the variable or assigning it?

#7

You should do what I stated in the above post when comparing the Z to a value.
Example:

function roundTo(num, digits)
    local mult = 10^digits
    return math.floor(num*mult)/mult
end
local faultyZ = workspace.Part.Position.Z
local accurateZ = roundTo(faultyZ, 2)
if accurateZ == 54.41 then
     print('yay')
end
#8

Thanks for the help. I have never come across this limitation of floating-point storage until now. I will probably end up just comparing the two numbers to make sure they are within a range of each other. I will look into all of these options.