Help with modulo in custom math module

Help and Feedback Scripting Support
#1

The module works by storing the mantissa and the exponent of a value (e.g. 150 becomes 1.5e2)
If I remember correctly, a mod b is a - (b * (floor(a/b))
So in my module, I have the exact same formula.

mt.__mod = function(Value, Modulus)
	if Value < Modulus then return Infinity.Clone(Value) end
	local Result = Value - (Modulus * (Infinity.floor(Value/Modulus)))
	return (Result.Mantissa ~= Result.Mantissa and Infinity.new(0) or Result:Normalize()) -- NaN ~= NaN
end

However, assuming that everything else is working fine, this keeps returning me weird values that are supposed to be normalized. But I am getting results such as 10.0e-1 where a=13 and b=2 which completely breaks my __eq metamethod.

mt.__eq = function(Value, Other)
	return (Value.Exponent == Other.Exponent and Value.Mantissa == Other.Mantissa)
end

I know something here is wrongly coded but I cannot seem to find the problem.

The whole module since this involved a bit more than just 2 simple functions (just ignore anything that this doesn’t use):
pastebin

#3

Have you tried using the lua debugger to step through the code line-by-line and make sure that all the variables are what you expect?

#4

Unless I’m mistaken you never defined Result.Mantissa, in line 4 of the first script (or, at least line 4 of what you showed us)

#5

It is defined

{Mantissa = (Mantissa or 0),Exponent = (Exponent or 0)}

Plus, every basic operation calls Infinity.new()

#6

The problem is the error in the roblox math library’s own calculations that you have not considered. When we work with floating point numbers we should never assume that they will be integers. So when you do “math.log10(math.abs(self.Mantissa)” in the normalization method, the mantissa turns out to be slightly less than one, so its logarithm is negative. So applying math.floor() to that will result in -1 and not 0 as you expected.

And since you also did not foresee negative cases, that error arose. This should fix the error, although it is only a patch

function mt:Normalize()
	local val = math.log10(math.abs(self.Mantissa))
	if val > 0 then  -- do not normalize small mantissa numbers
		self:Denormalize(-math.floor(val))
	end
	return self
end

I also found another error at a glance

mt.__lt = function(Value, Other)
	return (Value.Exponent > Other.Exponent and false or Value.Exponent < Other.Exponent or Value.Mantissa < Other.Mantissa)
end

you should change it to something like this

mt.__lt = function(Value, Other)
	if Value.Exponent > Other.Exponent then
		return false
	end
	return Value.Exponent < Other.Exponent or Value.Mantissa < Other.Mantissa
end