How do I quantize a number? (to the nearest eight)

Help and Feedback Scripting Support
#1

Hi, I just started making a really quick unicode progress bar generator, however I can’t figure out how to quantalize the mantessa of my number to the nearest eigth, like this:

antécédant image
0.124 -> 0.125
0.126 -> 0.125
0.248 -> 0.250
0.254 -> 0.250

I’m posting this while being very tired so if I don’t answer in the next 5 minutes, prepare for a very late reply.

Thank for helping

Here is my code 
local textProgressBar = {}

local BLOCKS = {[0] = "" --[[ Necessary as indices start at 1 ]], "▏","▎","▍","▌","▋","▊","▉","█"} -- to self: smooth right to left bars are not possible as the necessary characters do not exist in unicode
local INTERVAL = 1/8

function textProgressBar.generateBar(value: number, length: number, minRangeValue: number, maxRangeValue: number): string
	
	local bar = ""
	
	local normalizedValue = (value - minRangeValue) / (maxRangeValue - minRangeValue)
	local scaledValue = length * normalizedValue
	
	local scaledValueWhole, scaledValueMantissa = math.modf(scaledValue)
	
	local quantizedScaledValueMantissa = math.round(INTERVAL * math.floor(scaledValueMantissa / INTERVAL), 3) / INTERVAL -- TODO this is what I don't know how to doooo
	
	for index = 1, scaledValueWhole, 1 do
		bar ..= BLOCKS[8]
	end
	bar ..= BLOCKS[quantizedScaledValueMantissa]
	
	return bar, quantizedScaledValueMantissa, scaledValueMantissa
end

return textProgressBar
#2

You can try using a single integer and then dividing the result during the calculations. That integer would represent how many eighths you want.

That way you won’t have to round the number, since it will always force the number to any of the eighths. Just remember that the input must not be any fractional number and is an integer. If you want to eliminate the possibility of that from arbitrary inputs, just use math.floor or any rounding method pre-processing of the division of eight.

1 Like
#3

I’m sorry I don’t understand, what’s the single integer? and i should divide the result by what? Thanks

#4

By single integer, I meant a number that does not have any decimals: 0, 1, 2, 3 ... 100 ... n

The process would be:

  • arbitrary number input → integer via rounding
  • integer → division by 8, turning it into eights

However, if it is not possible to use integers as input because the data input is specifically containing decimals, that would be a different case.

1 Like
#5

Thanks for the answer! However, the input number will always be a fractionned number (that always starts with 0), so that can’t work, except if i misunderstood?

I’m trying to quantize only the mantessa of a number, and for that I separated the two parts – but I can’t figure out how to quantize it

EDIT: the mantessa is the part of a number after the dot btw

#6

Oh, and for the solution that strictly requires that the data input is arbitrarily decimals, then your solution lies around using modulus, the % operator.

Here’s how you can tackle it, although very rugged:

local function quantizeEights(value)
	local result = value
	if result % 0.125 >= 0.125 / 2 then
		result = result + (0.125 - result % 0.125)
	elseif result % 0.125 < 0.125 / 2 then
		result = result - result % 0.125
	end
	
	return result
end
2 Likes
#7 
local function Quantize(Input)
    return math.floor(Input*8+0.5)/8
end
2 Likes
#8

i don’t really know what quantize mean but ig that its the 8 quarters of 1 like 0.125, 0.250, 0.250, 0.375, 0.5, etc…

function quantizeToNearestEight(num :number)
	return math.floor((num * 8) + .5) / 8
end

print(quantizeToNearestEight(0.126))

if num is 0.124 we multiply it by 8 so it becomes 0.992 we add 0.5 to it to become 1.492 it will get floored down to 1 we divide it by 8 to return it to the original number it becomes 0.125

if num is 0.254 we multiply it by 8 so it becomes 2.032 we add 0.5 to it so it becomes 2.5 we floor it to 2 and divide by 8 so it becomes 0.25

we multiply it by 8 so we will round it to nearest eight based of integers and not decimal we add 0.5 so that the math.floor() will round to nearest whole number for example if num*8 was 1.994 we add 0.5 so that floor function will floor it to 2 and if its 1.499 the floor function will and we divide by 8 to return it back to a quantized number

i spend most of time writing this calculating numbers on calculator

3 Likes
#9

I should also caution that sometimes the resulting number after these operation can have floating-point errors, which result in many decimals in its place. Generally, you cannot fix this issue in the mathematical operations, but you can fix it through string-based representations.

1 Like
#10

i forgot that math.round was a think you can simplify it to

function quantizeToNearestEight(num :number)
	return math.round(num * 8) / 8
end
1 Like
#11

Thanks a lot this works, and Operatik’s works too, but you can’t put two answers as the solution so i flipped a coin and operatik won.

1 Like