How to convert strings into a floating point number?

Help and Feedback Scripting Support
#1

I’m going to be blunt. I’m looking for a way to convert numerical strings directly into their IEEE-754 (floating point) representation in binary. (Without converting the string into a number. “1000” → 1000 isn’t allowed!)
I’ll be using a buffer to store the data.

--example
function stringToFloat(str: string): buffer
    --magic
end

local float = stringToFloat("1000.1") --buffer: 0 10001000 11110100000011001100110
print(buffer.readf32(float, 0)) --1000.0999755859375

Here are some resources you may find helpful.

I’m open to all suggestions, even an explanation on how to do this would be great. :slightly_smiling_face:

#2 
print(tonumber("1.54"))

a very small line of code, i see you have experience with C

#3

Unfortunately for my case, I can’t use tonumber to convert it into a number.

If you want to know the exact reason why: tonumber only works with strings within the range of a double. Any value exceeding what a double can hold will be interpreted as inf. The strings I’m working with will exceed the range of doubles.
I wrote this post in the most literal sense possible.

#4

ohh understood, unfortunately my brain is stinky and small and i cant help you with this, but im very interested and i will try to find informations about. good luck!

1 Like
#5

Written using ChatGPT:

function toIEEE754Binary(numString)
	-- Helper function to convert an integer to a binary string
	local function integerToBinary(integer)
		local binary = ""
		while integer > 0 do
			binary = (integer % 2) .. binary
			integer = math.floor(integer / 2)
		end
		return binary
	end

	-- Helper function to convert a decimal part to a binary string
	local function fractionalToBinary(fractional)
		local binary = ""
		local precision = 23 -- Assuming single precision
		while precision > 0 and fractional > 0 do
			fractional = fractional * 2
			if fractional >= 1 then
				binary = binary .. "1"
				fractional = fractional - 1
			else
				binary = binary .. "0"
			end
			precision = precision - 1
		end
		return binary
	end

	-- Parse the numerical string
	local integerPart, fractionalPart = string.match(numString, "(%d+)%.?(%d*)")
	integerPart = tonumber(integerPart) or 0
	fractionalPart = tonumber("0." .. (fractionalPart or "0")) or 0.0

	-- Convert to binary representation
	local binaryInteger = integerToBinary(integerPart)
	local binaryFractional = fractionalToBinary(fractionalPart)

	-- Normalize the binary number
	local normalizedBinary = binaryInteger .. binaryFractional

	-- Calculate exponent and mantissa
	local exponent = #binaryInteger - 1
	local mantissa = string.sub(binaryInteger .. binaryFractional, 2, 24)

	-- IEEE-754 single precision: 1 sign bit, 8 exponent bits, 23 mantissa bits
	local sign = 0 -- Assuming positive number
	local exponentBits = integerToBinary(exponent + 127) -- Bias of 127 for single precision

	-- Ensure exponentBits is 8 bits long
	exponentBits = string.rep("0", 8 - #exponentBits) .. exponentBits

	-- Ensure mantissa is 23 bits long
	mantissa = mantissa .. string.rep("0", 23 - #mantissa)

	-- Combine sign, exponent, and mantissa
	local ieee754Binary = sign .. exponentBits .. string.sub(mantissa, 1, 23)
	return ieee754Binary
end

-- Example usage
local binaryRepresentation = toIEEE754Binary("1000")
print(binaryRepresentation) --output: 01000100011110100000000000000000

I’m not sure if this is what you are looking for but I hope it helps.

1 Like
#6

You should perhaps look into the work done by things like the Quake 3 quick square root. The code does somewhat whats asked, I have never worked with buffers, and the utility for this is quite low, as you probably will not face a number that is higher than a double that is 64 bits truly.

1 Like
#7

This might be my most complicated post yet.

This video explains how to convert a decimal number into a binary IEEE-754 representation. However, since you’re using numbers that exceed the limits of doubles, we’ll have to use Quadruple-precision floating-point formats instead. This format uses 16 bytes (128 bits).

First, we have to separate the parts of the number into the sign, the whole, and the fraction.

local function stringToFloat(str: string): buffer
    local sign, whole, fraction = string.match(str, "^(%--)(%d+)%.?(%d*)")
end

Next, we’ll calculate the binary representation for the whole number.

To convert the whole into binary, you must divide by two, grab the remainder, and repeat the process until you reach zero. Make sure that when dividing by two, drop the fractional part.

However, division doesn’t work properly on numbers beyond Doubles, so we’ll have to create our own makeshift division by two system with a predefined table.

local dividedBy2 = {
    --formatted as: [number: string] = {wholeNumber: number, decimal: number}
    ["0"] = {0, 0}, --0.0
    ["1"] = {0, 5}, --0.5
    ["2"] = {1, 0}, --1.0
    ["3"] = {1, 5}, --1.5
    ["4"] = {2, 0}, --2.0
    ["5"] = {2, 5}, --2.5
    ["6"] = {3, 0}, --3.0
    ["7"] = {3, 5}, --3.5
    ["8"] = {4, 0}, --4.0
    ["9"] = {4, 5}  --4.5
}

local function floorDivide2(str: string): (string, number) --returns the value and the remainder
    local result, remainder = "", 0
    for i = 1, #str do --go through every digit in str
        local character = string.sub(str, i, i)
        local divisionResult = dividedBy2[character] --get item in dividedBy2 table
        result ..= tostring(divisionResult[1] + remainder) --append digit to result
        remainder = divisionResult[2] --set remainder
    end
    result = string.gsub(result, "^0+", "") --remove prepended zeroes. this will remove the entire number if the string only contains zeroes
    return result, remainder
end

Now, we can use this division system to convert any number into binary.

local function wholeNumberToBinary(str: string): string
    local result, remainder = "", 0
    while str ~= "" do
        str, remainder = floorDivide2(str)
        result = math.sign(remainder) .. result --prepend digit. if remainder is 5 it equals one due to the math.sign() function
    end
    return result
end

And, we will update our stringToFloat function.

local function stringToFloat(str: string): buffer
    local sign, whole, fraction = string.match(str, "^(%--)(%d+)%.?(%d*)")
    local wholeInBinary = wholeNumberToBinary(whole)
end

Secondly, we’ll calculate the binary representation for the fraction. It is simply multiplying by two, taking the digit in the one’s place, and setting it back to zero, and repeating the process with that number instead. This process must be repeated 112 times, since the Quadruple floating point format stores 112 bits for the fraction.

--[[
0.3 * 2 = 0.6 | 0
0.6 * 2 = 1.2 | 1 -- 1.2 turns into 0.2
0.2 * 2 = 0.4 | 0
       ...
]]

We now have to create another makeshift function for multiplying by two. Again, with a predefined table. This is to avoid floating point errors.

local multipliedBy2 = {
    --formatted as: [number: string] = {wholeNumber: number, carryOn: number}
    ["0"] = {0, 0}, --0
    ["1"] = {2, 0}, --2
    ["2"] = {4, 0}, --4
    ["3"] = {6, 0}, --6
    ["4"] = {8, 0}, --8
    ["5"] = {0, 1}, --10
    ["6"] = {2, 1}, --12
    ["7"] = {4, 1}, --14
    ["8"] = {6, 1}, --16
    ["9"] = {8, 1}  --18
}

local function multiplyBy2(str: string): string --this only works for whole numbers
    local result, carry = "", 0
    for i = #str, 1, -1 do --go through every digit in str backwards
        local character = string.sub(str, i, i)
        local multipliedResult = multipliedBy2[character] --get item in multipliedBy2 table
        result = tostring(multipliedResult[1] + carry) .. result --prepend digit to result
        carry = multipliedResult[2] --set carry
    end
    if carry > 0 then result = tostring(carry) .. result end --prepend carry to result
    return result
end

Now, we can turn a fraction into binary.

local function fractionToBinary(str: string): string
    local result = ""
    for _ = 1, 112 do
        str = multiplyBy2(str)
        local digit = string.sub(str, 1, 1)
        result ..= digit
        str = `0{string.sub(str, 2)}`
    end
    return result
end

And, we will update the stringToFloat function again.

local function stringToFloat(str: string): buffer
    local sign, whole, fraction = string.match(str, "^(%--)(%d+)%.?(%d*)")
    local wholeInBinary = wholeNumberToBinary(whole)
    local fractionInBinary = fractionToBinary(`0{fraction}`) --always prepend a zero at the start
end

Afterwards, we’ll need to find the exponent. First, we can combine both the wholeInBinary and the fractionInBinary variables together. And, we’ll locate the first 1 in that string and take it’s index.
Afterwards, we’ll use that number to subtract it from the length of the wholeInBinary variable.
Then, take that result and use that to subtract it from the exponent bias which is 16383. Therefore, the equation is 16383 + (len of wholeInBinary - index of the first one in the string)

local function stringToFloat(str: string): buffer
    ...
    local binaryRepresentation = wholeInBinary .. fractionInBinary
    local index = string.find(binaryRepresentation, "1")
    local exponent = 127 + (#wholeInBinary - index)
end

Afterwards, convert the exponent into 15 bits.

local function stringToFloat(str: string): buffer
    ...
    exponent = wholeNumberToBinary(tostring(exponent))
    for _ = 1, 15 - #exponent do
        exponent = "0" .. exponent
    end
end

Next, we’ll truncate the binaryRepresentation into a string of 112 bits where the starting point is after the first 1. This is the mantissa.

local function stringToFloat(str: string): buffer
    ...
    binaryRepresentation = string.sub(binaryRepresentation, index + 1, 113 + index)
end

Now, we’ll convert the sign into either a 1 or 0, depending on whether the number is negative or positive.

local function stringToFloat(str: string): buffer
    local sign, whole, fraction = string.match(str, "^(%--)(%d+)%.?(%d*)")
    ...
    sign = sign == "-" and "1" or "0"
end

We have our sign, exponent, and mantissa, so we can combine them together to get a Quadruple precision floating point number!

local function stringToFloat(str: string): buffer
    ...
    local binaryResult = `{sign}{exponent}{binaryRepresentation}`
end

Finally, we can put them inside a buffer type, which will store our number.

local function stringToFloat(str: string): buffer
    ...
    local numberBuffer = buffer.create(16) --create the buffer
    local encodedString = ""
    for i = 1, 128, 8 do --convert the binary result into a string
        local byte = tonumber(string.sub(binaryResult, i, i + 7), 2)
        encodedString ..= string.char(byte)
    end
    buffer.writestring(numberBuffer, 0, encodedString) --write it to the buffer
    return numberBuffer
end

Full code (Contains no comments):

local dividedBy2 = {
    ["0"] = {0, 0},
    ["1"] = {0, 5},
    ["2"] = {1, 0},
    ["3"] = {1, 5},
    ["4"] = {2, 0},
    ["5"] = {2, 5},
    ["6"] = {3, 0},
    ["7"] = {3, 5},
    ["8"] = {4, 0},
    ["9"] = {4, 5}
}

local multipliedBy2 = {
    ["0"] = {0, 0},
    ["1"] = {2, 0},
    ["2"] = {4, 0},
    ["3"] = {6, 0},
    ["4"] = {8, 0},
    ["5"] = {0, 1},
    ["6"] = {2, 1},
    ["7"] = {4, 1},
    ["8"] = {6, 1},
    ["9"] = {8, 1}
}

local function floorDivide2(str: string): (string, number)
    local result, remainder = "", 0
    for i = 1, #str do
        local character = string.sub(str, i, i)
        local divisionResult = dividedBy2[character]
        result ..= tostring(divisionResult[1] + remainder)
        remainder = divisionResult[2]
    end
    result = string.gsub(result, "^0+", "")
    return result, remainder
end

local function multiplyBy2(str: string): string
    local result, carry = "", 0
    for i = #str, 1, -1 do
        local character = string.sub(str, i, i)
        local multipliedResult = multipliedBy2[character]
        result = tostring(multipliedResult[1] + carry) .. result
        carry = multipliedResult[2]
    end
    if carry > 0 then result = tostring(carry) .. result end
    return result
end

local function wholeNumberToBinary(str: string): string
    local result, remainder = "", 0
    while str ~= "" do
        str, remainder = floorDivide2(str)
        result = math.sign(remainder) .. result
    end
    return result
end

local function fractionToBinary(str: string): string
    local result = ""
    for _ = 1, 112 do
        str = multiplyBy2(str)
        local digit = string.sub(str, 1, 1)
        result ..= digit
        str = `0{string.sub(str, 2)}`
    end
    return result
end

local function stringToFloat(str: string): buffer
    local sign, whole, fraction = string.match(str, "^(%--)(%d+)%.?(%d*)")
    local wholeInBinary = wholeNumberToBinary(whole)
    local fractionInBinary = fractionToBinary(`0{fraction}`)
    local binaryRepresentation = wholeInBinary .. fractionInBinary
    local index = string.find(binaryRepresentation, "1")
    local exponent = 16383 + (#wholeInBinary - index)
    exponent = wholeNumberToBinary(tostring(exponent))
    for _ = 1, 15 - #exponent do
        exponent = "0" .. exponent
    end
    binaryRepresentation = string.sub(binaryRepresentation, index + 1, index + 113)
    sign = sign == "-" and "1" or "0"
    local binaryResult = `{sign}{exponent}{binaryRepresentation}`
    local numberBuffer = buffer.create(16)
    local encodedString = ""
    for i = 1, 128, 8 do
        local byte = tonumber(string.sub(binaryResult, i, i + 7), 2)
        encodedString ..= string.char(byte)
    end
    buffer.writestring(numberBuffer, 0, encodedString)
    return numberBuffer
end
2 Likes
#8

Holy crap! What an elegant and brilliant solution! You blew my mind! Thank you for the thorough explanation and code. I mean it.
:smile:

1 Like
#9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.