ALP, ALP UTF-8 and IEEE 754 Compression (floating-point precision array Compression)

[Wyv_n (Wyvern)]

these modules (ALP, ALP UTF-8 and IEEE 754) compresses an array into a string of compressed characters. The regular ALP cannot be stored in DataStores so thats why theres an ALP and ALP UTF-8. ALP UTF-8 can be stored in datastores.

Its use case is to compress an array of float numbers and output a string. ALP UTF-8 is a version of ALP but instead of using bytes it uses UTF-8 characters.

ALP UTF-8 will also use byte characters like regular ALP, but only ones in UTF8.

Other website for IEEE 754

IEEE 754 Compression (floating-point array Compression)

I created a compression module very easy to use. So basically it converts an Array of float numbers (decimal numbers) to a string of characters. Thought I shared this with yall so if yall need floating point number compression here it is. I’ll make ALP compression in the future. (ALP: Adaptive Lossless floating-Point Compression) Demo code local IEEE754 = require(script.IEEE754) local myLongDecimalArray = {} for i = 1, 10 do -- put 10 random numbers in array myLongDecimalArray[i] = math.ran…

Example Usage

local ALP = require(script.ALP)
local ALPUTF8 = require(script.ALPUTF8)
local IEEE754 = require(script.IEEE754)

local myArray = {}
for i = 1, 10 do
	myArray[i] = math.random()
end

local ALPCompression = ALP.compress(myArray)
print("-ALP-")
print("Length:", #ALPCompression, ALPCompression)

local IEEE754Compression = IEEE754.compress(myArray)
print("-IEEE754-")
print("Length:", #IEEE754Compression, IEEE754Compression)

local ALPUTF8Compression = ALPUTF8.compress(myArray)
print("-ALPUTF8-")
print("Length:", #ALPUTF8Compression, ALPUTF8Compression)

-- Decompression

local myALPArray = ALP.decompress(ALPCompression)

local myIEEE754Array = IEEE754.decompress(IEEE754Compression)

local myALPUTF8Array = ALPUTF8.decompress(ALPUTF8Compression)

On decompression they all return the same results. It returns the array you sent through with a 0.000000001 number difference (doesnt really matter even for big scale projects.)

Compression Result

ALP and ALP UTF-8 provide the best array compression, but regular ALP is not compatible with datastores…
ALP UTF-8 is compatible with datastores
IEEE754 is another compression method alternative.

They all provide the same results on decompression but the compression size is different.

How much it compresses your data
ALP Compression will compress your data 500% smaller
ALP UTF-8 Compression will compress your data 500% smaller
IEEE754 Compression will compress your data 250% smaller
ALP Base64 Compression will compress your data 350% smaller

Compression Ratio
ALP: 5
ALP UTF8: 5
IEEE754: 2.5
ALP Base64: 3.5 (BROKEN, DEPRECATED)

ALP Compression, ALP UTF-8 Compression, ALP Base64 Compression, IEEE 754 Compression

Downloads For Compression Modules
ALP.lua (1.6 KB)
IEEE754.lua (1.6 KB)
ALPUTF8.lua (1.6 KB)

(DONT USE, OLD)
ALPBase64.lua (2.8 KB) (BROKEN, DEPRECATED)

This is used on my Machine Learning Module, to compress very big AI brains (Neural Networks)

OpenML - Machine Learning

OpenML An Open Source Machine Learning module. This module supports Neural Networks. I plan to add more to this module. OpenML is easy to use and understand, it was built for ease of use and beginner knowledge. All usage cases and examples are explained as much as possible as I could for new programmers. OpenML has a lot of Type Checking so when using functions or getting objects it should tell you what’s there. What is Machine Learning Its a type of Artificial Intelligence that uses algo…

[bluebxrrybot (blueberry)]

Could you benchmark compressing and decompressing all of these modules? I would love to see which is the fastest!