Storing large amounts of text (100 MB) in a single script

[dinil900]

I’m writing a basic LLM based on llama2.c with 15M weights, and it turns out that to store all the data (in hex), I will need to store (15*4)*2=120MB of characters in a script! Will this much of characters break anything if I upload the game with the script?

To not store big strings, I store small strings in an array, for example:

bin = {"1B00000000000000050000003C756E6B",
	   "3E00000000050000000A3C733E0A0000",
...

Should I perhaps divide all the data in smaller module scripts (example - 60 scripts for 2 MB each)? Will Roblox think that I’m trying to make a DDOS attack on the servers?

[SeargentAUS (Seargent)]

I would recommend against trying to do a LLM in Roblox, as I doubt Roblox’s servers will be able to handle the processing required. However, you should be fine to store it all in a script.

[12345koip]

You could also try using lossless compression methods like Run-Length Encoding to compress your hex strings further if you need.

[dinil900]

I’m using a relatively small model (15M weights is very small compared to the original 8B), so it could work at, let’s say 1 t/s. The original C implementation runs at 100t/s, and the Python one runs at 4t/s, so it should work.

[dinil900]

I highly doubt that it will do a lot, as most of data is practically random. The data I gave you was from the tokenizer, my bad. The model itself is just a list of floats in hexadecimal. Example:

"D19DDC3C5F75C53B69F2E63C1502A83A",
"9A37C93C76792BBD8E60B4BA59B2A33C",
"4D24AEBC36FB0D3BED266E3A5AE94F3D",
"B84FAB3A7A5EF3BCF3B525BD8127333D",

[12345koip]

RLE was just an example for the strings you provided, you could apply other lossless compression methods, such as Lempel-Ziv-Welch encoding, to these more complex strings.

[NyrionDev (Nyrion)]

@dinil900

store all the data (in hex)

store it in base64, it should take 80MB of space. Also you can try compressing the base64 string and rounding the weights to some decimal point. Alternatively you can also store the LLM data in multiple datastore keys and fetch them one by one to construct the weights so your studio editing doesn’t lag(you should need around 20 different datastore keys/API calls).

You can also store the data in an external source and use HTTP service to fetch it. In general its good to avoid having unreasonably large scripts during studio edit, because it can cause lag, increase place load time and uploading/updating time.

[dinil900]

Here is a possible method to converting to base64 with loss:

--   D       1       9       D       D       C       3       C
--                      hex to binary
--1:1:0:1:0:0:0:1|1:0:0:1:1:1:0:1|1:1:0:1:1:1:0:0|0:0:1:1:1:1:0:0
--               little endian to big endian
--0|0:1:1:1:1:0:0:1|1:0:1:1:1:0:0:1:0:0:1:1:1:0:1:1:1:0:1:0:0:0:1
--Sign   Exponent                   Mantissa
--0|0 : 1:1:1:0:0:1|1:0:1:1:1:0:0:1:0:0:1:1:1:0:1:1       remove precision
--A A   A A A A B B B B B B C C C C C C D D D D D D     which letter to encode in
--0:0 : 1:1:1:0|0:1:1:0:1:1|1:0:0:1:0:0|1:1:1:0:1:1
--       O           b           k           7          encode to base64
--lossy compression x2, max error of .00153%

Using the HTTP service is a good idea, since there is a copy of the file at huggingface.

[dinil900]

Here is the solution, using HttpService

--loads httpservice
HttpService = game:GetService("HttpService")
--model url
URL = "https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin"
--model, as text file
response = HttpService:GetAsync(URL)
--display the first 12 values
print(string.unpack("iiiiiiifffff",string.sub(response,1,256)))

Output:

288 768 6 6 6 32000 256 -0.05982464551925659 -0.0145612433552742 0.10614923387765884 0.021650167182087898 -0.004335478879511356 49