I recently noticed that some players in my game are losing specific data. After trying to find out what is happening I found that some specific numbers get converted to nil after gathering the data from DataStore. I tried converting the table to JSON manually and the same issue happened. When using HttpService:JSONDecode some specific numbers get converted to nil. The only range of numbers I know of right now is from 1e19 to 1.84e19.
Repro:
local HttpService = game:GetService("HttpService")
local t = {
A = 1e18;
B = 1e19;
C = 1e20;
D = 1e300;
E = 1.84e19;
F = 1.85e19
}
local jsonString = HttpService:JSONEncode(t)
local jsonTable = HttpService:JSONDecode(jsonString)
print(jsonString)
for key, value in pairs(t) do
print(key, jsonTable[key])
end
Output:
{"F":18500000000000000000,
"D":1.0000000000000000525047602552e+300,
"E":18400000000000000000,
"B":10000000000000000000,
"C":100000000000000000000,
"A":1000000000000000000}
A 1e+18
B nil
C 1e+20
D 1e+300
E nil
F 1.85e+19
While you can see that JSONEncode works properly after trying to convert the string back to a table The output is nil for some numbers.
All of the numbers you’ve listed are larger than what a single lua number can hold without losing precision, which is roughly 2^53 or ~9e+15, so it’s not particularly surprising you’re seeing odd behavior. Would be useful if Roblox would at least warn the user in these cases however.
One approach to getting around this would be to look into “big number” implementations which would represent your large values in memory using multiple numbers, and would serialize them as strings.
Numbers may lose precision at these levels, but Lua can still hold numbers this large with some trade-offs. The highest I can go before turning into inf is 1e+308:
> print(1e308)
1e+308
> print(1e309)
inf
If you can JSONEncode the number to valid JSON, then obviously the number must be valid in Lua, otherwise there’s no way you could pass it into JSONEncode in the first place. If you pass this number back through JSONDecode then you should get a same or a very near number back.
To me, it looks like these numbers are too large for the JSON library, not Lua.
In fact, it look like it can decode numbers above 1e19 just fine! Look at C, 1e20. It decodes that fine. It can’t decode 1e19, but it can decode 1e18 and 1e20 fine? This is a weird bug on the JSON decoding end, not a limit to Lua numbers or even a limit to numbers in the JSON library.
There’s a difference between “maximum integer that can be represented in a 64 bit double” and “maximum integer that can be represented within 64 bit double without losing precision”. Here’s a simple example to demonstrate this point.
local a = 9007199254740992 -- 2^53
local b = 9007199254740993 -- 2^53 + 1
print(a == b) -- prints "true"
Yea, precision is not good at high numbers but it shouldn’t convert the number to nil. 1e19 is the only range I know of. However, there are much lower numbers where the same thing happens. I got a few reports of coins getting reset when they had around 1e12 but I couldn’t replicate that one.
I’m not sure if this is part of the issue as well, but my game utilizes trello to get information to input into a table. It has worked perfectly fine for the last year and I haven’t really changed anything since then. However, all of a sudden, I was made aware that the game wasn’t fetching information from trello anymore and I got this error when I was trying to investigate.
It’s my assumption it’s an error on ROBLOX’s part since it randomly failed to work today.
Thank you for bringing this to our attention, yesterday (July 10th 2019) we enabled a fix that we think will address this issue, please let me know if you are continuing to have this problem.
Thanks @LordRugdumph for this. We were seeing this as well, and I couldn’t seem to track down the issue, but this could potentially explain it. I will let you know if this continues to happen.
