Saving large numbers in datastores does not work properly (turns into nil)

(Edit: see how to reproduce the bug here)

Hello,
One of the players in my game has reported a specific value of his data keeps resetting. I looked into it and it seems like his data is getting corrupted everytime GetAsync is used.

I printed the specific value of the table that is being saved when a player joins, or leaves (or whenever the save function is called). I got the player to go in my VIP server and this is what I got:

(excuse my horrible debugging methods)

Whole screenshot

I set the player’s Investors value object to a number, and when he leaves it saves as seen in the screenshot below:

However, when he rejoins, the Investors key’s value is nil as seen in the screenshot below:

This happens to this player on the dev game too (a copy of the game but on another game). This one specific key becomes nil whenever he joins (which is when GetAsync is used). I’m not sure why this happens. It doesn’t happen to other players, only this one player.

This is a really weird bug and I’m not sure what’s causing it. If it’s some sort of datastore limitation on the length of the value that a specific key can have, I’m sure it’s not because it would, first of all, error, and second of all, more players would be experiencing this.

What should I do?

wrap the GetAsync() function in a pcall to get the specific error that’s happening you can read up more on this link: https://developer.roblox.com/articles/Datastore-Errors

There is no error occuring. And I am using pcall, I’m printing them too. No errors in console.
Here’s the code where I use GetAsync:

local success, dataTable = pcall(function()
	
	local data = dataStore:GetAsync(key)
	
	if data then
		
		return data
	end
	
	return nil
end)

if success then -- irrelevant code below

(just realised the return nil is redundant, but that’s besides the point)

I am aware of how to use pcall. See my reply above.

No, I have to explicitly return the data as whatever is returned in pcall() is what the second variable it returns will be (in my example code, the second variable is dataTable).

Look, if I was doing it wrong, none of the players playing my game would be able to have their data saved and loaded. It works fine. The pcall isn’t the issue.

If GetAsync is returning a nil value, that means the player’s data fails to save in all your calls. I can tell you right now it’s not the GetAsync call because while pcall won’t explicitly write the error to the console after being caught, the proceeding code after it will not run. It returns nil, meaning that GetAsync doesn’t fail and it falls back to its alternate return value, nil, if no data exists.

Have you yet attempted to print the results of a write request in pcall?

-- Or UpdateAsync with newValue as transformFunction instead
local success, result = pcall(dataStore.SetAsync, dataStore, key, newValue)
if not success then
    warn(result)
end

The best way to utilise a pcall is to make use of all the arguments it returns rather than throwing them away. The charm of pcall is returning the error mesasge if the function doesn’t succeed in running.


@RomeoEDD If you aren’t running anything else in a pcall, this is a pretty awful way to do it. You have a pointless upvalue hanging around now. It’s somewhat misusing a pcall, even though this is fine. Instead of assigning a value to an upvalue, either return the data or wrap the method directly.

-- Return
local success, result = pcall(function ()
    return goldDataStore:GetAsync(playerKey)
end)

-- Direct wrap
local success, result = pcall(goldDataStore.GetAsync, goldDataStore, playerKey)

Untrue. If GetAsync errors, none of the code after it will run. The function call will terminate and an error message will be returned as the second result from the pcall. The GetAsync call works.

“If GetAsync is returning a nil value, that means the player’s data fails to save in all your calls”

GetAsync doesn’t return nil unless the player is a new player. My issue here is that a specific key in the table that GetAsync returns is somehow nil even though I’m sure I saved it using SetAsync (as seen in screenshots in main post).

Moreover, I log every time the pcall with GetAsync fails on my discord webhook and it works fine:
image

Here’s a look at what I do after the pcall:

“Have you yet attempted to print the results of a write request in pcall?”
Hmm, seems like I missed that out:


However, if SetAsync errored, the next 3 warnings wouldn’t show (and the error would show in console), but in my screenshot in the main post, the warnings showed up. I will wrap the SetAsync with pcall now, but if I’m not wrong that still doesn’t explain why a specific key in a table that I save to a specific key (specific player’s user id, since this only happens to 1 player) in a datastore dissappears the next time I call GetAsync.

Wrong. If there is no data saved to a key, GetAsync will return nil. Your code makes a redundant check by returning data if it is non-nil or returning nil if data doesn’t exist, meaning GetAsync returns nil. GetAsync is not your issue, it is your saving code. Said code is not saving data to the key, which is why GetAsync is returning a nil value.

Unrelated note to your loading code

By the way, in your loading code, you have a blank if statment. This is fairly bad practice:

image

You can simply write

if not success then
    -- Handle no success
end

in this scenario.


It doesn’t explain it because it’s not supposed to. It was a suggestion to attempt. There’s clearly a discrepancy or oversight on your behalf that isn’t being found here and I can’t quite see it right now. Yes, you are correct in saying that a SetAsync call erroring would place a message in your console and terminate the script, however.

Wrong. If there is no data saved to a key, GetAsync will return nil.

Yea that’s what I meant, sorry for poor phrasing. A new player wouldn’t have their key saved to the datastore, which is what I meant.


Unrelated note to your loading code

If you look at the screenshot, I used the arrow thing to hide whatever is in that first if statement so that I can show the else statement in 1 screenshot.

Thank you for trying to help. I have a feeling this isn’t only happening to 1 player/it’s going to happen to more players in the future, so I have to fix it quickly. However, I’m not sure if it’s something I missed out, it seems like a roblox issue, though if it was more developers would be reporting this.

On the flip side, you could use UpdateAsync to transform data. I highly doubt this would help to be frank, but I suppose a short in the dark it is. I really can’t see any immediate issues.

dataStore:UpdateAsync(key, function(old)
    if not old then
        return dataTable
    end

    for key, value in pairs(dataTable) do
        old[key] = value
    end

    return old
end)

This does indeed seem to be a bug with saving large numbers to DataStores.

It can be reproduced by running this snippet on a live game:

local ds = game:GetService("DataStoreService"):GetDataStore("Test")

local key = "numbertest"
local value = 1e19

ds:SetAsync(key,value)
print("Saved:",value)
wait(1)
print("Loaded:", ds:GetAsync(key) or "nil")

Output will be:
image

The SetAsync call doesn’t error and nil is returned when the same key is loaded. Smaller numbers such as 1e18 are saving correctly.

3 Likes

Yeah, this is a weird thing with data stores. Saving large numbers seems to break things. Here’s what worked for me:

  • As simple solution is just to save the number as a string, and then transform it back later
--Saving:
local num = 1e19
local str = tostring(num)
dataStore:SetAsync(key, num)
--Loading:
local num = 0
local result = dataStore:GetAsync(key)
if result then 
    num = tonumber(result)
end

Note that in normal code you would use pcalls, I am typing this on mobile so I had to make it as simple as possible. Obviously, this code will need slight modification. I was just trying to show the concept.

5 Likes

Lets remove the ROBLOXCRITICAL tag, this behavior has been known about forever.

Yeah, but it is not documented. I wish stuff like this would be added to to documentation pages, so people stop finding the same problems.

Put in a documentation request #platform-feedback:documentation-requests they’re pretty quick adding this kinda stuff cause there are so few documentation requests made.

Documenting that might be somewhat helpful, but there still shouldn’t be cases where DataStore calls silently fail and data is lost.

Then it would probably have been a good idea to report it as a bug.

2 Likes

We’ve reproduced this and will be working on a fix. Thanks for the report, this is something our unit tests have (obviously) missed.

8 Likes

Yesterday (July 10th 2019) we enabled a fix that we think will address this issue, please let me know if you are continuing to have this problem.

1 Like