How can I find the location of userdata in my tables?

Hey,
So, sparingly, there’s an issue in my DataStore system that causes :SetAsync() to error. I believe some userdata is sneaking into the player’s session data table, but the error message doesn’t give me much information, and differs from what’s in the official documentation for DataStore errors found here.

My error says:
104: Cannot store Dictionary in DataStore

I was told this could happen when you to save userdata to a datastore.
I’m adding and changing a lot of the player’s sessionData prior to attempting to save it to a datastore, but I’m not intentionally trying to save userdata… it must be slipping in by accident.

Is there a bit of code I can run to recursively scan the contents of a table to determine if there exists userdata, and more importantly, determine which of its contents are the perpetrators?

Thanks,
– Soybeen

Untested, but…

local function search(_table)
    for index, value in pairs(_table) do
        if type(index) == "userdata" or type(value) == "userdata" then
            print(index, value)
        elseif typeof(index) == "table" then
            search(index)
        elseif typeof(value) == "table" then
            search(value)
        end
    end
end

For an alternative you might want to change how you save.

Really easy to implement:


local PlayerData = {}

local HttpService = game:GetService("HtttpService")
local baseData = HttpService:JSONEncode({
     Money = 0,
     Items = {},
     Level = 1,
     Something = {
            Dunno = 0,
            End = 1,
}
})

game.Players.PlayerAdded:connect(function(Player)
       local Data = DataStore:GetAsync(Player.userId) or baseData
       Data = HttpService:JSONDecode(Data)

       PlayerData[Player] = Data
end)

game.Players.PlayerRemoving:connect(function(Player)
       local Data = PlayerData[Player]
       Data = HttpService:JSONEncode(Data)
       DataStore:SetAsync(Player.userId,Data)
       PlayerData[Player] = nil
end)

local Items = require(game.ReplicatedStorage.Items)
function game.ReplicatedStorage.RemoteFunction.OnServerInvoke(Player,...)
     local t = {...}
     if  t[1] == "Buy" then
           local item = t[2]
           local Price = Items[item].Price
           if PlayerData[Player].Money >= Price then
                 PlayerData[Player].Money = PlayerData[Player].Money - Price
                 table.insert(PlayerData[Player].Items,item)
                 return "success"
           else
                 return "not enough money"
           end
     end
end

I’ve never had data problems using this.

You can easily change from not using this to using it by implementing a dataswap when they join if they have existing data.

You can save literally anything, and you’re only saving and loading once, or if you want you could do something like:

spawn(function()
     while wait(180) do --save every 3 minutes incase of unexpected things
           for i,Player in pairs (game.Players:GetPlayers()) do
                  local Data = PlayerData[Player]
                  Data = HttpService:JSONEncode(Data)
                  DataStore:SetAsync(Player.userId,Data)
           end
     end
end)
2 Likes

I wrote a neat thing that traverses the given table and will return if it’s valid or not. If invalid, it returns the path to the invalid key/value, the reason, and any extra info.

I’ve included test cases. It should be easy to remove those and use this to your advantage by looking at the test code as an example.

local function typeValid(data)
	return type(data) ~= 'userdata', typeof(data)
end

local function scanValidity(tbl, passed, path)
	if type(tbl) ~= 'table' then
		return scanValidity({input = tbl}, {}, {})
	end
	passed, path = passed or {}, path or {'input'}
	passed[tbl] = true
	local tblType
	do
		local key, value = next(tbl)
		if type(key) == 'number' then
			tblType = 'Array'
		else
			tblType = 'Dictionary'
		end
	end
	local last = 0
	for key, value in next, tbl do
		path[#path + 1] = tostring(key)
		if type(key) == 'number' then
			if tblType == 'Dictionary' then
				return false, path, 'Mixed Array/Dictionary'
			elseif key%1 ~= 0 then  -- if not an integer
				return false, path, 'Non-integer index'
			elseif key == math.huge or key == -math.huge then
				return false, path, '(-)Infinity index'
			end
		elseif type(key) ~= 'string' then
			return false, path, 'Non-string key', typeof(key)
		elseif tblType == 'Array' then
			return false, path, 'Mixed Array/Dictionary'
		end
		if tblType == 'Array' then
			if last ~= key - 1 then
				return false, path, 'Array with non-sequential indexes'
			end
			last = key
		end
		local isTypeValid, valueType = typeValid(value)
		if not isTypeValid then
			return false, path, 'Invalid type', valueType
		end
		if type(value) == 'table' then
			if passed[value] then
				return false, path, 'Cyclic'
			end
			local isValid, keyPath, reason, extra = scanValidity(value, passed, path)
			if not isValid then
				return isValid, keyPath, reason, extra
			end
		end
		path[#path] = nil
	end
	passed[tbl] = nil
	return true
end

local function getStringPath(path)
	return table.concat(path, '.')
end

local function warnIfInvalid(input)
	local isValid, keyPath, reason, extra = scanValidity(input)
	if not isValid then
		if extra then
			warn('Invalid at '..getStringPath(keyPath)..' because: '..reason..' ('..tostring(extra)..')')
		else
			warn('Invalid at '..getStringPath(keyPath)..' because: '..reason)
		end
	else
		print('Valid')
	end
end

---

local cyclicTest = {
	a = {{b = {}}}
}

cyclicTest.a[1].b[1] = cyclicTest

local testCases = {
	true, 'hello', 5, 5.7,  -- all valid
	CFrame.new(),  -- invalid: type
	{
		true, 'hello', 5, 5.7
	},  -- valid array
	{
		a = true, b = 'hello', c = 5, d = 5.7
	},  -- valid dictionary
	{
		a = true, 'hello', 5, 5.7
	},  -- invalid: array/dictionary mix
	{
		CFrame.new()
	},  -- invalid: type in array
	{
		in1 = {
			{
				in2 = {
					a = true, 'hello'
				}
			},
			5
		},
		in3 = {}
	},  -- invalid: array/dictionary mix deep in path
	{
		[5.7] = 'hello'
	},  -- invalid: decimal index
	{
		[{}] = 'hello'
	},  -- invalid: non-string key
	{
		[1] = 'hello',
		[3] = 'WRONG',
	},  -- invalid: non-sequential array
	cyclicTest,  -- invalid: cyclic
	{
		[math.huge] = 'hello'
	},  -- invalid: infinity index
}

for _, case in next, testCases do
	warnIfInvalid(case)
end

image

Non-sequential arrays are technically valid, but everything after the sequence breaks will be cut off and you will lose data. It’s better to recognize that as invalid data than to ignore it.

11 Likes

Make a separate Table

PlayersData[Player] = {}
PlayersData[Player].SaveData ={}

this should work just fine since you are only saving what you want to and not everything in the PlayerData.

The last few times I had to store playerdata (and not just one or two values), I worked with a class that also had a :Serialize() (and deserialize) method, which would give me a table (or string? I forgot) that could easily be saved to DataStore, transferred over RemoteEvents, … seems better than just storing a whole data table if it also contains session-specific variables that get reset on rejoin or so anyway.

2 Likes

I’ve had the same issue in the past, maybe this topic could help you.

(Read all replies)

Thanks, this is a very useful tool