How Can I Detect Changes in a Nested Table with metamethods and newproxy?

Help and Feedback Scripting Support
#1

Hi! I’m trying to figure out how I can take this table,

local data = {
	Inventory = {
		Apple = 5,
		Knife = 2,
		Gun = 0
	},
	Effects = {
		Fire = true,
		Sparkles = false,
	}
}

And be able to detect changes in every single index of the table with one proxy no matter what level the index is in the table. This is the proxy I have for detecting table changes which only work on the first level (Inventory and Effects)

local proxy = newproxy(true)
local meta = getmetatable(proxy)
meta.__index = data
meta.__newindex = function(self, i, v)
	if data[i] ~= v then
		print('change from', data[i], 'to', v)
		data[i] = v
	end
end

If I were to do
proxy.Inventory = nil
The change would be detected successfully!

However, if I did:
proxy.Inventory.Apple += 5
The data table is successfully modified, but __newindex is not called! How can I have __newindex get called within a nested table?

#2

well what you could do is set the __index to a function, and when a table is returned you can set the metatable of that too

#3

I’m not sure how I would do this.

-- Here I turn __index into a function and self is returned table. No matter what part of the table I index, it always returns the parent `data` table.
meta.__index = function(self,i)
	print(self,i)
	return self
end

data.Inventory.Apple = 35 -- this actually creates an index at `data.Apple = 35`, not `data.Inventory.Apple`. `data.Inventory.Apple` remains = 5
-- It also does not fire __newindex.

Edit
I managed to create a sort of listener function by wrapping a proxy around a table using a function like so:

local function proxyify(tab)
	local proxy = newproxy(true)
	local meta = getmetatable(proxy)
	meta.__index = tab
	meta.__newindex = function(self, i, v)
		if tab[i] ~= v then
			tab[i] = v
			print('change from', i, 'to', v)
		end
	end	
	return proxy
end

mainproxy = proxyify(data)
invproxy = proxyify(data.Inventory)

invproxy.Apple=2
invproxy.Gun=5
print(data) -- data.Inventory.Apple changes to 2 and fires __newindex! Awesome!

The only caveat is that I wish there could be a way to just create one proxy which would fire __newindex for all tables in the nested table instead of specifying which index of the nested table I want to be tracked using my function. I will keep this unsolved if anyone has any solutions to that problem in italics.

#4

I can’t think of a way to do this with a single proxy table, but if you use empty tables as proxies instead of using userdatas created with newproxy, you’ll only need one metatable for the whole nested table. You can then make a proxy table hierarchy that is used when detecting changes. If you store data about the parent and child relationships of the proxy tables, you can even find the whole key/index path from the top table to the changed key/index in a descendant table.

I wrote some code that does the things mentioned above. It’s meant to be a class in a modulescript. If a new table, which can be nested too, is added somewhere inside the nested table, it automatically gets its own proxies in the proxy hierarchy. References to proxies for removed tables are cleaned up.

I’ve tested the code a little but not much so it’s possible that there are problems that I haven’t noticed. For example, I’m not sure if I did the reference clean up properly, so there might be memory leaks.

I’ll go to sleep soon, so if you have any questions, I probably won’t reply before tomorrow.

local NestedTableWithChangeDetection = {}

local SPECIAL_DATA_INDEX = "_CHANGE_DETECTION_DATA"

-- This is similar to Instance:GetFullName(). The first key in this string is a key in the main table
local function getHierarchyString(k, proxyTable, hierarchy)
    local parentProxies, keys = hierarchy.parentProxies, hierarchy.keys
    -- tostring in case the key is something else than a string
    local hierarchyString = tostring(k)
    local childProxy = proxyTable
    while parentProxies[childProxy] ~= nil do
        hierarchyString = tostring(keys[childProxy]) .. "." .. hierarchyString
        childProxy = parentProxies[childProxy]
    end
    return hierarchyString
end

local function clearProxyData(proxyTable, hierarchy)
    local keys, parentProxies, childProxies, dataTables = hierarchy.keys, hierarchy.parentProxies, hierarchy.childProxies, hierarchy.dataTables
    local proxyTablesToRemove = {proxyTable}
    local currentParents = {proxyTable}
    while #currentParents > 0 do
        local newParents = {}
        for _, parent in ipairs(currentParents) do
            for _, child in ipairs(parent) do
                table.insert(newParents, child)
                table.insert(proxyTablesToTemove, child)
            end
        end
        currentParents = newParents
    end
    childProxies[ parentProxies[proxyTable] ][ keys[proxyTable] ] = nil
    for _, proxy in ipairs(proxyTablesToRemove) do
        keys[proxy], parentProxies[proxy], childProxies[proxy], dataTables[proxy] = nil, nil, nil, nil
    end
end

-- last three are only use when adding a new table to an existing nested table
local function createProxyData(dataTable, meta, existingHierarchy, proxyParent, key)
    -- keys: key of a proxy in its parent's childProxies tables
    -- childProxies tables are used instead of proxy[childKey] = childProxy so that changes to childKey's value are detected by newindex
    local hierarchy = existingHierarchy or {
        keys = {},
        parentProxies = {},
        childProxies = {},
        dataTables = {}
    }
    local keys, parentProxies, childProxies, dataTables = hierarchy.keys, hierarchy.parentProxies, hierarchy.childProxies, hierarchy.dataTables

    local proxyMain = {}
    childProxies[proxyMain] = {}
    dataTables[proxyMain] = dataTable

    local currentParents = {dataTable}
    local currentProxyParents = {proxyMain}
    while #currentParents > 0 do
        local newParents, newProxyParents = {}, {}
        for i, parent in ipairs(currentParents) do
            local proxyParent = currentProxyParents[i]
            -- v is a child table or a value
            for k, v in pairs(parent) do
                if type(v) == "table" then
                    local vProxy = {}
                    dataTables[vProxy] = v
                    childProxies[proxyParent][k] = vProxy
                    parentProxies[vProxy] = proxyParent
                    keys[vProxy] = k
                    childProxies[vProxy] = {}
                    table.insert(newParents, v)
                    table.insert(newProxyParents, vProxy)
                    setmetatable(vProxy, meta)
                end
            end
        end
        currentParents = newParents
        currentProxyParents = newProxyParents
    end
    
    if existingHierarchy then
        keys[proxyMain] = key
        parentProxies[proxyMain] = proxyParent
        childProxies[proxyParent][key] = proxyMain
    else
        dataTable[SPECIAL_DATA_INDEX] = {
            hierarchy = hierarchy
        }
    end
    
    setmetatable(proxyMain, meta)
    return proxyMain
end

-- constructor
-- if onChangeFunction has parameters they should be for hierarchy string, old value and new value.
function NestedTableWithChangeDetection.new(data, onChangeFunction)
    -- Create the metatable. This will be set as the metatable of every table in the nested proxy table.
    local meta = {}
    function meta:__newindex(k, newVal)
        local specialData = data[SPECIAL_DATA_INDEX]
        local hierarchy = specialData.hierarchy
        
        local dataTable = hierarchy.dataTables[self]
        if dataTable[k] ~= newVal then
            local oldVal = dataTable[k]
            if type(oldVal) == "table" then
                clearProxyData(hierarchy.childProxies[self][k], hierarchy)
            end
            if type(newVal) == "table" then
                createProxyData(newVal, meta, hierarchy, self, k)
            end
            local hierarchyString = getHierarchyString(k, self, hierarchy)
            dataTable[k] = newVal
            onChangeFunction(hierarchyString, oldVal, newVal)
        end
    end
    
    function meta:__index(k)
        local specialData = data[SPECIAL_DATA_INDEX]
        local hierarchy = specialData.hierarchy
        
        local dataVal = hierarchy.dataTables[self][k]
        if type(dataVal) == "table" then
            return hierarchy.childProxies[self][k]
        end
        return dataVal
    end

    -- Create hierarchy data in the above hierarchy table.
    return createProxyData(data, meta)
end

return NestedTableWithChangeDetection

Edit: Here’s an example of using it. I also made some other small edits.

local NestedTableWithChangeDetection = require(--[[modulescript]])

local data = {
    Inventory = {
        Apple = 5,
        Knife = 2,
        Gun = 0
    },
    Effects = {
        Fire = true,
        Sparkles = false
    }
}


local function printChange(hierarchyString, oldVal, newVal)
    print(hierarchyString .. " changed from " .. tostring(oldVal) .. " to " .. tostring(newVal) .. ".")
end


local changeDetectData = NestedTableWithChangeDetection.new(data, printChange)

changeDetectData.Inventory.Apple = 6
-- output: Inventory.Apple changed from 5 to 6.
1 Like
#5 
function proxify(tab) -- defining it globally allows us to use it inside of itself
	local proxy = newproxy(true)
	local meta = getmetatable(proxy)
	meta.__index = function(self, i)
        local idx = tab[i]
        if idx and type(idx) == "table" then
            idx = proxify(idx)
        end

        return idx
    end

	meta.__newindex = function(self, i, v)
		if tab[i] ~= v then
			tab[i] = v
			print('change from', i, 'to', v)
		end
	end	
	return proxy
end

local data = {
	Inventory = {
		Apple = 5,
		Knife = 2,
		Gun = 0
	},
	Effects = {
		Fire = true,
		Sparkles = false,
	}
}

local mainproxy = proxify(data)

mainproxy.Inventory.Apple=2
mainproxy.Inventory.Gun=5
print(data)
3 Likes
#6

You don’t need to define the function globally to be able to use it inside itself. If you wrote

local proxify = function(tab)
    -- code
end

you could not use the function inside itself. But if you use one of the syntaxes below, you can.
option 1:

local function proxify(tab)
    -- code
end

option 2:

local proxify
proxify = function(tab)
    -- code
end
1 Like