Removing repeated items in a table

I have a table like so:

local mytable = {
"Yes",
"Hello",
"How do you do?",
"Hello",
"Yes",
"No"
}

This is just a very simplified version of what I have in my game. (I’m basically looping through recipes, adding all recipes matching the items given to a table like the one above, and then trying to remove repeating variables then output what you can craft with those items on a piece of paper (like Cube Combination) but I won’t get into it)

I’m wondering, as in the table above, I have 2 “Yes” and 2 “Hello” how do I refine the table so it ends up being this:

local mytable = {
"Yes",
"Hello",
"How do you do?",
"No"
}

(removed the repeating variables)

1 Like

I’m not sure, but… how are they getting in there in the first place? Maybe if you check if something is already there before you add it, then you wouldn’t need to remove it.

1 Like

This is one way I see it possible.

local _table = { "hi", "no", "hi", "hi" } 

for index, item in _table do
	local searchIndex = table.find(_table, item)
	
	if searchIndex ~= index then
		print("found duplicate")
		
		table.remove(_table, searchIndex)
	end
end

It only works for arrays, I can write another one for dictionaries if you want, and the only problem is that it only removes one, let me rewrite it, this is a better one. It’s O(n^2), but it should work for dictionaries, also.

local _table = { "hi", "no", "no", "hi", "hi" } 

for index, item in _table do
	for searchIndex, searchItem in _table do
		if searchIndex == index then continue end
		
		print("found duplicate")

		table.remove(_table, searchIndex)
	end
end
2 Likes

Since you don’t want elements to show up more than once in the collection, there’s a better type of collection that you can use: a set. A set is basically a collection where there aren’t repetitions. The usual way of making sets in Lua is like so:

local set = {} -- a new, empty set
set[someValue] = true -- adding a value to a set (any non-nil value could be used in place of true)
set[someValue] = true -- "adding" a value again doesn't change anything
set[someValue] = nil -- removing a value from a set
local isValueInSet = set[value] ~= nil --checking if a value is in a set
for value, _ in pairs(set) do -- Iterating over the set. pairs won't do as we're not working with an array-like table.
  print(value)
end

Sets have some cool properties, like constant-time lookup and insertion in some cases. If you really want to find the unique values of a list instead, here’s an implementation that uses sets to make it faster for large input sizes:

function unique(list)
    local seen = {}
    for _, v in list do
        seen[v] = true
    end
    local result = {}
    for v, _ in seen do
        table.insert(result, v)
    end
    return result
end
function unique(tb)
	local keep,scan={},{}
	for _,v in ipairs(tb) do
		if not scan[v] then
			table.insert(keep, v)
			scan[v]=true
		end
	end
	return(keep)
end

Emma, sorry your function name was too good to pass up! What’s with the double for loop? :stuck_out_tongue_winking_eye:

Removing halfway through the iteration might cause errors. Maybe make a duplicate, and you could use next or select to do things with that?

local myTable = {"Hello", "Hello", "Bye", "Good day"}
local clone = table.clone(myTable)

for key, value in next, myTable, nil do
    local isClone = table.find(select(table.unpack(myTable), key))
    if isClone then
        table.remove(clone, isClone)
    end
end

myTable = clone

this is not tested, it might not work

local myTable = {"hello", "yes", "no", "idk", "hello", "yes"}

local function removeDuplicates(t)
    local newTable = {}
    for _, v in t do
        if table.find(newTable, v) then continue end
        table.insert(newTable, v)
    end
    return newTable
end

print(removeDuplicates(myTable))

This only applies to Python, another programming language. In lua, we don’t have sets. I know this as a professional python and lua scripter. Lua defines {} as tables (arrays and dictionaries are types of tables). You can’t automatically remove items in a ‘list’ (python terminology for table).

True, I believe it would be something like a hashmap in lua. It’s still possible and arguably has the same functionality, though.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.