There is something wrong with my voting system

It registers votes for all the players in the server when just one person steps on the trigger part (I know this as it only does 1 vote with 1 player in the server and 3 votes with 3 total players in the server). Below is a video demonstration of this problem.
here

So here is my code that I used to script the voting system. See if it has any issues here (ignore the print statements pls)

-- Server
local function GetVoteTally(button)
	local voteCount = {}
	local winningMode = nil
	local winningVotes = 0
	for mode, data in modeVotes do
		voteCount[mode] = 0
		for _, v in data do
			voteCount[mode] += 1
		end
		if voteCount[mode] > winningVotes then
			winningVotes = voteCount[mode]
			winningMode = mode
		end
		
		local modeTally = workspace.VotingPlace.Modes[mode]
		modeTally.SurfaceGui.Votes.Text = voteCount[mode]
	end
	
	--VoteRemote:FireAllClients(voteCount)
	
	return winningMode
end

local function ProcessVote(player, button)
	local mode = button.UI.Value.ModeName.Text
	if modeVotes[mode][player.UserId] then
		return
	end
	
--	print(player, "voted for", mode)
	for i, oldmode in modeVotes do
		oldmode[player.UserId] = nil
	end
	print(modeVotes)
	
	modeVotes[mode][player.UserId] = true
	print(modeVotes)
	GetVoteTally()
end

VoteRemote.OnServerEvent:Connect(ProcessVote)
-- Client script

-- Handling mode voting in the lobby
for _, button in workspace.VotingPlace.Surfaces:GetChildren() do
	local NonVotedColour = BrickColor.new("Fossil")
	local votedColour = BrickColor.new("Bright green")
	local debounce
	
	button.Touched:Connect(function(part)
		if not part:IsDescendantOf(player.Character) and button.Debounce.Value == true then
			return
		end
		print(debounce)
		debounce = button.UI.Value.ModeName.Name
		
		for i, value in pairs(workspace.VotingPlace.Surfaces:GetChildren()) do
			if value ~= button or value.UI.Value.ModeName.Text == "" then
				value.BrickColor = NonVotedColour
				button.Debounce.Value = false
			else
				voteRemote:FireServer(button)
				value.BrickColor = votedColour
				button.Debounce.Value = true
			end
		end
		
--		task.wait(0.1)
--		debounce = false
	end)
end

If you can help, reply below!

Change the and into an or. You want to early return if EITHER of those conditions are true (either the hit is not the player’s character OR debounce is true). Currently your logic is returning only if both conditions are true.

Surprisingly, it works although still not bug-free

Update: I’ve removed the part-based debounce system (which has some problems associated with it and isn’t really that useful) and replaced it with an in-script debounce to provide buffer time between each vote