Why is this so inconsistent? What do I change?

I’m trying to create a playerlist that just clones a button template for every player in a server, and sometimes it works and sometimes it doesn’t. I ran with 2 clients, and the button didn’t even get cloned, I ran with 3 clients, it worked fine, I ran with 3 clients again, and 2/3 of the clients actually loaded the buttons.

Code:
local player = game.Players.LocalPlayer
local playerUI = player:WaitForChild(“PlayerGui”)

local players = game:GetService(“Players”)
local UIS = game:GetService(“UserInputService”)
local ReplicatedStorage = game:GetService(“ReplicatedStorage”)
local RS = game:GetService(“RunService”)

local partyUIFolder = playerUI:WaitForChild(“ScreenGui”):WaitForChild(“PartyElements”)

local function refreshPlayerlist()
for _, child in ipairs(partyUIFolder.PlayerlistContainer:GetChildren()) do
if child:IsA(“Frame”) and child.Visible then
child:Destroy()
end
end

for _, p in ipairs(players:GetPlayers())  do
	local template = partyUIFolder:WaitForChild("PlayerlistContainer"):WaitForChild("Template"):Clone()
	
	template.Parent = partyUIFolder.PlayerlistContainer
	template.Visible = true
	template.UsernameLabel.Text = p.Name

	if ReplicatedStorage.RemoteFunctions.PlayerInParty:InvokeServer(p) then
		--print(player.Name .. " " .. p.Name)
		template.InviteButton.Text = "IN-PARTY"	
	else
		template.InviteButton.Text = "+"
	end
end

end

refreshPlayerlist()
game.Players.PlayerAdded:Connect(refreshPlayerlist)
game.Players.PlayerRemoving:Connect(refreshPlayerlist)

Hierarchy:

Apologies for the formatting issues I just copy and pasted what I have

the template is probably getting destroyed

first, define the template beforehand for best practices
then parent the template to the script (or somewhere else) so that it doesnt get effected by the frame destruction pass

local template = partyUIFolder:WaitForChild("PlayerlistContainer"):WaitForChild("Template")

template.Parent = script

now the frame destruction pass also doesn’t have to check for the visible property

if child:IsA("Frame") then
	child:Destroy()
end

The loop is synchronous, but actual rendering and replication timing can vary slightly per client.
Trying to adjust for that here..

LocalScript
--LocalScript in StarterPlayerScripts
local player = game.Players.LocalPlayer
local playerUI = player:WaitForChild("PlayerGui")
local players = game:GetService("Players")
local ReplicatedStorage = game:GetService("ReplicatedStorage")

local partyUIFolder = playerUI:WaitForChild("ScreenGui"):WaitForChild("PartyElements")
local container = partyUIFolder:WaitForChild("PlayerlistContainer")
local template = container:WaitForChild("Template")

local function refreshPlayerlist()
	for _, child in ipairs(container:GetChildren()) do
		if child:IsA("Frame") and child.Visible then
			child:Destroy()
		end
	end

	for _, p in ipairs(players:GetPlayers()) do
		local cloned = template:Clone()
		cloned.Visible = true
		cloned.Parent = container
		cloned.UsernameLabel.Text = p.Name

		if ReplicatedStorage.RemoteFunctions.PlayerInParty:InvokeServer(p) then
			cloned.InviteButton.Text = "IN-PARTY"
		else
			cloned.InviteButton.Text = "+"
		end
	end
end

refreshPlayerlist()
players.PlayerAdded:Connect(refreshPlayerlist)
players.PlayerRemoving:Connect(refreshPlayerlist)