How can I optimize my raycast algorithm?

[ThanLeaf (Leaf)]

I’m trying to create a flood fill algorithm for my paper io inspired game to change the color of an enclosed, irregular polygon. However, the code I’ve written is unoptimized, causing the client to freeze when it runs. I have tried adding wait() statements however this just makes it slower and unplayable.

local function floodFill(node, targetColor, replacementColor, vectorDirections)
	local filterRaycast3 = {}

	for _, filterHexagon in ipairs(hexagons) do
		if filterHexagon.BrickColor == BrickColor.new("Really black") then
			table.insert(filterRaycast3, filterHexagon)
		end
	end
	
	local filter = filterRaycast3
	local raycastParams2 = RaycastParams.new()
	raycastParams2.FilterDescendantsInstances = filter
	raycastParams2.FilterType = Enum.RaycastFilterType.Include
	raycastParams2.CollisionGroup = "Inner"
	
	node.BrickColor = replacementColor
	node.CollisionGroup = "Outer"

	for i = 1, #vectorDirections do
		local direction = vectorDirections[i]
		local raycastResult = workspace:Raycast(node.Position, direction * 3.75, raycastParams2)
		if raycastResult then
			local part = raycastResult.Instance
			if part.BrickColor == targetColor then
				floodFill(part, targetColor, replacementColor, vectorDirections)
			end
		end
	end
end

[xXShinyGalaxyXx (Gxlzyz)]

The issue with your flood fill algorithm causing the client to freeze likely stems from a few key problems like preforming raycast for every directory at each nodes, etc. We also need time for Roblox to register the nodes as well so it does not build up and lag.

Let me know if this code doesn’t work, a few changes are Is a more efficient raycast, and checking if the nodes is already there instead of stacking onto each other lagging/freezing the game.

local function floodFill(startNode, targetColor, replacementColor, vectorDirections)
	local filterRaycast3 = {}

	for _, filterHexagon in ipairs(hexagons) do
		if filterHexagon.BrickColor == BrickColor.new("Really black") then
			table.insert(filterRaycast3, filterHexagon)
		end
	end
	
	local filter = filterRaycast3
	local raycastParams2 = RaycastParams.new()
	raycastParams2.FilterDescendantsInstances = filter
	raycastParams2.FilterType = Enum.RaycastFilterType.Include
	raycastParams2.CollisionGroup = "Inner"

	local stack = {startNode}
	local visited = {}

	while #stack > 0 do
		local node = table.remove(stack)

		if visited[node] then
			continue
		end
		visited[node] = true

		node.BrickColor = replacementColor
		node.CollisionGroup = "Outer"

		for i = 1, #vectorDirections do
			local direction = vectorDirections[i]
			local raycastResult = workspace:Raycast(node.Position, direction * 3.75, raycastParams2)

			if raycastResult then
				local part = raycastResult.Instance

				if part.BrickColor == targetColor and not visited[part] then
					table.insert(stack, part)
				end
			end
		end

		if #visited % 50 == 0 then
			wait()
		end
	end
end

[ThanLeaf (Leaf)]

Thank you so much! This drastically improves the lag when it runs, although there is a very small lag spike at the start which only happens when testing in studio; not in the published game.