Help needed in brainstorming a good pathfinding system

I think you’d get all kinds of false-negatives with the system you’re talking about. If the block cast starts inside terrain there’s a good chance it hits nothing at all. If it collided with the terrain when starting inside of it that would definitely be a solid option.

Sometimes Roblox’s pathfinding dips like 60 studs into terrain, and it has tears all over the nav-mesh which break continuity. It’s fairly accurate but I don’t know if that matters all that much. Would be helpful to actually build a testable system and see whether that’s necessary or not.

I’m in the process of building a test system that uses the method I described and I’m not sure if the raycasting is what causes the majority of the slowdown anymore, because I can’t even run a simple program that moves the cells around and adds the results to an array, without any collision tests, without the script timing out or the game crashing (no errors):

local function createGrid(position, size, depth)
	depth = depth or 0
	local node = cells[size]
	if not node then warn("node size does not exist") return end
	node.Position = position
	if depth < OCTREE_DEPTH - 1 then
		local childSize = size * 0.5
		for i = 1, 8 do
			local multiplier = SUB_REGION_POSITION_OFFSET[i]
			createGrid(position + (multiplier * childSize * 2), childSize, depth + 1)
		end
	else
		grid[position.X] = grid[position.X] or {}
		grid[position.X][position.Z] = grid[position.X][position.Z] or {}
		grid[position.X][position.Z][position.Y] = position
	end
end

-- ### RUNTIME
print("beginning grid construction")
local t = os.clock()
createGrid(Vector3.zero, OCTREE_SIZE)
local total_1 = os.clock() - t
print("grid construction complete:", string.format("%.2f", total_1))
print(grid)

I’m doing it to receive an array of values that I can “snap” future raycast results to. I’m sure there’s another way to do it, but I think that tying it into the parts collision scan will cut down on redundancy by a lot, and anyways we’ve gone through dozens of examples now where this hasn’t been a problem.

I figured out that what’s making it crash is trying to print the grid at the end, which I don’t understand, as it’s just the same type of 3D array we’ve been using for the nodes up to this point, except this one contains all possible nodes. I need to be able to check that the grid is being filled out correctly before I can feel comfortable continuing with my experimental top-down raycasting method, and right now if I try to figure out what the size of the array’s first layer is with print(#grid) I get 1. If I specify slices like print(grid[127]) it can work, but that doesn’t give me any idea of what the first layer of the array looks like.

Also worth mentioning is that I still have to add task.wait() for running the octree at scale 512, because the script execution will time out if I don’t. This is despite there being no raycasting or any other type of collision checking occurring at the moment.

@CompilerError I got around to verifying the structure of the grid array and managed to create a prototype of the Raycast system I was talking about. You were worried about false negatives, but there actually seems to be the opposite problem; there’s too many false positives.



The first images look pretty similar to the results from the octree scanner I was working on before trying this experiment, but in the last image, when I test it on caves, it seems to be catching the cave ceilings and walls as well. I haven’t tested the original scanner on caves yet, but I would imagine it would get similar results. I have no idea what could be causing this in the top-down Raycast method, because it only casts from the top down, not to the sides or upward. Either way, if these methods return these results, then I can see it easily messing with the array of surfaces.

The thing that surprised me the most is just how fast it is. It took me roughly 3 1/2 seconds to perform hundreds of thousands of workspace:Blockcast()s, and I didn’t have to worry about padding out the run time with task.wait()s. Like I mentioned earlier in this thread, I’m starting to think that the problem isn’t the number of Raycasts, but moving around the cells.

Here’s the place file if you want to mess around with it yourself:
terrain-raycasting.rbxl (2.2 MB)

1 Like

I combined the raycasting system with the old parts scan and surface parsing. By far, the longest part of the process is moving the cells around in the beginning and adding the data gained to the appropriate tables.

As mentioned in previous posts, I combined the :GetPartsInPart() collision scan and grid creation (filling out a table of all possible node positions) in the same step. I could separate those processes into separate steps to get a more in-depth time breakdown, but based on the initial raycast test place, I would imagine the collision scan would only take a second or two at most. The biggest bottleneck right now is trying to move the cells around without timing out the script or crashing Studio.

Hoping to get peoples’ thoughts on this. I don’t have any ideas of my own on how to cut down on the completion time or strain on the system. I’d be willing to provide the place file, if necessary. I’m also looking to exclude cave ceilings from the finished node array; the current surface parsing function I have isn’t capable of that.

Quick update, but I tried the typical optimization when it comes to BaseParts of making the cells completely transparent while they were being moved around and making collision checks; it made no meaningful difference and might have even made it slower.

Try not to use table.insert its slow try setting the index after the last index of the array instead same result.

Something like this:

local Table = {}
Table[#Table+1] = --whatever value you want

EDIT: I may be wrong I haven’t tested this I just hear that table.insert is slow.

Nope, they’re virtually identical. Either way, this kind of stuff is the wrong thing to focus on when talking about performance.

1 Like

Does this work on moving parts, models, mesh etc .

Fo you have an example .rbxl with s working npc utilizing it ?

Thanks.

This scan is meant to be run once on startup and then never again. It assumes the environment doesn’t move or change. It doesn’t factor in NPCs or players because it’s meant to be a pathfinding grid; if it did and treated them like any other part of the environment, it would cause problems.

hi thanks for the place file was fun playing with this. I took out all the parts and just used the info.
The scan only take about 21 seconds.

-- ### VARIABLES & CONSTANTS
-- cell variables
task.wait(10)
local SS = game:GetService("ServerStorage")
local octreeCell = SS:WaitForChild("OctreeCell")

-- creating octree model
local octree = Instance.new("Folder")
octree.Name = "Octree"
octree.Parent = workspace
-- parameters for scans
local paramsOL = OverlapParams.new()
paramsOL.FilterDescendantsInstances = {octree}
paramsOL.FilterType = Enum.RaycastFilterType.Exclude
paramsOL.MaxParts = 1
local paramsRC = RaycastParams.new()
paramsRC.FilterDescendantsInstances = {octree}
paramsRC.FilterType = Enum.RaycastFilterType.Exclude
-- nodes table
local grid = {}
local nodeCount = 0
local nodes = {}
local octreeTable = {}
-- octree constants and variables
local OCTREE_SIZE = 512
local OCTREE_DEPTH = 9
local MINIMUM_SIZE = OCTREE_SIZE / (2 ^ (OCTREE_DEPTH - 1))
local SUB_REGION_POSITION_OFFSET = {
	Vector3.new(0.25, 0.25, -0.25),
	Vector3.new(-0.25, 0.25, -0.25),
	Vector3.new(0.25, 0.25, 0.25),
	Vector3.new(-0.25, 0.25, 0.25),
	Vector3.new(0.25, -0.25, -0.25),
	Vector3.new(-0.25, -0.25, -0.25),
	Vector3.new(0.25, -0.25, 0.25),
	Vector3.new(-0.25, -0.25, 0.25)
}
-- populating cell table
local cells = {}
local lastSize = OCTREE_SIZE * 2
for i = 1, OCTREE_DEPTH do
	local size = lastSize * 0.5

	--local p = octreeCell:Clone()
	--p.Size = Vector3.one * size
	--p.Position = Vector3.zero
	--p.Anchored = true
	--p.CanCollide = false
	--p.CanQuery = false
	--p.CanTouch = false
	--p.Transparency = 1
	--p.Parent = workspace

	cells[size] = Vector3.zero
	lastSize = size
end
print(cells)
-- raycast params
local raycastParams = RaycastParams.new()
raycastParams.FilterDescendantsInstances = {workspace.Terrain}
raycastParams.FilterType = Enum.RaycastFilterType.Include

-- ### FUNCTIONS
-- creates a debug node
local function debugNode(position)
	local p = Instance.new("Part")
	p.Transparency = 1
	p.Position = position
	p.Size = Vector3.one * MINIMUM_SIZE
	p.Color = octreeCell.Color
	--p.Transparency = octreeCell.Transparency
	p.Anchored = true
	p.CanCollide = false
	p.CanQuery = false
	p.CanTouch = false
	p.Locked = true
	p.Parent = octree
	p.Name = octreeCell.Name
end

-- ### MAIN FUNCTIONS
-- function to generate a grid of nodes
local function createGrid(position, size, depth)
	depth = depth or 0
	if depth < 3 then task.wait() end
	local node = cells[size]
	if not node then warn("node size does not exist") return end
	node = position
	if depth < OCTREE_DEPTH - 1 then
		local childSize = size * 0.5
		for i = 1, 8 do
			local multiplier = SUB_REGION_POSITION_OFFSET[i]
			createGrid(position + (multiplier * childSize * 2), childSize, depth + 1)
		end
	else
		grid[position.Y] = grid[position.Y] or {}
		grid[position.Y][position.X] = grid[position.Y][position.X] or {}
		grid[position.Y][position.X][position.Z] = position
	end
end

local function terrainScan()
	for i, cell in cells do
		cell = Vector3.zero
		table.remove(cells,i)
	end
	local max = OCTREE_SIZE * 0.5 - (MINIMUM_SIZE * 0.5) if not grid[max] or not grid[-max] then warn("grid incomplete") return end
	for x, zList in pairs(grid[max]) do
		--task.wait()
		--local castCount = 0
		for z, position in zList do
			--task.wait()
			local lastSpot = Vector3.new(x, max, z)
			local noResult = false
			--local castCount = 0
			repeat
				--task.wait()
				local result = workspace:Blockcast(CFrame.new(lastSpot), Vector3.one * MINIMUM_SIZE, Vector3.new(x, -max, z) - lastSpot)
				if result and result.Position.Y >= -max then
					local node = Vector3.new(x, math.floor((result.Position.Y - 1) * 0.5 + 0.5) * 2 + 1, z)
					--print(lastSpot, "\n", string.format("%.2f", result.Position.X), string.format("%.2f", result.Position.Y), string.format("%.2f", result.Position.Z), "\n", node)
					nodes[node.X] = nodes[node.X] or {}
					nodes[node.X][node.Y] = nodes[node.X][node.Y] or {}
					nodes[node.X][node.Y][node.Z] = node
					
					lastSpot = node + Vector3.new(0, -MINIMUM_SIZE, 0)
					if lastSpot.Y < -max then noResult = true end
				else
					noResult = true
				end
				--castCount += 1
			until noResult == true
			--print(x, z, castCount)
		end
		--print(x, castCount)
	end
end

local function generateMap()
	local buff = 0
	for x, yList in nodes do
		if buff % 10 == 0 then task.wait() end
		task.wait()
		for y, zList in yList do
			for z, position in zList do
				table.insert(octreeTable, position)
				--debugNode(position)
			end
		end
		buff += 1
	end
end

-- ### RUNTIME
print("beginning grid construction")
local t = os.clock()
createGrid(Vector3.zero, OCTREE_SIZE)
local total_1 = os.clock() - t
print("grid construction complete:", string.format("%.2f", total_1))
print("beginning surface scan")
t = os.clock()
terrainScan()
local total_2 = os.clock() - t
print("terrain scan complete:", string.format("%.2f", total_2))
print("generating map")
t = os.clock()
generateMap()
local total_3 = os.clock() - t
print("visualization complete:", string.format("%.2f", total_3))
print(#octreeTable, "nodes found:", string.format("%.2f", total_1 + total_2 + total_3))

Have you tried using what I made and building on it? I don’t think BlockCast is in a good enough spot to be used for this case just yet. The reason it hits your cave walls is likely because the inside of the terrain isn’t even considered collision tests and the BlockCast ends up hitting the inside (inside of the terrain side) of the cave wall.

1 Like

Are you able to provide a diagram of what you think is happening? I’m afraid I don’t quite follow you. I did take another look at your scanner and was able to fix my code so that it takes only 20 seconds to complete (18 of which are making the debug nodes), so I could go either route at the moment. The problem there was the whole grid array, which I got rid of entirely and just used math to calculate the points to raycast to and from.

1 Like

I extended functionality so that it gets the bounding box of the entire map, calculates the minimum amount of root nodes needed to cover the distance, and then fills in a roots table with the positions of those nodes that it feeds into the scanning functions. I tested this on one of the starter maps and was able to generate a node map for the entire build in about 6-7 seconds (this is before I added the debug nodes in). Something I discovered is that some terrain types are better for not getting false positives than others.

As I started adding on more terrain, though, the function I made for placing the root nodes stopped working. It won’t examine the long arms of terrain I added onto the original map. It’s important to get this (or something like it) working because my maps are not going to be as small as the starter maps; they’re going to be quite large. I have no idea what could be causing this.

I’ve shifted focus to trying to complete the system with the current terrain scan rather than continue testing between mine and @CompilerError’s different forms. That way, I can get something working and test with AI such that I can see the effects the two different methods of scanning have and testing between the two is as simple as swapping them in and out.

Here’s what I suspect is happening. The Blockcast starts inside of the terrain and collides with certain parts of the other side of the terrain collision mesh. This is likely because the normal of the polygon in the collision mesh it hit is facing outward relative to the Blockcast. Think of terrain as a hollow object where there are chunks of terrain, the blockcast can hit either side of the outer faces of the terrain collision mesh (in certain cases*)

This is what I thought was happening, too, but testing one of the place files I’ve made with the system and leaving out the surface-finding process, I think the answer may be more complicated.

This is a shot of an underground cave system in the terrain. In the center of the frame is a dead end in that cave system that doesn’t peak above ground. There are nodes on the edges that would definitely support your hypothesis, but the ceiling and the opposing wall not being included make me think there’s more to it. If a raycasting error like you described was the only problem, wouldn’t they also have false-positive nodes?

It’s strange and almost seems random how, in the image above, certain chunks of the wall create extraneous nodes and others don’t. What I think may be happening is that it detects all the relatively jagged protrusions in the walls and counts them as surfaces, which is why some materials are more prone to having errors in the walls than others. Then again, there seem to be a lot of cases in the above images where the shape of the walls should be setting off false positives but isn’t.

I think the angle of the face hit by the Blockcast is playing a role here. Likely, the internal ray cast logic can “cull” out faces that it should never hit, this reduces the number of necessary polygons to check. The faces that are being hit are being considered by the ray caster because they’re facing in a direction that the ray technically could hit. There’s not really going to be a great way to deal with this using Blockcast, which is why I was thinking that a separate terrain scan step made more sense here. To be perfectly honest, I don’t think you’ll ever get any kind of reliability going this route, even when they do make it so ray casts collide with objects they’re inside of.

I went back to take another look at your overfill method and I found out that one of the biggest reasons your nodes were inaccurate was because the script was placing them in the wrong place. Your offsets were wrong, which I can relate to because I had the same problems when figuring out the script for automatically placing the root nodes a week ago. I managed to fix it and it’s much more accurate now:

local TERRAIN_CELL_OFFSETS = {
	Vector3.new(-1, -1, -1),
	Vector3.new(1, -1, -1),
	Vector3.new(-1, 1, -1),
	Vector3.new(-1, -1, 1),
	Vector3.new(1, 1, -1),
	Vector3.new(1, -1, 1),
	Vector3.new(-1, 1, 1),
	Vector3.new(1, 1, 1)
}

local function terrainScan(origin, root)
	local min = origin - (Vector3.one * root * 0.5);

	local materials, occupancies = workspace.Terrain:ReadVoxels(Region3.new(min, origin + (Vector3.one * root * 0.5)), 4);

	local size = materials.Size;

	for x = 1, size.X do
		for y = 1, size.Y do
			for z = 1, size.Z do
				local material = materials[x][y][z];
				local position = min + (Vector3.one * 2) + (Vector3.new(x - 1, y - 1, z - 1) * 4);
				
				if (material ~= Enum.Material.Air and material ~= Enum.Material.Water) then
					--debugNode(position, 4)
					nodes[position.X - 1] = nodes[position.X - 1] or {};
					nodes[position.X + 1] = nodes[position.X + 1] or {};
					nodes[position.X - 1][position.Y - 1] = nodes[position.X - 1][position.Y - 1] or {};
					nodes[position.X - 1][position.Y + 1] = nodes[position.X - 1][position.Y + 1] or {};
					nodes[position.X + 1][position.Y - 1] = nodes[position.X + 1][position.Y - 1] or {};
					nodes[position.X + 1][position.Y + 1] = nodes[position.X + 1][position.Y + 1] or {};

					nodes[position.X - 1][position.Y - 1][position.Z - 1] = position + TERRAIN_CELL_OFFSETS[1];
					nodes[position.X + 1][position.Y - 1][position.Z - 1] = position + TERRAIN_CELL_OFFSETS[2];
					nodes[position.X - 1][position.Y + 1][position.Z - 1] = position + TERRAIN_CELL_OFFSETS[3];
					nodes[position.X - 1][position.Y - 1][position.Z + 1] = position + TERRAIN_CELL_OFFSETS[4];
					nodes[position.X + 1][position.Y + 1][position.Z - 1] = position + TERRAIN_CELL_OFFSETS[5];
					nodes[position.X + 1][position.Y - 1][position.Z + 1] = position + TERRAIN_CELL_OFFSETS[6];
					nodes[position.X - 1][position.Y + 1][position.Z + 1] = position + TERRAIN_CELL_OFFSETS[7];
					nodes[position.X + 1][position.Y + 1][position.Z + 1] = position + TERRAIN_CELL_OFFSETS[8];
				end
			end
		end
	end
end



I was actually going to combine my and your methods and have raycasting within the colliding voxels to find the surfaces, but now that the voxels are accurate, that doesn’t seem necessary or practical. I’m going to go with your method for now and come back to mine if problems come up.

2 Likes

Having issues with my script that spaces out 512x512x512 cells across the entire map on startup. It only seems to work for terrain that gets added at some arbitrary point in development, near the beginning, and doesn’t factor in terrain that gets added in afterwards. It does factor in BaseParts that get added in whenever, though. I need to know if my method is flawed:

local function generateMapGrid()
	local cframe, size = workspace:GetBoundingBox()
	local cellsX, cellsY, cellsZ = math.ceil(size.X / OCTREE_SIZE), math.ceil(size.Y / OCTREE_SIZE), math.ceil(size.Z / OCTREE_SIZE)
	local cellBounds, origin = OCTREE_SIZE * Vector3.new(cellsX, cellsY, cellsZ), cframe.Position
	for x = 1, cellsX do
		for y = 1, cellsY do
			for z = 1, cellsZ do
				local position = origin + Vector3.new(x, y, z) * OCTREE_SIZE - cellBounds * 0.5 - Vector3.one * OCTREE_SIZE * 0.5
				table.insert(roots, position)
				debugRootNode(position)
			end
		end
	end
end

Without BaseParts:


With BaseParts:

Hello,

Is the terrain still loading while the script is running? Have you put a delay of like 20 or 30 seconds depending on our terrain map size to allow it to load?

If you move you spawn point to a place where it is having issues, does the issue go away from that position, because it is is loading it first? sooner? (Someone else can comment if terrain loads relative to where a player spawns, or does it start loading at 0 0 0 and the spawn place is irrelevant. I do not know the answer to terrain loading order)