How to detect if part is on screen (not position)

Camera:WorldToViewportPoint It returns a bool for whether or not it’s on screen or not. Next time, try looking at all the returns. Hope this helps!

That returned bool doesn’t help in this case, as it only returns true if the point is on screen, which, if you’ve read OP’s post, is their exact issue.

I’ve faced this issue too before, and I got around it by getting all the corners of the part, and then checking each one of those, but it does have it’s own flaws and cases where it’s not amazingly accurate.

Next time, read my question properly before giving an answer I clearly stated I tried

2 Likes

Then just put small parts in the corners and use the function on all of them.

That still is not reliable. There’s no guarantee the parts 8 corners will be visible at any one times. if the camera is zoomed in on a face, and no corners are visible then it’d still print false

Why don’t you create a part of size Vector3.new(0.1, 0.1, 0.1) (or smaller if necessary) and position it at the highlight?

That would have no affect on the result. The size of the part will have no affect on its position Vector. The issue is Roblox requires that specific vector to be on screen for it to work. I don’t always intend for the centre of a part to always be directly visible, especially when the part is incredibly large (hundreds of studs)

Oh, my bad, I thought you were trying to determine if a particular point on a part was visible as opposed to its center, not if the any point of the part is visible.

If you don’t want to perform any raycasting then you’ll either need to place tiny parts inside the part every ‘n’ studs, check if each is in the camera’s view, if one is then break then loop or get the camera’s position, the part’s position and size, the distance between the camera and the part and perform some math.

There is a scenario where the part is visible but neither those conditions are true.

When you are looking at the edge of the part.
image
A corner is beyond the screen’s left side, another beyond the right side, and another below, but nothing at the top.

Simply do this then
http://www.lighthouse3d.com/tutorials/view-frustum-culling/

What if we based whether or not the part is on screen on two different conditions:

Condition 1 - A vertex of the part is on the screen, in which case WorldToViewportPoint works just fine
Condition 2 - No vertices of the part are visible, but if we flatten every edge of the part to the plane of the camera, at least one of the edges intersects with the edges of our camera.

For both conditions, actually, we’ll need our vertices/corners of our part:

local part = workspace.MagicalPart
local cam = workspace.CurrentCamera

local vertices = {
	(part.CFrame * CFrame.new(part.Size.X/2, part.Size.Y/2, part.Size.Z/2)).Position, -- like doing part.Position + lookVector*(part.Size.Z/2) + ... and so on
	(part.CFrame * CFrame.new(part.Size.X/2, part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(part.Size.X/2, -part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(part.Size.X/2, -part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, -part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, -part.Size.Y/2, -part.Size.Z/2)).Position,
}

-- creating display parts
for i, vertex in pairs(vertices) do
	local displayPart = Instance.new("Part", workspace)
	displayPart.Anchored = true
	displayPart.Color = Color3.new(1, 0, 0)
	displayPart.Material = Enum.Material.Neon
	displayPart.CFrame = CFrame.new(vertex)
	displayPart.Size = Vector3.new(2, 2, 2)
end

That code gives us this:
image

Main problem with this method is it won’t work properly for meshes or other types of parts that aren’t rectangular prisms, but as long as you have some way of getting each individual vertex, the rest should still work, theoretically anyway.


For condition 1, all we need to do is flatten those vertices to the camera plane using Camera:WorldToViewportPoint, as mentioned before, and check if each individual vertex is visible in the camera:

game:GetService("RunService").Heartbeat:Connect(function()
	
	-- Flattening Vertices
	local flattenedVertices = {}
	for i, vertex in pairs(vertices) do
		local flattenedPoint = cam:WorldToViewportPoint(vertex)
		table.insert(flattenedVertices, flattenedPoint)
	end
	
	-- Checking Condition 1
	local condition1 = false
	for i, flattenedVertex in pairs(flattenedVertices) do
		local xFits = flattenedVertex.X <= cam.ViewportSize.X and flattenedVertex.X >= 0
		local yFits = flattenedVertex.Y <= cam.ViewportSize.Y and flattenedVertex.Y >= 0
		local zFits = flattenedVertex.Z >= 0 -- make sure the point isn't behind us
		if xFits and yFits and zFits then
			condition1 = true
			break
		end
	end
	print(condition1)
	
end)

And just like that, we have condition 1.


Now luckily, condition 2 will use the same flattened vertices we already got, so there’s no need to recaclulate those. For condition 2, however, we will need to pair the vertices in sets of 2 to act as the “edges” of our part and run some 2 dimensional line-line intersection math to determine whether or not they intersect with our camera’s edges.

My line-line intersection math will be pulled straight from this wikipedia article, as I’ve never learned this stuff professionally: Line–line intersection - Wikipedia

And here’s a function for detecting line-line intersection of two 2 dimensional lines:

-- Line Line Intersection
local function lineLineInt(v1: Vector3, v2: Vector3, v3: Vector3, v4: Vector3) -- v1-v2 is one line, v3-v4 is another line
	-- note we're using Vector3s for v1, v2, v3, and v4, but this is 2 dimensional
	-- WorldToviewportPoint just happens to returns Vector3s, so that's what I went with
	
	-- Variables
	local x1, y1 = v1.X, v1.Y
	local x2, y2 = v2.X, v2.Y
	local x3, y3 = v3.X, v3.Y
	local x4, y4 = v4.X, v4.Y
	
	-- Calculations
	local den = (x1 - x2) * (y3 - y4) - (y1 - y2) * (x3 - x4) -- both t and u have the same denominator, so let's just get that out of the way
	if den == 0 then
		return false -- avoid any div 0 errors
	end
	
	local t = ((x1 - x3) * (y3 - y4) - (y1 - y3) * (x3 - x4))/den
	local u = ((x1 - x3) * (y1 - y2) - (y1 - y3) * (x1 - x2))/den
	
	-- Final Answer
	local tBool = t <= 1 and t >= 0
	local uBool = u <= 1 and u >= 0
	
	return tBool and uBool -- if both are true then the lines intersect
	
end

Now then, big complicated math out of the way, let’s get into how we actually use it:

-- Checking Condition 2
local camTopLeft = Vector3.new(0, 0, 0)
local camTopRight = Vector3.new(cam.ViewportSize.X, 0, 0)
local camBottomLeft = Vector3.new(0, cam.ViewportSize.Y, 0)
local camBottomRight = Vector3.new(cam.ViewportSize.X, cam.ViewportSize.Y, 0)
	
local condition2 = false
local lastVertex = flattenedVertices[#flattenedVertices]
for i, flattenedVertex in pairs(flattenedVertices) do
	if flattenedVertex.Z < 0 or lastVertex.Z < 0 then -- make sure the points aren't behind us as they might cause a mis-trigger
		lastVertex = flattenedVertex -- updating the lastVertex variable to create pairs of vertices for the edges
		continue
	end
	
        -- line line intersection with each camera edge
	local topEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopLeft, camTopRight)
	local bottomEdgeInt = lineLineInt(lastVertex, flattenedVertex, camBottomLeft, camBottomRight)
	local leftEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopLeft, camBottomLeft)
	local rightEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopRight, camBottomRight)
	
	if topEdgeInt or bottomEdgeInt or leftEdgeInt or rightEdgeInt then
		condition2 = true -- if even one intersects then we know the part can be seen
		break -- the math will be most resource intensive when the part cannot be seen, but this should save some computing power
	end
	lastVertex = flattenedVertex -- updating the lastVertex variable to create pairs of vertices for the edges
end
	
-- Final Result
local partCanBeSeen = condition1 or condition2

Basically, we define the edges of our camera and for each edge we use our lineLineInt() function to determine whether or not that edge of the part intersects with any of our four camera edges.

The final script looks something like this:

local part = workspace.MagicalPart
local cam = workspace.CurrentCamera

local vertices = {
	(part.CFrame * CFrame.new(part.Size.X/2, part.Size.Y/2, part.Size.Z/2)).Position, -- like doing part.Position + lookVector*(part.Size.Z/2) + ... and so on
	(part.CFrame * CFrame.new(part.Size.X/2, part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(part.Size.X/2, -part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(part.Size.X/2, -part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, part.Size.Y/2, -part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, -part.Size.Y/2, part.Size.Z/2)).Position,
	(part.CFrame * CFrame.new(-part.Size.X/2, -part.Size.Y/2, -part.Size.Z/2)).Position,
}

-- creating display parts
for i, vertex in pairs(vertices) do
	local displayPart = Instance.new("Part", workspace)
	displayPart.Anchored = true
	displayPart.Color = Color3.new(1, 0, 0)
	displayPart.Material = Enum.Material.Neon
	displayPart.CFrame = CFrame.new(vertex)
	displayPart.Size = Vector3.new(2, 2, 2)
end

-- Line Line Intersection
local function lineLineInt(v1: Vector3, v2: Vector3, v3: Vector3, v4: Vector3) -- v1-v2 is one line, v3-v4 is another line
	-- note we're using Vector3s for v1, v2, v3, and v4, but this is 2 dimensional
	-- WorldToviewportPoint just happens to returns Vector3s, so that's what I went with
	
	-- Variables
	local x1, y1 = v1.X, v1.Y
	local x2, y2 = v2.X, v2.Y
	local x3, y3 = v3.X, v3.Y
	local x4, y4 = v4.X, v4.Y
	
	-- Calculations
	local den = (x1 - x2) * (y3 - y4) - (y1 - y2) * (x3 - x4) -- both t and u have the same denominator, so let's just get that out of the way
	if den == 0 then
		return false -- avoid any div 0 errors
	end
	
	local t = ((x1 - x3) * (y3 - y4) - (y1 - y3) * (x3 - x4))/den
	local u = ((x1 - x3) * (y1 - y2) - (y1 - y3) * (x1 - x2))/den
	
	-- Final Answer
	local tBool = t <= 1 and t >= 0
	local uBool = u <= 1 and u >= 0
	
	return tBool and uBool -- if both are true then the lines intersect
	
end

game:GetService("RunService").Heartbeat:Connect(function()
	
	-- Flattening Vertices
	local flattenedVertices = {}
	for i, vertex in pairs(vertices) do
		local flattenedPoint = cam:WorldToViewportPoint(vertex)
		table.insert(flattenedVertices, flattenedPoint)
	end
	
	-- Checking Condition 1
	local condition1 = false
	for i, flattenedVertex in pairs(flattenedVertices) do
		local xFits = flattenedVertex.X <= cam.ViewportSize.X and flattenedVertex.X >= 0
		local yFits = flattenedVertex.Y <= cam.ViewportSize.Y and flattenedVertex.Y >= 0
		local zFits = flattenedVertex.Z >= 0 -- make sure the point isn't behind us
		if xFits and yFits and zFits then
			condition1 = true
			break
		end
	end
	
	-- Checking Condition 2
	local camTopLeft = Vector3.new(0, 0, 0)
	local camTopRight = Vector3.new(cam.ViewportSize.X, 0, 0)
	local camBottomLeft = Vector3.new(0, cam.ViewportSize.Y, 0)
	local camBottomRight = Vector3.new(cam.ViewportSize.X, cam.ViewportSize.Y, 0)
	
	local condition2 = false
	local lastVertex = flattenedVertices[#flattenedVertices]
	for i, flattenedVertex in pairs(flattenedVertices) do
		if flattenedVertex.Z < 0 or lastVertex.Z < 0 then -- make sure the points aren't behind us as they might cause a mis-trigger
			lastVertex = flattenedVertex -- updating the lastVertex variable to create pairs of vertices for the edges
			continue
		end
		
		local topEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopLeft, camTopRight)
		local bottomEdgeInt = lineLineInt(lastVertex, flattenedVertex, camBottomLeft, camBottomRight)
		local leftEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopLeft, camBottomLeft)
		local rightEdgeInt = lineLineInt(lastVertex, flattenedVertex, camTopRight, camBottomRight)
		
		if topEdgeInt or bottomEdgeInt or leftEdgeInt or rightEdgeInt then
			condition2 = true -- if even one intersects then we know the part can be seen
			break -- the math will be most resource intensive when the part cannot be seen, but this should save some computing power
		end
		lastVertex = flattenedVertex -- updating the lastVertex variable to create pairs of vertices for the edges
	end
	
	-- Final Result
	local partCanBeSeen = condition1 or condition2
	
end)

Granted this may be a bit much for a heartbeat loop and I’m not sure how it is on performance. Nonetheless, I think it covers all scenarios for the part being visible.

I feel like this is way too complicated just to detect if a part is on screen, so there’s probably an easier way, but this is what I could come up with.

Edit: Here’s a gyazo of the result: link

7 Likes

What would happen if all the edges were offscreen, but the face they form is still visible? None would intersect the edges of the camera. You could probably get around this by doing a diagonal, but that would just create the same problem but smaller.

1 Like

Yeah that’s true. I didn’t think about that. Maybe there’s some 3rd condition we could check as well, but at that point it’d probably be laggy.

Great answer—your intersection code could instead be a convex polygon intersection check between the rectangle of the screen and each face of the object in screen space.

Or make a convex hull out of the 2D points of the part and then do the intersection check, but that would prevent you from extending it to custom concave meshes and things in the future and might be slower anyways.

Something like geometry - How do I determine if two convex polygons intersect? - Stack Overflow

Edit: Would be tricky to also cover the case of a part’s edge intersecting with the cameras culling frame, like going from the front to back. That would be a pretty niche corner case though.

1 Like

I’ve searched this question all over and there are still no solid answers. The vertices method seems to be close enough, but it’s a lot of work just to check if a part is on the screen.

Roblox should give us some built-in methods for this. Having the engine do the check would probably be much faster than trying to make a scripted method with Roblox Lua.

1 Like

Hi, i made a module for that and i think it might help you.
See more information about it in this post:
DevForum post

Was it useful
  • Yes.
  • A little bit.
  • No.

0 voters

Damn that’s an awesome you should make a tutorial on the devforum if you haven’t already, but I have 1 issue, whenever i keep the part on and off the screen and keep looking around my game gets very laggy, im using 5 parts and getting its vertices constantly, is there any way to reduce the lag without getting rid of some parts (that will be my plan B if i can’t get it to work with 5 parts)

1 Like

Yeah my method was a bit over the top and complex so it causes a bit of lag. I think the alternative that nicemike40 suggested with using convex polygon intersection detection would work better, and maybe even account for the situation that Pokemoncraft5290 pointed out.

First step is to be able to essentially create a polygon out of the part by projecting it’s vertices onto the camera. Before that though, we need to actually find the vertices:

-- Variables
local part = workspace.THEPart
local camera = workspace.CurrentCamera

-- Delay
task.wait(2)

-- Finding Vertices of Part
local vertices = {}
for i1, x in pairs({-0.5, 0.5}) do
	for i2, y in pairs({-0.5, 0.5}) do
		for i3, z in pairs({-0.5, 0.5}) do
			local vertex = (part.CFrame * CFrame.new(x * part.Size.X, y * part.Size.Y, z * part.Size.Z)).Position
			table.insert(vertices, vertex)
		end
	end
end

-- Displaying Vertices
local function displayVertices(vertices)
	for i, vertex in pairs(vertices) do
		local display = Instance.new("Part")
		display.CFrame = CFrame.new(vertex)
		display.Anchored = true
		display.Size = Vector3.new(1, 1, 1)
		display.Color = Color3.new(1, 0, 0)
		display.Material = Enum.Material.Neon
		display.Parent = workspace
	end
end
displayVertices(vertices)

And that should work nicely even if the part is rotated, since we did it with CFrames (or alternatively you could use the .LookVector, .RightVector, and .UpVector properties of the part).
image

Now that we have our vertices, let’s create a polygon out of them. In order to do that, we need to first project each vertex to the camera, using camera:WorldToScreenPoint().

-- Displaying Camera Points
local screenGui = Instance.new("ScreenGui", playerGui); screenGui.Name = "DisplayCameraPointsGui"
local function displayCameraPoints(points)
	
	-- Displaying Points
	for i, point in pairs(points) do
		local frame = Instance.new("Frame")
		frame.Size = UDim2.fromOffset(10, 10)
		frame.AnchorPoint = Vector2.new(0.5, 0.5)
		frame.BackgroundColor3 = Color3.new(1, 0, 0)
		frame.Position = UDim2.fromOffset(point.X, point.Y)
		frame.Parent = screenGui
	end
	
end

-- Turning Vertex List Into Polygon
local function createPolygon(vertices)
	
	-- Finding Camera Points
	local cameraPoints = {}
	for i, vertex in pairs(vertices) do
		local point, onScreen = camera:WorldToScreenPoint(vertex)
		if point.Z >= 0 then -- don't include points that are behind the camera
			table.insert(cameraPoints, Vector2.new(point.X, point.Y)) -- remove the z
		end
	end
	
	-- Displaying Camera Points
	displayCameraPoints(cameraPoints)
	
end

createPolygon(vertices)

And thus we get something:
image

Obviously we’ll have to do this on a loop but first we’ll just create a polygon with which we can reference the individual edges of. In order to do this, we will find the “convex hull” of our list of points.

Since this is something I’ve admittedly never done before and I just barely found out about this by looking it up, I will be following the method described in this video: Graham Scan Tutorial: Convex Hull of a Set of 2D Points - YouTube

The first step as mentioned in the video is to find the lowest point and order them based on the angle from that lowest point. In order to visualize the order of the points, I changed the displayCameraPoints() function to color them based on the order:

-- Displaying Camera Points
local screenGui = Instance.new("ScreenGui", playerGui); screenGui.Name = "DisplayCameraPointsGui"
local function displayCameraPoints(points)
	
	-- Displaying Points
	for i, point in pairs(points) do
		local frame = Instance.new("Frame")
		frame.Size = UDim2.fromOffset(10, 10)
		frame.AnchorPoint = Vector2.new(0.5, 0.5)
		frame.BackgroundColor3 = Color3.fromHSV(i / (#points * 4), 1, 1) -- to visualize the order of the points
		frame.Position = UDim2.fromOffset(point.X, point.Y)
		frame.Parent = screenGui
	end
	
end

And then in the createPolygon() function I orded the points as instructed in the video I linked and then moved the displayCameraPoints() function call underneath the ordering:

-- Turning Vertex List Into Polygon
local function createPolygon(vertices)
	
	-- Finding Camera Points
	local cameraPoints = {}
	for i, vertex in pairs(vertices) do
		local point, onScreen = camera:WorldToScreenPoint(vertex)
		if point.Z >= 0 then -- don't include points that are behind the camera
			table.insert(cameraPoints, Vector2.new(point.X, point.Y)) -- remove the z
		end
	end
	
	-- Ordering Points
	local startingPoint = cameraPoints[1]
	for i, point in pairs(cameraPoints) do
		if point.Y > startingPoint.Y or (point.Y == startingPoint.Y and point.X > startingPoint.X) then -- remember the position goes down as y goes up
			startingPoint = point
		end
	end
	
	table.sort(cameraPoints, function(a, b)
		if a == startingPoint then
			return true
		elseif b == startingPoint then
			return false
		end
		
		-- Finding Rise and Run of A and B
		local runA, runB = a.X - startingPoint.X, b.X - startingPoint.X
		local riseA, riseB = a.Y - startingPoint.Y, b.Y - startingPoint.Y
		
		-- Finding Negative Reciprocals
		local recipA = if riseA ~= 0 then -runA / riseA else -math.huge
		local recipB = if riseB ~= 0 then -runB / riseB else -math.huge
		
		return recipA > recipB
		
	end)
	
	-- Displaying Camera Points
	displayCameraPoints(cameraPoints)
	print(cameraPoints)
	
end

createPolygon(vertices)

And since this is what I get:
image

I think it worked, so let’s move on. Next step is to loop through these ordered points and essentially get rid of all of the inside points. First we’ll create two tables, “convexHullPoints”, which will store the points for the new polygon, and “recentLeftTurns”, which will store the indices of each point that marks a turn left, so that way we can keep track of them in order to undo them if the next turn goes right.

After that, we loop through all the points, check if each turn is left or right using the determinant (calculation which I just found online) and if the turn is left, we add the index of the third point in the turn and move on. If the turn is right, we remove the amount of points from convexHullPoints equal to the amount of left turns, and then re-add the new point as it would’ve been removed. If the determinant is zero, we remove the point just before this one, since it would be creating a straight line anyway, but I don’t think the determinant would really ever be zero. Anywhere here’s the code for it:

-- Turning Vertex List Into Polygon
local function createPolygon(vertices)
	
	-- Finding Camera Points
	local cameraPoints = {}
	for i, vertex in pairs(vertices) do
		local point, onScreen = camera:WorldToScreenPoint(vertex)
		if point.Z >= 0 then -- don't include points that are behind the camera
			table.insert(cameraPoints, Vector2.new(point.X, point.Y)) -- remove the z
		end
	end
	
	-- Ordering Points
	local startingPoint = cameraPoints[1]
	for i, point in pairs(cameraPoints) do
		if point.Y > startingPoint.Y or (point.Y == startingPoint.Y and point.X > startingPoint.X) then -- remember the position goes down as y goes up
			startingPoint = point
		end
	end
	
	table.sort(cameraPoints, function(a, b)
		if a == startingPoint then
			return true
		elseif b == startingPoint then
			return false
		end
		
		-- Finding Rise and Run of A and B
		local runA, runB = a.X - startingPoint.X, b.X - startingPoint.X
		local riseA, riseB = a.Y - startingPoint.Y, b.Y - startingPoint.Y
		
		-- Finding Negative Reciprocals
		local recipA = if riseA ~= 0 then -runA / riseA else -math.huge
		local recipB = if riseB ~= 0 then -runB / riseB else -math.huge
		
		return recipA > recipB
		
	end)
	
	-- Creating Convex Hull Polygon
	local convexHullPoints = {}
	local recentLeftTurns = {}
	
	for i, point in pairs(cameraPoints) do
		
		-- Adding Point
		table.insert(convexHullPoints, point)
		
		-- Checking Last Two Points
		if #convexHullPoints >= 3 then
			
			-- Finding Points
			local point1 = convexHullPoints[#convexHullPoints - 2]
			local point2 = convexHullPoints[#convexHullPoints - 1]
			local point3 = point
			
			-- Checking Direction of Turn
			local det = (point1.X*point2.Y + point2.X*point3.Y + point3.X*point1.Y) - (point2.Y*point3.X + point3.Y*point1.X + point1.Y*point2.X) -- determinant shortcut
			if det < 0 then
				print('left turn')
				table.insert(recentLeftTurns, #convexHullPoints)
			elseif det > 0 then
				print('right turn')
				
				-- Removing All Consecutive Left Turns Before
				print(recentLeftTurns)
				for i = 1, math.max(#recentLeftTurns, 2) do
					table.remove(convexHullPoints)
				end
				table.insert(convexHullPoints, point) -- re-adding new point
				recentLeftTurns = {}
				
			elseif det == 0 then
				table.remove(convexHullPoints, (#convexHullPoints - 1))
			end
			
		end
		
	end
	
	-- Returning Polygon
	return convexHullPoints
	
end

local polygon = createPolygon(vertices)
displayCameraPoints(polygon)

And now you can see we’ve removed the inner points and essentially have our polygon.
image
Now we just need to detect the actual collision between this polygon and the screen itself . . .
But before that let’s define a similar list of vertices for the camera as a polygon. This will just consist of all of the corners of the screen, from the bottom right around counter-clockwise.

-- Creating Polygons
local partPolygon = createPolygon(vertices)
local camPolygon = {
	Vector2.new(screenGui.AbsoluteSize.X, screenGui.AbsoluteSize.Y),
	Vector2.new(screenGui.AbsoluteSize.X, 0),
	Vector2.new(0, 0),
	Vector2.new(0, screenGui.AbsoluteSize.Y)
}

And if you run the same displayCameraPoints() function on the camPolygon table, then you can see the points line around the corners of the screen, not including the gui inset.

And now we’re ready for the actual detection part. You can also put this in a loop now. Here’s a few changes I made:

I added a remove time to the displayCameraPoints() function, so we can display the points in a loop without keeping the old points:

-- Displaying Camera Points
local screenGui = Instance.new("ScreenGui", playerGui); screenGui.Name = "DisplayCameraPointsGui"
local function displayCameraPoints(points, timeDelay)
	
	-- Displaying Points
	for i, point in pairs(points) do
		local frame = Instance.new("Frame")
		frame.Size = UDim2.fromOffset(10, 10)
		frame.AnchorPoint = Vector2.new(0.5, 0.5)
		frame.BackgroundColor3 = Color3.fromHSV(i / (#points * 4), 1, 1) -- to visualize the order of the points
		frame.Position = UDim2.fromOffset(point.X, point.Y)
		frame.Parent = screenGui
		
		if timeDelay then
			task.delay(timeDelay, function()
				frame:Destroy()
			end)
		end
	end
	
end

And then I created a main loop that we’ll use to organize the actual functionality:

-- Main Loop
while true do
	task.wait()
	
	-- Creating Polygons
	local partPolygon = createPolygon(vertices)
	local camPolygon = {
		Vector2.new(screenGui.AbsoluteSize.X, screenGui.AbsoluteSize.Y),
		Vector2.new(screenGui.AbsoluteSize.X, 0),
		Vector2.new(0, 0),
		Vector2.new(0, screenGui.AbsoluteSize.Y)
	}
	displayCameraPoints(partPolygon, 0)
	
end

I decided to play around and see what it’d look like if the screen were black and I could see the points on top, so here’s that: https://gyazo.com/cf0024c70ae1b407a798cd40ccce29cf

But so far, little to no lag.

The next step is detecting whether or not the polygons are intersecting. I will be following this article: Collision Detection Using the Separating Axis Theorem

Surprisingly this step was a bit easier after I got it figured out. I created a function to detect the intersection between the two polygons. If either polygon has 2 points or less (in the case that the part is behind the camera, or partially so), then obviously we can’t really do much with that so we just return false. I then looped through every point in poly1, keeping track of the last point before it, and found the normal, or the outward direction perpendicular to the edge.

I then went through all of the points of both polygons and projected them to this normal line using their dot product. It’s sort of like creating a shadow onto the axis that is the normal direction. Afterwards, I found the minimum and maximum dot products for both polygons and checked if the “shadows” overlapped. If they didn’t I immediately returned false, and if every one of the shadows overlapped one another on every normal axis, I returned true at the end. On top of this, I ran the function both ways, first with the part being poly1 and the camera being poly2, and then again with them swapped, so as to check the normals of both polygons.

-- Detecting Intersection
local function detectIntersection(poly1, poly2)
	if #poly1 < 3 or #poly2 < 3 then return false end
	
	-- Looping Normals of First Polygon
	local lastPoint = poly1[#poly1]
	for i, point in pairs(poly1) do
		
		-- Variables
		local edgeDir = (point - lastPoint).Unit
		local normal = Vector2.new(edgeDir.Y, -edgeDir.X)
		
		-- Projecting Polygon Points
		local projected1 = {}
		local projected2 = {}
		
		for i2, point in pairs(poly1) do
			local projectedPoint = point:Dot(normal) -- this is like casting a shadow onto the axis that the "normal" creates
			table.insert(projected1, projectedPoint)
		end
		
		for i2, point in pairs(poly2) do
			local projectedPoint = point:Dot(normal)
			table.insert(projected2, projectedPoint)
		end
		
		-- Finding Minimums and Maximums
		local min1, max1 = projected1[1], projected1[1]
		local min2, max2 = projected2[1], projected2[1]
		for i2, dot in pairs(projected1) do
			min1, max1 = math.min(dot, min1), math.max(dot, max1)
		end
		for i2, dot in pairs(projected2) do
			min2, max2 = math.min(dot, min2), math.max(dot, max2)
		end
		
		-- Checking Overlap
		if max1 < min2 or max2 < min1 then
			return false -- if any of the normal projections don't overlap then the polygons don't either
		end
		
		-- Setting Last Point
		lastPoint = point
		
	end
	
	-- Returning
	return true -- only returns if all normal projections go through as true
	
end

-- Main Loop
while true do
	task.wait(0.1)
	
	-- Creating Polygons
	local partPolygon = createPolygon(vertices)
	local camPolygon = {
		Vector2.new(screenGui.AbsoluteSize.X, screenGui.AbsoluteSize.Y),
		Vector2.new(screenGui.AbsoluteSize.X, 0),
		Vector2.new(0, 0),
		Vector2.new(0, screenGui.AbsoluteSize.Y)
	}
	--displayCameraPoints(partPolygon, 0)
	
	-- Detecting Intersection
	local doesIntersect1 = detectIntersection(partPolygon, camPolygon)
	local doesIntersect2 = detectIntersection(camPolygon, partPolygon)
	
	local doesIntersect = doesIntersect1 and doesIntersect2
	print(doesIntersect)
	
end

I’m not sure how it compares lag-wise, but it seems to work in pretty much 100% of cases, so there’s that as an improvement at the very least.

Here’s a demonstration: https://gyazo.com/58cb135dd901fb11fcc4b13cb281a81f
Here’s proof that it works even in Pokemoncraft5290’s situation: https://gyazo.com/82a4280d5696c3bc4154c14c610b3974

Here’s the final, ultimately beefy, script:

-- Player Variables
local player = game.Players.LocalPlayer
local playerGui = player.PlayerGui

-- Variables
local part = workspace.THEPart
local camera = workspace.CurrentCamera

-- Delay
task.wait(2)

-- Finding Vertices of Part
local vertices = {}
for i1, x in pairs({-0.5, 0.5}) do
	for i2, y in pairs({-0.5, 0.5}) do
		for i3, z in pairs({-0.5, 0.5}) do
			local vertex = (part.CFrame * CFrame.new(x * part.Size.X, y * part.Size.Y, z * part.Size.Z)).Position
			table.insert(vertices, vertex)
		end
	end
end

-- Displaying Vertices
local function displayVertices(vertices)
	for i, vertex in pairs(vertices) do
		local display = Instance.new("Part")
		display.CFrame = CFrame.new(vertex)
		display.Anchored = true
		display.Size = Vector3.new(1, 1, 1)
		display.Color = Color3.new(1, 0, 0)
		display.Material = Enum.Material.Neon
		display.Parent = workspace
	end
end
--displayVertices(vertices)

-- Displaying Camera Points
local screenGui = Instance.new("ScreenGui", playerGui); screenGui.Name = "DisplayCameraPointsGui"
local function displayCameraPoints(points, timeDelay)
	
	-- Displaying Points
	for i, point in pairs(points) do
		local frame = Instance.new("Frame")
		frame.Size = UDim2.fromOffset(10, 10)
		frame.AnchorPoint = Vector2.new(0.5, 0.5)
		frame.BackgroundColor3 = Color3.fromHSV(i / (#points * 4), 1, 1) -- to visualize the order of the points
		frame.Position = UDim2.fromOffset(point.X, point.Y)
		frame.Parent = screenGui
		
		if timeDelay then
			task.delay(timeDelay, function()
				frame:Destroy()
			end)
		end
	end
	
end

-- Turning Vertex List Into Polygon
local function createPolygon(vertices)
	
	-- Finding Camera Points
	local cameraPoints = {}
	for i, vertex in pairs(vertices) do
		local point, onScreen = camera:WorldToScreenPoint(vertex)
		if point.Z >= 0 then -- don't include points that are behind the camera
			table.insert(cameraPoints, Vector2.new(point.X, point.Y)) -- remove the z
		end
	end
	
	-- Ordering Points
	local startingPoint = cameraPoints[1]
	for i, point in pairs(cameraPoints) do
		if point.Y > startingPoint.Y or (point.Y == startingPoint.Y and point.X > startingPoint.X) then -- remember the position goes down as y goes up
			startingPoint = point
		end
	end
	
	table.sort(cameraPoints, function(a, b)
		if a == startingPoint then
			return true
		elseif b == startingPoint then
			return false
		end
		
		-- Finding Rise and Run of A and B
		local runA, runB = a.X - startingPoint.X, b.X - startingPoint.X
		local riseA, riseB = a.Y - startingPoint.Y, b.Y - startingPoint.Y
		
		-- Finding Negative Reciprocals
		local recipA = if riseA ~= 0 then -runA / riseA else -math.huge
		local recipB = if riseB ~= 0 then -runB / riseB else -math.huge
		
		return recipA > recipB
		
	end)
	
	-- Creating Convex Hull Polygon
	local convexHullPoints = {}
	local recentLeftTurns = {}
	
	for i, point in pairs(cameraPoints) do
		
		-- Adding Point
		table.insert(convexHullPoints, point)
		
		-- Checking Last Two Points
		if #convexHullPoints >= 3 then
			
			-- Finding Points
			local point1 = convexHullPoints[#convexHullPoints - 2]
			local point2 = convexHullPoints[#convexHullPoints - 1]
			local point3 = point
			
			-- Checking Direction of Turn
			local det = (point1.X*point2.Y + point2.X*point3.Y + point3.X*point1.Y) - (point2.Y*point3.X + point3.Y*point1.X + point1.Y*point2.X) -- determinant shortcut
			if det < 0 then
				table.insert(recentLeftTurns, #convexHullPoints)
			elseif det > 0 then
				
				-- Removing All Consecutive Left Turns Before
				for i = 1, math.max(#recentLeftTurns, 2) do
					table.remove(convexHullPoints)
				end
				table.insert(convexHullPoints, point) -- re-adding new point
				recentLeftTurns = {}
				
			elseif det == 0 then
				table.remove(convexHullPoints, (#convexHullPoints - 1))
			end
			
		end
		
	end
	
	-- Returning Polygon
	return convexHullPoints
	
end

-- Detecting Intersection
local function detectIntersection(poly1, poly2)
	if #poly1 < 3 or #poly2 < 3 then return false end
	
	-- Looping Normals of First Polygon
	local lastPoint = poly1[#poly1]
	for i, point in pairs(poly1) do
		
		-- Variables
		local edgeDir = (point - lastPoint).Unit
		local normal = Vector2.new(edgeDir.Y, -edgeDir.X)
		
		-- Projecting Polygon Points
		local projected1 = {}
		local projected2 = {}
		
		for i2, point in pairs(poly1) do
			local projectedPoint = point:Dot(normal) -- this is like casting a shadow onto the axis that the "normal" creates
			table.insert(projected1, projectedPoint)
		end
		
		for i2, point in pairs(poly2) do
			local projectedPoint = point:Dot(normal)
			table.insert(projected2, projectedPoint)
		end
		
		-- Finding Minimums and Maximums
		local min1, max1 = projected1[1], projected1[1]
		local min2, max2 = projected2[1], projected2[1]
		for i2, dot in pairs(projected1) do
			min1, max1 = math.min(dot, min1), math.max(dot, max1)
		end
		for i2, dot in pairs(projected2) do
			min2, max2 = math.min(dot, min2), math.max(dot, max2)
		end
		
		-- Checking Overlap
		if max1 < min2 or max2 < min1 then
			return false -- if any of the normal projections don't overlap then the polygons don't either
		end
		
		-- Setting Last Point
		lastPoint = point
		
	end
	
	-- Returning
	return true -- only returns if all normal projections go through as true
	
end

-- Main Loop
while true do
	task.wait(0.1)
	
	-- Creating Polygons
	local partPolygon = createPolygon(vertices)
	local camPolygon = {
		Vector2.new(screenGui.AbsoluteSize.X, screenGui.AbsoluteSize.Y),
		Vector2.new(screenGui.AbsoluteSize.X, 0),
		Vector2.new(0, 0),
		Vector2.new(0, screenGui.AbsoluteSize.Y)
	}
	--displayCameraPoints(partPolygon, 0)
	
	-- Detecting Intersection
	local doesIntersect1 = detectIntersection(partPolygon, camPolygon)
	local doesIntersect2 = detectIntersection(camPolygon, partPolygon)
	
	local doesIntersect = doesIntersect1 and doesIntersect2
	print(doesIntersect)
	
end

And here’s the place file:
part on screen test.rbxl (44.2 KB)

In Summary

I don't think I fixed the lag or complexity issue, but at least it seems to cover all cases. It's something to try nonetheless. Maybe it was somehow optimized to be more efficient with this method as opposed to my last method.

Also a small note, in this attempt I used :WorldToScreenPoint() instead of :WorldToViewportPoint(). There was no real reason for this, I just unintentionally swapped one in place for the other. As far as I can tell the main difference is whether or not it ignores the gui inset on the top of the screen.
7 Likes

You should make this a post in Community Tutorials since that’s a lot of effort just to reply to someone, and it’s also an important topic

3 Likes

I created one here now: How to detect if a part is on the screen with a local script