Detect part visibility on screen

Hey DevForum!

What’s the best way to detect if a part is visible on screen?

I made this system, which isn’t bad but doesn’t work as well as rays. However, rays aren’t really good for performance in my case since I need lots of rendering at once.

My current code:

function IsPartVisible(Part)
	local Size = Part.Size / 2
	local Points = {}
	for X = -1, 1, Accuracy do
		for Y = -1, 1, Accuracy do
			for Z = -1, 1, Accuracy do
				local Point = Part.CFrame:PointToWorldSpace(Vector3.new(Size.X * X, Size.Y * Y, Size.Z * Z))
				table.insert(Points, Point)
			end
		end
	end
	for _, Point in pairs(Points) do
		local ScreenPoint, Visible = Camera:WorldToScreenPoint(Point)
		if Visible then
			return true
		end
	end
	return false
end

Help ASAP!
I would appreciete any help!

So do you mean the part is visible on the screen, or the player’s mouse is on the part?

If part is visible anywhere on screen.

Ah, ok. I think I’ve seen some tutorials on Youtube, I’ll try look for one. Off of my mind, I don’t know how to do this, I think it’s related to the LookVector but I’m probably wrong. I’ll reply if I find one.

I tried YouTube, but most of them are just RayCasting tutorials which aren’t ideal for me. Thanks!

Okay, I’ve just come across this DevForum post you might want to look at: How would I detect if the player can see an object?

Here is the code by Proligant too:

local _, withinScreenBounds = Camera:WorldToScreenPoint(part.Position)

if withinScreenBounds then
   -- We have successfully detected if the player can see the part.
end

I currently use a modified version of this system.

The issue with the one you sent is that it only detects the center of an object, making it difficult to use with larger objects.

How about this: Check if an object is in the screen

Nope. This is similar to RayCast. returns a value based on if part is covered by another object

I don’t think there are any other ways. I’m not 100% sure though.

you can test if a part is on screen, without using rays, if both the front-left-top bounds position and the bottom-right-back bounds position is not on screen.

you might aswell also check if the center point is on screen.
get the bounds of your part, and boom:

--i didn't test this with parts, only in an open field, and i didn't test this 
--thoroughly so there might be edge cases that exist, like obscuring parts 
--but i commented how to cover that if you need to
local cam = workspace.CurrentCamera
local lel = workspace:WaitForChild("lel")

local minBound = lel.Position - (lel.Size/2)
local maxBound = lel.Position + (lel.Size/2)
	
local _, minRes = cam:WorldToScreenPoint(minBound)
local _, maxRes = cam:WorldToScreenPoint(maxBound)
local _, midRes = cam:WorldToScreenPoint(lel.Position)
--local obscuredParts = cam:GetPartsObscuringTarget({lel.Position, minBound, maxBound})
	
if minRes or maxRes or midRes then --and #obscuredParts <= 0 
	print("Is on screen!")
else
	print("Is not on screen!")
end

edit: this assumes your part is rectangular or (like a cylinder) has a rectangular hitbox.
this might behave differently on unions, spheres, 3d models which vaguely represent a rectangle etc.

edit: spelling
edit: added more casting points for extra accuracy.

edit: edge case that I found is if the part is this huge that you’re staring at it but all of the bounds + center are off screen because it is so big, but I’m assuming you don’t have really big parts.

I actually answered a very similar question yesterday.

At minimum, you would need to perform a Frustum-OBB intersection test. You can learn more about this from the following links: viewing frustum, OBB, separating axis theorem and a good starting resource relating to intersection tests here.


Take a look at the thread I mentioned here - there’s a video demoing it - or to try it yourself, see the following:

Example File

FrustumExamples.rbxm (10.0 KB)

Example Code

Frustum debug tests
local RunService = game:GetService('RunService')
local packages = game:GetService('ServerStorage').mono.packages.shared

local Maid = require(packages.utils.maid)

-- indices that define the edges of the frustum
-- so that we can render it through a gizmo
local FRUSTUM_EDGES = {
  -- right plane
  1, 5,   4, 8,

  -- left plane
  2, 6,   3, 7,

  -- near plane
  1, 2,   2, 3,
  3, 4,   4, 1,

  -- far plane
  5, 6,   6, 7,
  7, 8,   8, 5,
}

local function getBoundingBox(obj)
  -- safely computes the bounding box for the given param
  -- expects either (a) a BasePart or (b) a Model
  -- otherwise defaults to identity matrix + zero vec
  if typeof(obj) == 'Instance' then
    if obj:IsA('BasePart') then
      return obj.CFrame, obj.Size
    elseif obj:IsA('Model') then
      return obj:GetBoundingBox()
    end
  end

  return CFrame.identity, Vector3.zero
end

local function planeFromPoints(a, b, c, output)
  -- computes a plane from 3 points
  --    learn more about planes here, e.g. ref:
  --     - https://gdbooks.gitbooks.io/3dcollisions/content/Chapter1/plane.html
  --     - https://en.wikipedia.org/wiki/Plane_(mathematics)
  --
  local normal = (b - a):Cross(c - a).Unit
  local distance = -normal:Dot(a)

  if not output then
    output = table.create(5)
  end

  output[1] = normal
  output[2] = distance
  output[3] = normal.X
  output[4] = normal.Y
  output[5] = normal.Z

  return output
end

local function buildFrustumFromCamera(camera, farPlaneZ, output)
  -- builds a frustum from a Camera object instance
  --
  -- NOTE:
  --   far plane is user-defined since Roblox doesn't allow us to read it
  --   from their camera; but that's fine because you can use the far plane
  --   to adjust what you would prefer the render distance to be. Though,
  --   in this case, i just default to 200.
  --
  --   Learn more about viewing frustum e.g. ref @ https://en.wikipedia.org/wiki/Viewing_frustum
  --
  farPlaneZ = farPlaneZ or 200

  local transform = camera.CFrame
  local translation = transform.Position
  local nearPlaneZ = camera.NearPlaneZ

  local fieldOfView = camera.FieldOfView
  local tanHalfFov = math.tan(math.rad(fieldOfView * 0.5))

  local viewportSize = camera.ViewportSize
  local viewportAspectRatio = viewportSize.X / viewportSize.Y

  local farPlaneHeight_2 = tanHalfFov*farPlaneZ
  local farPlaneWidth_2 = farPlaneHeight_2*viewportAspectRatio

  local nearPlaneHeight_2 = tanHalfFov*-nearPlaneZ
  local nearPlaneWidth_2 = nearPlaneHeight_2*viewportAspectRatio

  farPlaneZ *= -1

  local farTopLeft =      transform * Vector3.new( -farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
  local farTopRight =     transform * Vector3.new(  farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
  local farBottomLeft =   transform * Vector3.new( -farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)
  local farBottomRight =  transform * Vector3.new(  farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)

  local nearTopLeft =     transform * Vector3.new(-nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
  local nearTopRight =    transform * Vector3.new( nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
  local nearBottomLeft =  transform * Vector3.new(-nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)
  local nearBottomRight = transform * Vector3.new( nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)

  output = output or { }
  output.transform = transform
  output.translation = translation
  output.viewportSize = viewportSize

  local frustumPoints = output.points
  if not frustumPoints then
    frustumPoints = table.create(8)
    output.points = frustumPoints
  end
  frustumPoints[1] = nearTopRight
  frustumPoints[2] = nearTopLeft
  frustumPoints[3] = nearBottomLeft
  frustumPoints[4] = nearBottomRight
  frustumPoints[5] = farTopRight
  frustumPoints[6] = farTopLeft
  frustumPoints[7] = farBottomLeft
  frustumPoints[8] = farBottomRight

  local frustumPlanes = output.planes
  if not frustumPlanes then
    frustumPlanes = table.create(6)
    output.planes = frustumPlanes
  end
  frustumPlanes[1] = planeFromPoints(   nearTopRight, nearBottomRight,     nearTopLeft, frustumPlanes[1]) -- near
  frustumPlanes[2] = planeFromPoints(    farTopRight,      farTopLeft,  farBottomRight, frustumPlanes[2]) -- far
  frustumPlanes[3] = planeFromPoints(   nearTopRight,     nearTopLeft,     farTopRight, frustumPlanes[3]) -- top
  frustumPlanes[4] = planeFromPoints(nearBottomRight,  farBottomRight,  nearBottomLeft, frustumPlanes[4]) -- bottom
  frustumPlanes[5] = planeFromPoints(    nearTopLeft,  nearBottomLeft,      farTopLeft, frustumPlanes[5]) -- left
  frustumPlanes[6] = planeFromPoints(   nearTopRight,     farTopRight, nearBottomRight, frustumPlanes[6]) -- right

  return output
end

local function testFrustumSphere(frustum, spherePosition, sphereRadius, padding)
  -- tests whether the frustum intersects a sphere
  padding = padding or 0

  local planes = frustum.planes
  local x, y, z = spherePosition.X, spherePosition.Y, spherePosition.Z

  local nx, ny, nz, d, test
  test = -sphereRadius - padding
  for i = 1, 6, 1 do
    local plane = planes[i]
    d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

    local m = nx*x + ny*y + nz*z + d
    if m < test then
      return false
    end
  end

  return true
end

local function testFrustumOBB(frustum, obbOrigin, obbExtents, padding)
  -- tests whether the frustum intersects with an OBB
  --
  -- TODO for you:
  --   do a less computationally expensive test first to check
  --   if it's even possible for these to be intersecting
  --
  local planes = frustum.planes
  padding = padding or 0

  -- m11, m12, m13, x
  -- m21, m22, m23, y
  -- m31, m32, m33, z
  --   0,   0,   0, 1
  local sx, sy, sz = obbExtents.X, obbExtents.Y, obbExtents.Z
  local  x,  y,  z,
        rx, ux, bx,
        ry, uy, by,
        rz, uz, bz = obbOrigin:GetComponents()

  local s,  r,  u,  b
  local d, nx, ny, nz
  for i = 1, 6, 1 do
    local plane = planes[i]
    d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

    s = x*nx + y*ny + z*nz
    r = sx*math.abs(nx*rx + ny*ry + nz*rz)
    u = sy*math.abs(nx*ux + ny*uy + nz*uz)
    b = sz*math.abs(nx*bx + ny*by + nz*bz)

    if (s + r + u + b) < -d - padding then
      return false
    end
  end

  return true
end

local function computeFrustumBox(frustum, output)
  -- compute the frustum's OBB
  --    can be used to compute the AABB which you could use
  --    for some kind of broad-phase search. a method to create
  --    an aabb from an obb can be found here:
  --      - Ref @ https://zeux.io/2010/10/17/aabb-from-obb-with-component-wise-abs/
  --
  --    though, i should note that you can get the AABB much faster by min-maxing the transform + the far plane points
  --

  local points = frustum.points
  local transform = frustum.transform
  local translation = frustum.translation

  local x, y, z
  local xmin, xmax = math.huge, -math.huge
  local ymin, ymax = math.huge, -math.huge
  local zmin, zmax = math.huge, -math.huge
  for i = 0, 4, 1 do
    local point
    if i == 0 then
      point = translation
    else
      point = points[i + 4]
    end

    point = transform:PointToObjectSpace(point)
    x, y, z = point.X, point.Y, point.Z

    xmin = math.min(xmin, x)
    xmax = math.max(xmax, x)
    ymin = math.min(ymin, y)
    ymax = math.max(ymax, y)
    zmin = math.min(zmin, z)
    zmax = math.max(zmax, z)
  end

  local offset = Vector3.new(
    0.5*(xmin + xmax),
    0.5*(ymin + ymax),
    0.5*(zmin + zmax)
  )

  local extents = Vector3.new(xmax - xmin, ymax - ymin, zmax - zmin)
  local minBounds = Vector3.new(xmin, ymin, zmin)
  local maxBounds = Vector3.new(xmax, ymax, zmax)

  local rotation = transform.Rotation
  output = output or { }
  output.origin = transform + rotation:PointToWorldSpace(offset)
  output.extents = extents

  local bounds = output.bounds
  if not bounds then
    bounds = table.create(2)
    output.bounds = bounds
  end
  bounds[1] = minBounds
  bounds[2] = maxBounds

  return output
end

local function drawBox(box, part, cleanup)
  -- draws an OBB gizmo
  if not part then
    part = Instance.new('Part')
    part.Name = '__FrustumBox'
    part.Shape = Enum.PartType.Block
    part.Anchored = true
    part.CanQuery = false
    part.CanTouch = false
    part.CanCollide = false
    part.CastShadow = false
    part.BrickColor = BrickColor.Red()
    part.TopSurface = 0
    part.BottomSurface = 0
    part.Transparency = 0.9
    part.Parent = workspace

    if cleanup then
      cleanup(part)
    end
  end

  part.CFrame = box.origin
  part.Size = box.extents

  return part
end

local function drawEdges(points, indices, lines, cleanup)
  -- draws a frustum given it's points and the indices
  -- that make up its edges
  lines = lines or { }

  local vertCount = #indices
  local edgeCount = math.floor(vertCount*0.5)
  local cornerCount = #points

  local key, index, maxAlpha
  for i = 0, edgeCount - 1, 1 do
    key = i + 1
    index = i*2 + 1

    local p0 = points[indices[index + 0]]
    local p1 = points[indices[index + 1]]

    local line = lines[key]
    if not line or not line.Parent then
      line = Instance.new('Part')
      line.Name = '__FrustumEdge'
      line.Shape = Enum.PartType.Block
      line.Anchored = true
      line.CanQuery = false
      line.CanTouch = false
      line.CanCollide = false
      line.CastShadow = false
      line.TopSurface = 0
      line.BottomSurface = 0
      line.Transparency = 0.5
      line.Parent = workspace
      lines[key] = line

      if cleanup then
        cleanup(line)
      end
    end

    local id = line:GetAttribute('EdgeId')
    if id ~= key then
      line:SetAttribute('EdgeId', id)

      if not maxAlpha then
        maxAlpha = math.floor(((edgeCount-1)*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
      end

      local alpha = math.floor((i*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
      line.Color = Color3.fromHSV(alpha / maxAlpha, 0.9, 0.95)
    end

    local length = (p1 - p0).Magnitude
    line.Size = Vector3.new(0.2, 0.2, length)
    line.CFrame = CFrame.lookAt(p0, p1, Vector3.yAxis) * CFrame.new(0, 0, -length*0.5)
  end

  return lines
end

local function createFakeCameraFromInstance(inst, fieldOfView, nearPlaneZ, viewportSize)
  -- this is just used for me to create a debug instance
  -- of a camera without actually having to supply a camera
  -- so that we can move a block around in studio as a 'fake camera'
  --
  local internal = { }
  internal.Instance = inst
  internal.FieldOfView = fieldOfView or 70
  internal.NearPlaneZ = nearPlaneZ or -0.1
  internal.ViewportSize = viewportSize or workspace.CurrentCamera.ViewportSize

  local fakeCamera = newproxy(true)
  local proxyMetatable = getmetatable(fakeCamera)
  function proxyMetatable:__index(index)
    if internal[index] then
      return internal[index]
    end

    local succ, res = pcall(function ()
      return inst[index]
    end)

    assert(succ, string.format('No property %q for FakeCamera', index))
    return res
  end

  return fakeCamera
end

return function (target)
  -- cleanup
  local maid = Maid.new()

  local lineMaid = Maid.new()
  maid._lines = lineMaid

  -- var(s)
  --   note: these can be changed to modify the frustum
  --         so that we can easily visualise the effect(s)
  local farPlaneZ = 50
  local nearPlaneZ = -0.1
  local fieldOfView = 70
  local viewportSize = workspace.CurrentCamera.ViewportSize -- e.g. Vector2.new(1920, 1080)

  -- create camera from block for debug viewing
  local cameraBlock = Instance.new('Part')
  cameraBlock.Name = 'FakeCamera'
  cameraBlock.Size = Vector3.one*4
  cameraBlock.CFrame = CFrame.new(0, 12, 0)
  cameraBlock.Shape = Enum.PartType.Block
  cameraBlock.Anchored = true
  cameraBlock.CanQuery = false
  cameraBlock.CanTouch = false
  cameraBlock.CanCollide = false
  cameraBlock.TopSurface = 0
  cameraBlock.BottomSurface = 0
  cameraBlock.Transparency = 0.5
  cameraBlock.Parent = workspace
  maid:addTask(cameraBlock)

  cameraBlock:SetAttribute('ShowBox', false)
  cameraBlock:SetAttribute('ShowFrustum', false)

  local showBox, showFrustum
  maid:addTask(
    cameraBlock.AttributeChanged:Connect(function ()
      -- i.e. listen to attr changes so we can toggle the
      --      frustum gizmo(s)
      local attributes = cameraBlock:GetAttributes()
      if showBox and not attributes.ShowBox then
        maid._box = nil
      end
      showBox = attributes.ShowBox

      if showFrustum and not attributes.ShowFrustum then
        lineMaid:cleanup()
      end
      showFrustum = attributes.ShowFrustum
    end)
  )

  local camera, frustum
  camera = createFakeCameraFromInstance(cameraBlock, fieldOfView, nearPlaneZ, viewportSize)

  -- create test sphere
  local someTestSphere = Instance.new('Part')
  someTestSphere.Name = 'SomeTestSphere'
  someTestSphere.Size = Vector3.one*2
  someTestSphere.CFrame = CFrame.new(6, 12, 6)
  someTestSphere.Shape = Enum.PartType.Ball
  someTestSphere.Anchored = true
  someTestSphere.CanQuery = false
  someTestSphere.CanTouch = false
  someTestSphere.CanCollide = false
  someTestSphere.TopSurface = 0
  someTestSphere.BottomSurface = 0
  someTestSphere.Transparency = 0.5
  someTestSphere.Parent = workspace
  maid:addTask(someTestSphere)

  local sphereSize = someTestSphere.Size
  local spherePosition = someTestSphere.Position
  local sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5

  local sphereWatchdog
  sphereWatchdog = someTestSphere:GetPropertyChangedSignal('CFrame'):Connect(function ()
    -- listen to changes so we can view
    -- the changes from the perspective of the sphere
    sphereSize = someTestSphere.Size
    spherePosition = someTestSphere.Position
    sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5
  end)
  maid:addTask(sphereWatchdog)

  -- create test obb box
  local someTestBox = Instance.new('Part')
  someTestBox.Name = 'SomeTestBox'
  someTestBox.Size = Vector3.one*4
  someTestBox.CFrame = CFrame.new(-6, 12, 6)
  someTestBox.Shape = Enum.PartType.Block
  someTestBox.Anchored = true
  someTestBox.CanQuery = false
  someTestBox.CanTouch = false
  someTestBox.CanCollide = false
  someTestBox.TopSurface = 0
  someTestBox.BottomSurface = 0
  someTestBox.Transparency = 0.5
  someTestBox.Parent = workspace
  maid:addTask(someTestBox)

  local obbOrigin, obbExtents = getBoundingBox(someTestBox)
  obbExtents *= 0.5

  local boxWatchdog
  boxWatchdog = someTestBox:GetPropertyChangedSignal('CFrame'):Connect(function ()
    -- listen to changes so we can view
    -- the changes from the perspective of the box
    obbOrigin, obbExtents = getBoundingBox(someTestBox)
    obbExtents *= 0.5
  end)
  maid:addTask(boxWatchdog)

  -- runtime
  local runtime, lines, box
  runtime = RunService.Heartbeat:Connect(function (gt, dt)
    -- update frustum
    frustum = buildFrustumFromCamera(camera, farPlaneZ, frustum)

    -- draw frustum edge(s) gizmo
    if showFrustum then
      lines = drawEdges(frustum.points, FRUSTUM_EDGES, lines, function (obj)
        lineMaid:addTask(obj)
      end)
    end

    -- draw frustum box gizmo
    if showBox then
      box = computeFrustumBox(frustum, box)

      drawBox(box, maid._box, function (obj)
        maid._box = obj
      end)
    end

    -- test whether our box/sphere is inside
    -- or outside the viewing frustum
    local boxInFrustum = testFrustumOBB(frustum, obbOrigin, obbExtents)
    someTestBox.BrickColor = boxInFrustum and BrickColor.Green() or BrickColor.Red()

    local sphereInFrustum = testFrustumSphere(frustum, spherePosition, sphereRadius)
    someTestSphere.BrickColor = sphereInFrustum and BrickColor.Green() or BrickColor.Red()
  end)
  maid:addTask(runtime)

  return function ()
    maid:cleanup()
  end
end

1 Like

Hello. I’ve just found a perfect solution! This video will show you how to do this. I hope this helps you: https://www.youtube.com/watch?v=lOu8v176NC8