Detect part visibility on screen

Help and Feedback Scripting Support
, , , ,
#1

Hey DevForum!

What’s the best way to detect if a part is visible on screen?

I made this system, which isn’t bad but doesn’t work as well as rays. However, rays aren’t really good for performance in my case since I need lots of rendering at once.

My current code:

function IsPartVisible(Part)
	local Size = Part.Size / 2
	local Points = {}
	for X = -1, 1, Accuracy do
		for Y = -1, 1, Accuracy do
			for Z = -1, 1, Accuracy do
				local Point = Part.CFrame:PointToWorldSpace(Vector3.new(Size.X * X, Size.Y * Y, Size.Z * Z))
				table.insert(Points, Point)
			end
		end
	end
	for _, Point in pairs(Points) do
		local ScreenPoint, Visible = Camera:WorldToScreenPoint(Point)
		if Visible then
			return true
		end
	end
	return false
end

Help ASAP!
I would appreciete any help!

#2

So do you mean the part is visible on the screen, or the player’s mouse is on the part?

#3

If part is visible anywhere on screen.

#4

Ah, ok. I think I’ve seen some tutorials on Youtube, I’ll try look for one. Off of my mind, I don’t know how to do this, I think it’s related to the LookVector but I’m probably wrong. I’ll reply if I find one.

#5

I tried YouTube, but most of them are just RayCasting tutorials which aren’t ideal for me. Thanks!

#6

Okay, I’ve just come across this DevForum post you might want to look at: How would I detect if the player can see an object?

Here is the code by Proligant too:

local _, withinScreenBounds = Camera:WorldToScreenPoint(part.Position)

if withinScreenBounds then
   -- We have successfully detected if the player can see the part.
end
#7

I currently use a modified version of this system.

The issue with the one you sent is that it only detects the center of an object, making it difficult to use with larger objects.

#8

How about this: Check if an object is in the screen

#9

Nope. This is similar to RayCast. returns a value based on if part is covered by another object

#10

I don’t think there are any other ways. I’m not 100% sure though.

#11

you can test if a part is on screen, without using rays, if both the front-left-top bounds position and the bottom-right-back bounds position is not on screen.

you might aswell also check if the center point is on screen.
get the bounds of your part, and boom:

--i didn't test this with parts, only in an open field, and i didn't test this 
--thoroughly so there might be edge cases that exist, like obscuring parts 
--but i commented how to cover that if you need to
local cam = workspace.CurrentCamera
local lel = workspace:WaitForChild("lel")

local minBound = lel.Position - (lel.Size/2)
local maxBound = lel.Position + (lel.Size/2)
	
local _, minRes = cam:WorldToScreenPoint(minBound)
local _, maxRes = cam:WorldToScreenPoint(maxBound)
local _, midRes = cam:WorldToScreenPoint(lel.Position)
--local obscuredParts = cam:GetPartsObscuringTarget({lel.Position, minBound, maxBound})
	
if minRes or maxRes or midRes then --and #obscuredParts <= 0 
	print("Is on screen!")
else
	print("Is not on screen!")
end

edit: this assumes your part is rectangular or (like a cylinder) has a rectangular hitbox.
this might behave differently on unions, spheres, 3d models which vaguely represent a rectangle etc.

edit: spelling
edit: added more casting points for extra accuracy.

edit: edge case that I found is if the part is this huge that you’re staring at it but all of the bounds + center are off screen because it is so big, but I’m assuming you don’t have really big parts.

#12

I actually answered a very similar question yesterday.

At minimum, you would need to perform a Frustum-OBB intersection test. You can learn more about this from the following links: viewing frustum, OBB, separating axis theorem and a good starting resource relating to intersection tests here.

Take a look at the thread I mentioned here - there’s a video demoing it - or to try it yourself, see the following:

Example File

FrustumExamples.rbxm (10.0 KB)

Example Code

Frustum debug tests 
local RunService = game:GetService('RunService')
local packages = game:GetService('ServerStorage').mono.packages.shared

local Maid = require(packages.utils.maid)

-- indices that define the edges of the frustum
-- so that we can render it through a gizmo
local FRUSTUM_EDGES = {
  -- right plane
  1, 5,   4, 8,

  -- left plane
  2, 6,   3, 7,

  -- near plane
  1, 2,   2, 3,
  3, 4,   4, 1,

  -- far plane
  5, 6,   6, 7,
  7, 8,   8, 5,
}

local function getBoundingBox(obj)
  -- safely computes the bounding box for the given param
  -- expects either (a) a BasePart or (b) a Model
  -- otherwise defaults to identity matrix + zero vec
  if typeof(obj) == 'Instance' then
    if obj:IsA('BasePart') then
      return obj.CFrame, obj.Size
    elseif obj:IsA('Model') then
      return obj:GetBoundingBox()
    end
  end

  return CFrame.identity, Vector3.zero
end

local function planeFromPoints(a, b, c, output)
  -- computes a plane from 3 points
  --    learn more about planes here, e.g. ref:
  --     - https://gdbooks.gitbooks.io/3dcollisions/content/Chapter1/plane.html
  --     - https://en.wikipedia.org/wiki/Plane_(mathematics)
  --
  local normal = (b - a):Cross(c - a).Unit
  local distance = -normal:Dot(a)

  if not output then
    output = table.create(5)
  end

  output[1] = normal
  output[2] = distance
  output[3] = normal.X
  output[4] = normal.Y
  output[5] = normal.Z

  return output
end

local function buildFrustumFromCamera(camera, farPlaneZ, output)
  -- builds a frustum from a Camera object instance
  --
  -- NOTE:
  --   far plane is user-defined since Roblox doesn't allow us to read it
  --   from their camera; but that's fine because you can use the far plane
  --   to adjust what you would prefer the render distance to be. Though,
  --   in this case, i just default to 200.
  --
  --   Learn more about viewing frustum e.g. ref @ https://en.wikipedia.org/wiki/Viewing_frustum
  --
  farPlaneZ = farPlaneZ or 200

  local transform = camera.CFrame
  local translation = transform.Position
  local nearPlaneZ = camera.NearPlaneZ

  local fieldOfView = camera.FieldOfView
  local tanHalfFov = math.tan(math.rad(fieldOfView * 0.5))

  local viewportSize = camera.ViewportSize
  local viewportAspectRatio = viewportSize.X / viewportSize.Y

  local farPlaneHeight_2 = tanHalfFov*farPlaneZ
  local farPlaneWidth_2 = farPlaneHeight_2*viewportAspectRatio

  local nearPlaneHeight_2 = tanHalfFov*-nearPlaneZ
  local nearPlaneWidth_2 = nearPlaneHeight_2*viewportAspectRatio

  farPlaneZ *= -1

  local farTopLeft =      transform * Vector3.new( -farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
  local farTopRight =     transform * Vector3.new(  farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
  local farBottomLeft =   transform * Vector3.new( -farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)
  local farBottomRight =  transform * Vector3.new(  farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)

  local nearTopLeft =     transform * Vector3.new(-nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
  local nearTopRight =    transform * Vector3.new( nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
  local nearBottomLeft =  transform * Vector3.new(-nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)
  local nearBottomRight = transform * Vector3.new( nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)

  output = output or { }
  output.transform = transform
  output.translation = translation
  output.viewportSize = viewportSize

  local frustumPoints = output.points
  if not frustumPoints then
    frustumPoints = table.create(8)
    output.points = frustumPoints
  end
  frustumPoints[1] = nearTopRight
  frustumPoints[2] = nearTopLeft
  frustumPoints[3] = nearBottomLeft
  frustumPoints[4] = nearBottomRight
  frustumPoints[5] = farTopRight
  frustumPoints[6] = farTopLeft
  frustumPoints[7] = farBottomLeft
  frustumPoints[8] = farBottomRight

  local frustumPlanes = output.planes
  if not frustumPlanes then
    frustumPlanes = table.create(6)
    output.planes = frustumPlanes
  end
  frustumPlanes[1] = planeFromPoints(   nearTopRight, nearBottomRight,     nearTopLeft, frustumPlanes[1]) -- near
  frustumPlanes[2] = planeFromPoints(    farTopRight,      farTopLeft,  farBottomRight, frustumPlanes[2]) -- far
  frustumPlanes[3] = planeFromPoints(   nearTopRight,     nearTopLeft,     farTopRight, frustumPlanes[3]) -- top
  frustumPlanes[4] = planeFromPoints(nearBottomRight,  farBottomRight,  nearBottomLeft, frustumPlanes[4]) -- bottom
  frustumPlanes[5] = planeFromPoints(    nearTopLeft,  nearBottomLeft,      farTopLeft, frustumPlanes[5]) -- left
  frustumPlanes[6] = planeFromPoints(   nearTopRight,     farTopRight, nearBottomRight, frustumPlanes[6]) -- right

  return output
end

local function testFrustumSphere(frustum, spherePosition, sphereRadius, padding)
  -- tests whether the frustum intersects a sphere
  padding = padding or 0

  local planes = frustum.planes
  local x, y, z = spherePosition.X, spherePosition.Y, spherePosition.Z

  local nx, ny, nz, d, test
  test = -sphereRadius - padding
  for i = 1, 6, 1 do
    local plane = planes[i]
    d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

    local m = nx*x + ny*y + nz*z + d
    if m < test then
      return false
    end
  end

  return true
end

local function testFrustumOBB(frustum, obbOrigin, obbExtents, padding)
  -- tests whether the frustum intersects with an OBB
  --
  -- TODO for you:
  --   do a less computationally expensive test first to check
  --   if it's even possible for these to be intersecting
  --
  local planes = frustum.planes
  padding = padding or 0

  -- m11, m12, m13, x
  -- m21, m22, m23, y
  -- m31, m32, m33, z
  --   0,   0,   0, 1
  local sx, sy, sz = obbExtents.X, obbExtents.Y, obbExtents.Z
  local  x,  y,  z,
        rx, ux, bx,
        ry, uy, by,
        rz, uz, bz = obbOrigin:GetComponents()

  local s,  r,  u,  b
  local d, nx, ny, nz
  for i = 1, 6, 1 do
    local plane = planes[i]
    d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

    s = x*nx + y*ny + z*nz
    r = sx*math.abs(nx*rx + ny*ry + nz*rz)
    u = sy*math.abs(nx*ux + ny*uy + nz*uz)
    b = sz*math.abs(nx*bx + ny*by + nz*bz)

    if (s + r + u + b) < -d - padding then
      return false
    end
  end

  return true
end

local function computeFrustumBox(frustum, output)
  -- compute the frustum's OBB
  --    can be used to compute the AABB which you could use
  --    for some kind of broad-phase search. a method to create
  --    an aabb from an obb can be found here:
  --      - Ref @ https://zeux.io/2010/10/17/aabb-from-obb-with-component-wise-abs/
  --
  --    though, i should note that you can get the AABB much faster by min-maxing the transform + the far plane points
  --

  local points = frustum.points
  local transform = frustum.transform
  local translation = frustum.translation

  local x, y, z
  local xmin, xmax = math.huge, -math.huge
  local ymin, ymax = math.huge, -math.huge
  local zmin, zmax = math.huge, -math.huge
  for i = 0, 4, 1 do
    local point
    if i == 0 then
      point = translation
    else
      point = points[i + 4]
    end

    point = transform:PointToObjectSpace(point)
    x, y, z = point.X, point.Y, point.Z

    xmin = math.min(xmin, x)
    xmax = math.max(xmax, x)
    ymin = math.min(ymin, y)
    ymax = math.max(ymax, y)
    zmin = math.min(zmin, z)
    zmax = math.max(zmax, z)
  end

  local offset = Vector3.new(
    0.5*(xmin + xmax),
    0.5*(ymin + ymax),
    0.5*(zmin + zmax)
  )

  local extents = Vector3.new(xmax - xmin, ymax - ymin, zmax - zmin)
  local minBounds = Vector3.new(xmin, ymin, zmin)
  local maxBounds = Vector3.new(xmax, ymax, zmax)

  local rotation = transform.Rotation
  output = output or { }
  output.origin = transform + rotation:PointToWorldSpace(offset)
  output.extents = extents

  local bounds = output.bounds
  if not bounds then
    bounds = table.create(2)
    output.bounds = bounds
  end
  bounds[1] = minBounds
  bounds[2] = maxBounds

  return output
end

local function drawBox(box, part, cleanup)
  -- draws an OBB gizmo
  if not part then
    part = Instance.new('Part')
    part.Name = '__FrustumBox'
    part.Shape = Enum.PartType.Block
    part.Anchored = true
    part.CanQuery = false
    part.CanTouch = false
    part.CanCollide = false
    part.CastShadow = false
    part.BrickColor = BrickColor.Red()
    part.TopSurface = 0
    part.BottomSurface = 0
    part.Transparency = 0.9
    part.Parent = workspace

    if cleanup then
      cleanup(part)
    end
  end

  part.CFrame = box.origin
  part.Size = box.extents

  return part
end

local function drawEdges(points, indices, lines, cleanup)
  -- draws a frustum given it's points and the indices
  -- that make up its edges
  lines = lines or { }

  local vertCount = #indices
  local edgeCount = math.floor(vertCount*0.5)
  local cornerCount = #points

  local key, index, maxAlpha
  for i = 0, edgeCount - 1, 1 do
    key = i + 1
    index = i*2 + 1

    local p0 = points[indices[index + 0]]
    local p1 = points[indices[index + 1]]

    local line = lines[key]
    if not line or not line.Parent then
      line = Instance.new('Part')
      line.Name = '__FrustumEdge'
      line.Shape = Enum.PartType.Block
      line.Anchored = true
      line.CanQuery = false
      line.CanTouch = false
      line.CanCollide = false
      line.CastShadow = false
      line.TopSurface = 0
      line.BottomSurface = 0
      line.Transparency = 0.5
      line.Parent = workspace
      lines[key] = line

      if cleanup then
        cleanup(line)
      end
    end

    local id = line:GetAttribute('EdgeId')
    if id ~= key then
      line:SetAttribute('EdgeId', id)

      if not maxAlpha then
        maxAlpha = math.floor(((edgeCount-1)*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
      end

      local alpha = math.floor((i*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
      line.Color = Color3.fromHSV(alpha / maxAlpha, 0.9, 0.95)
    end

    local length = (p1 - p0).Magnitude
    line.Size = Vector3.new(0.2, 0.2, length)
    line.CFrame = CFrame.lookAt(p0, p1, Vector3.yAxis) * CFrame.new(0, 0, -length*0.5)
  end

  return lines
end

local function createFakeCameraFromInstance(inst, fieldOfView, nearPlaneZ, viewportSize)
  -- this is just used for me to create a debug instance
  -- of a camera without actually having to supply a camera
  -- so that we can move a block around in studio as a 'fake camera'
  --
  local internal = { }
  internal.Instance = inst
  internal.FieldOfView = fieldOfView or 70
  internal.NearPlaneZ = nearPlaneZ or -0.1
  internal.ViewportSize = viewportSize or workspace.CurrentCamera.ViewportSize

  local fakeCamera = newproxy(true)
  local proxyMetatable = getmetatable(fakeCamera)
  function proxyMetatable:__index(index)
    if internal[index] then
      return internal[index]
    end

    local succ, res = pcall(function ()
      return inst[index]
    end)

    assert(succ, string.format('No property %q for FakeCamera', index))
    return res
  end

  return fakeCamera
end

return function (target)
  -- cleanup
  local maid = Maid.new()

  local lineMaid = Maid.new()
  maid._lines = lineMaid

  -- var(s)
  --   note: these can be changed to modify the frustum
  --         so that we can easily visualise the effect(s)
  local farPlaneZ = 50
  local nearPlaneZ = -0.1
  local fieldOfView = 70
  local viewportSize = workspace.CurrentCamera.ViewportSize -- e.g. Vector2.new(1920, 1080)

  -- create camera from block for debug viewing
  local cameraBlock = Instance.new('Part')
  cameraBlock.Name = 'FakeCamera'
  cameraBlock.Size = Vector3.one*4
  cameraBlock.CFrame = CFrame.new(0, 12, 0)
  cameraBlock.Shape = Enum.PartType.Block
  cameraBlock.Anchored = true
  cameraBlock.CanQuery = false
  cameraBlock.CanTouch = false
  cameraBlock.CanCollide = false
  cameraBlock.TopSurface = 0
  cameraBlock.BottomSurface = 0
  cameraBlock.Transparency = 0.5
  cameraBlock.Parent = workspace
  maid:addTask(cameraBlock)

  cameraBlock:SetAttribute('ShowBox', false)
  cameraBlock:SetAttribute('ShowFrustum', false)

  local showBox, showFrustum
  maid:addTask(
    cameraBlock.AttributeChanged:Connect(function ()
      -- i.e. listen to attr changes so we can toggle the
      --      frustum gizmo(s)
      local attributes = cameraBlock:GetAttributes()
      if showBox and not attributes.ShowBox then
        maid._box = nil
      end
      showBox = attributes.ShowBox

      if showFrustum and not attributes.ShowFrustum then
        lineMaid:cleanup()
      end
      showFrustum = attributes.ShowFrustum
    end)
  )

  local camera, frustum
  camera = createFakeCameraFromInstance(cameraBlock, fieldOfView, nearPlaneZ, viewportSize)

  -- create test sphere
  local someTestSphere = Instance.new('Part')
  someTestSphere.Name = 'SomeTestSphere'
  someTestSphere.Size = Vector3.one*2
  someTestSphere.CFrame = CFrame.new(6, 12, 6)
  someTestSphere.Shape = Enum.PartType.Ball
  someTestSphere.Anchored = true
  someTestSphere.CanQuery = false
  someTestSphere.CanTouch = false
  someTestSphere.CanCollide = false
  someTestSphere.TopSurface = 0
  someTestSphere.BottomSurface = 0
  someTestSphere.Transparency = 0.5
  someTestSphere.Parent = workspace
  maid:addTask(someTestSphere)

  local sphereSize = someTestSphere.Size
  local spherePosition = someTestSphere.Position
  local sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5

  local sphereWatchdog
  sphereWatchdog = someTestSphere:GetPropertyChangedSignal('CFrame'):Connect(function ()
    -- listen to changes so we can view
    -- the changes from the perspective of the sphere
    sphereSize = someTestSphere.Size
    spherePosition = someTestSphere.Position
    sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5
  end)
  maid:addTask(sphereWatchdog)

  -- create test obb box
  local someTestBox = Instance.new('Part')
  someTestBox.Name = 'SomeTestBox'
  someTestBox.Size = Vector3.one*4
  someTestBox.CFrame = CFrame.new(-6, 12, 6)
  someTestBox.Shape = Enum.PartType.Block
  someTestBox.Anchored = true
  someTestBox.CanQuery = false
  someTestBox.CanTouch = false
  someTestBox.CanCollide = false
  someTestBox.TopSurface = 0
  someTestBox.BottomSurface = 0
  someTestBox.Transparency = 0.5
  someTestBox.Parent = workspace
  maid:addTask(someTestBox)

  local obbOrigin, obbExtents = getBoundingBox(someTestBox)
  obbExtents *= 0.5

  local boxWatchdog
  boxWatchdog = someTestBox:GetPropertyChangedSignal('CFrame'):Connect(function ()
    -- listen to changes so we can view
    -- the changes from the perspective of the box
    obbOrigin, obbExtents = getBoundingBox(someTestBox)
    obbExtents *= 0.5
  end)
  maid:addTask(boxWatchdog)

  -- runtime
  local runtime, lines, box
  runtime = RunService.Heartbeat:Connect(function (gt, dt)
    -- update frustum
    frustum = buildFrustumFromCamera(camera, farPlaneZ, frustum)

    -- draw frustum edge(s) gizmo
    if showFrustum then
      lines = drawEdges(frustum.points, FRUSTUM_EDGES, lines, function (obj)
        lineMaid:addTask(obj)
      end)
    end

    -- draw frustum box gizmo
    if showBox then
      box = computeFrustumBox(frustum, box)

      drawBox(box, maid._box, function (obj)
        maid._box = obj
      end)
    end

    -- test whether our box/sphere is inside
    -- or outside the viewing frustum
    local boxInFrustum = testFrustumOBB(frustum, obbOrigin, obbExtents)
    someTestBox.BrickColor = boxInFrustum and BrickColor.Green() or BrickColor.Red()

    local sphereInFrustum = testFrustumSphere(frustum, spherePosition, sphereRadius)
    someTestSphere.BrickColor = sphereInFrustum and BrickColor.Green() or BrickColor.Red()
  end)
  maid:addTask(runtime)

  return function ()
    maid:cleanup()
  end
end