Camera Tool that Saves Images

[Crimson_VID (super_paper)]

I saw a Reddit post where someone created a camera that was able to take pictures of the game and display them. I was wondering how I could achieve this and what services I need to use (if it is possible of course).

Here is the: reddit post

[a19_9 (Nol)]

its a viewport frame, he clones everything in the workspace into it and then changes the cameras cframe in a loop and then stops looping when the picture is taken, though this would prob be kind of laggy if youre using alot of parts

[Crimson_VID (super_paper)]

Thanks for explaining how it worked, is there any other way to do effectively the same thing but less laggy?

[a19_9 (Nol)]

you can just clone a specific folder instead of the whole workspace and clone it locally so it doesnt apply to other players

[Crimson_VID (super_paper)]

I’ll try this out, thanks for helping!

[DevSynaptix]

@Crimson_VID

Thanks for explaining how it worked, is there any other way to do effectively the same thing but less laggy?

Preparation

You could optionally create and use something like an octree or a BVH to store references to object(s) within your game that you would like to be captured - this would allow you to limit your world queries to a subset of the map which will reduce computation time.

This doesn’t solve having to iterate through dynamic objects, e.g. players, unless you update their reference at runtime; but it’s almost guaranteed you have significantly fewer dynamic objects than static ones.

When capturing

1. Create or update a viewing frustum from the Camera object that the player is holding, learn more about that here

2. Query the world using the method described in #Preparation, or alternatively, use Roblox’s spatial query API (learn more here) to find relevant game instances

3. Using the Camera frustum, you can cull the objects from step (2) by using Frustum tests, e.g. Frustum-OBB intersection via SAT tests (see resource here)

   • I should note, however, that for your use case it probably doesn’t need to be perfect assuming you’ve added other optimisations; so you could consider a simple Frustum-Sphere test. Learn more here and here

4. Instantiate a viewport frame such that:

   • The Viewport’s CurrentCamera instances reflects the Camera object’s transform
   • The Viewport contains the cloned game instance(s) found in step (3) - since we’re cloning and we’ve already updated the CurrentCamera coordinates, each of the instance’s coordinates should remain the same
   • This isn’t shown in the video you linked, but you can achieve a skybox-like effect in ViewportFrames if you also wanted to capture the sky in the background, e.g. example model here

[Crimson_VID (super_paper)]

How do I use frustum culling with the camera frustum?

[DevSynaptix]

You’d have to implement it yourself in this instance; when I wake up tomorrow I’ll put something together for you as an example but it’s 2am for me atm - will comment it here at some point tomorrow

P.S. in the meantime, if you wanted to experiment yourself before tomorrow, this is a fairly good resource which might give you some more info: here and here

[DevSynaptix]

I included examples on Frustum-Sphere and Frustum-OBB tests for you but the rest of it, e.g. spatial indexing per the “Preparation” step and the like will be up to you. This should help you get started at least.

File Example

Here’s an .rbxm file containing the example code:
FrustumExamples.rbxm (10.0 KB)

Note that you can toggle attributes on the FakeCamera instance to show the debug views, e.g.:

Video

Code Example

Example Code

Note: I’m relying on a Maid library here to cleanup signals / parts - this isn’t necessary for the frustum side of things. You can find a Maid library in the attached .rbxm file, or you can bring your own - there’s several different ones available on the DevForum.

local Maid = require(script.maid)


-- CONST

-- indices that define the edges of the frustum
-- so that we can render it through a gizmo
local FRUSTUM_EDGES = {
	-- right plane
	1, 5,   4, 8,

	-- left plane
	2, 6,   3, 7,

	-- near plane
	1, 2,   2, 3,
	3, 4,   4, 1,

	-- far plane
	5, 6,   6, 7,
	7, 8,   8, 5,
}


-- UTILS

local function getBoundingBox(obj)
	-- safely computes the bounding box for the given param
	-- expects either (a) a BasePart or (b) a Model
	-- otherwise defaults to identity matrix + zero vec
	if typeof(obj) == 'Instance' then
		if obj:IsA('BasePart') then
			return obj.CFrame, obj.Size
		elseif obj:IsA('Model') then
			return obj:GetBoundingBox()
		end
	end

	return CFrame.identity, Vector3.zero
end

local function planeFromPoints(a, b, c, output)
	-- computes a plane from 3 points
	--    learn more about planes here, e.g. ref:
	--     - https://gdbooks.gitbooks.io/3dcollisions/content/Chapter1/plane.html
	--     - https://en.wikipedia.org/wiki/Plane_(mathematics)
	--
	local normal = (b - a):Cross(c - a).Unit
	local distance = -normal:Dot(a)

	if not output then
		output = table.create(5)
	end

	output[1] = normal
	output[2] = distance
	output[3] = normal.X
	output[4] = normal.Y
	output[5] = normal.Z

	return output
end

local function buildFrustumFromCamera(camera, farPlaneZ, output)
	-- builds a frustum from a Camera object instance
	--
	-- NOTE:
	--   far plane is user-defined since Roblox doesn't allow us to read it
	--   from their camera; but that's fine because you can use the far plane
	--   to adjust what you would prefer the render distance to be. Though,
	--   in this case, i just default to 200.
	--
	--   Learn more about viewing frustum e.g. ref @ https://en.wikipedia.org/wiki/Viewing_frustum
	--
	farPlaneZ = farPlaneZ or 200

	local transform = camera.CFrame
	local translation = transform.Position
	local nearPlaneZ = camera.NearPlaneZ

	local fieldOfView = camera.FieldOfView
	local tanHalfFov = math.tan(math.rad(fieldOfView * 0.5))

	local viewportSize = camera.ViewportSize
	local viewportAspectRatio = viewportSize.X / viewportSize.Y

	local farPlaneHeight_2 = tanHalfFov*farPlaneZ
	local farPlaneWidth_2 = farPlaneHeight_2*viewportAspectRatio

	local nearPlaneHeight_2 = tanHalfFov*-nearPlaneZ
	local nearPlaneWidth_2 = nearPlaneHeight_2*viewportAspectRatio

	farPlaneZ *= -1

	local farTopLeft =      transform * Vector3.new( -farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
	local farTopRight =     transform * Vector3.new(  farPlaneWidth_2,   farPlaneHeight_2, farPlaneZ)
	local farBottomLeft =   transform * Vector3.new( -farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)
	local farBottomRight =  transform * Vector3.new(  farPlaneWidth_2,  -farPlaneHeight_2, farPlaneZ)

	local nearTopLeft =     transform * Vector3.new(-nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
	local nearTopRight =    transform * Vector3.new( nearPlaneWidth_2,  nearPlaneHeight_2, nearPlaneZ)
	local nearBottomLeft =  transform * Vector3.new(-nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)
	local nearBottomRight = transform * Vector3.new( nearPlaneWidth_2, -nearPlaneHeight_2, nearPlaneZ)

	output = output or { }
	output.transform = transform
	output.translation = translation
	output.viewportSize = viewportSize

	local frustumPoints = output.points
	if not frustumPoints then
		frustumPoints = table.create(8)
		output.points = frustumPoints
	end
	frustumPoints[1] = nearTopRight
	frustumPoints[2] = nearTopLeft
	frustumPoints[3] = nearBottomLeft
	frustumPoints[4] = nearBottomRight
	frustumPoints[5] = farTopRight
	frustumPoints[6] = farTopLeft
	frustumPoints[7] = farBottomLeft
	frustumPoints[8] = farBottomRight

	local frustumPlanes = output.planes
	if not frustumPlanes then
		frustumPlanes = table.create(6)
		output.planes = frustumPlanes
	end
	frustumPlanes[1] = planeFromPoints(   nearTopRight, nearBottomRight,     nearTopLeft, frustumPlanes[1]) -- near
	frustumPlanes[2] = planeFromPoints(    farTopRight,      farTopLeft,  farBottomRight, frustumPlanes[2]) -- far
	frustumPlanes[3] = planeFromPoints(   nearTopRight,     nearTopLeft,     farTopRight, frustumPlanes[3]) -- top
	frustumPlanes[4] = planeFromPoints(nearBottomRight,  farBottomRight,  nearBottomLeft, frustumPlanes[4]) -- bottom
	frustumPlanes[5] = planeFromPoints(    nearTopLeft,  nearBottomLeft,      farTopLeft, frustumPlanes[5]) -- left
	frustumPlanes[6] = planeFromPoints(   nearTopRight,     farTopRight, nearBottomRight, frustumPlanes[6]) -- right

	return output
end

local function testFrustumSphere(frustum, spherePosition, sphereRadius, padding)
	-- tests whether the frustum intersects a sphere
	padding = padding or 0

	local planes = frustum.planes
	local x, y, z = spherePosition.X, spherePosition.Y, spherePosition.Z

	local nx, ny, nz, d, test
	test = -sphereRadius - padding
	for i = 1, 6, 1 do
		local plane = planes[i]
		d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

		local m = nx*x + ny*y + nz*z + d
		if m < test then
			return false
		end
	end

	return true
end

local function testFrustumOBB(frustum, obbOrigin, obbExtents, padding)
	-- tests whether the frustum intersects with an OBB
	--
	-- TODO for you:
	--   do a less computationally expensive test first to check
	--   if it's even possible for these to be intersecting
	--
	local planes = frustum.planes
	padding = padding or 0

	-- m11, m12, m13, x
	-- m21, m22, m23, y
	-- m31, m32, m33, z
	--   0,   0,   0, 1
	local sx, sy, sz = obbExtents.X, obbExtents.Y, obbExtents.Z
	local  x,  y,  z,
	rx, ux, bx,
	ry, uy, by,
	rz, uz, bz = obbOrigin:GetComponents()

	local s,  r,  u,  b
	local d, nx, ny, nz
	for i = 1, 6, 1 do
		local plane = planes[i]
		d, nx, ny, nz = plane[2], plane[3], plane[4], plane[5]

		s = x*nx + y*ny + z*nz
		r = sx*math.abs(nx*rx + ny*ry + nz*rz)
		u = sy*math.abs(nx*ux + ny*uy + nz*uz)
		b = sz*math.abs(nx*bx + ny*by + nz*bz)

		if (s + r + u + b) < -d - padding then
			return false
		end
	end

	return true
end

local function computeFrustumBox(frustum, output)
	-- compute the frustum's OBB
	--    can be used to compute the AABB which you could use
	--    for some kind of broad-phase search. a method to create
	--    an aabb from an obb can be found here:
	--      - Ref @ https://zeux.io/2010/10/17/aabb-from-obb-with-component-wise-abs/
	--
	--    though, i should note that you can get the AABB much faster by min-maxing the transform + the far plane points
	--

	local points = frustum.points
	local transform = frustum.transform
	local translation = frustum.translation

	local x, y, z
	local xmin, xmax = math.huge, -math.huge
	local ymin, ymax = math.huge, -math.huge
	local zmin, zmax = math.huge, -math.huge
	for i = 0, 4, 1 do
		local point
		if i == 0 then
			point = translation
		else
			point = points[i + 4]
		end

		point = transform:PointToObjectSpace(point)
		x, y, z = point.X, point.Y, point.Z

		xmin = math.min(xmin, x)
		xmax = math.max(xmax, x)
		ymin = math.min(ymin, y)
		ymax = math.max(ymax, y)
		zmin = math.min(zmin, z)
		zmax = math.max(zmax, z)
	end

	local offset = Vector3.new(
		0.5*(xmin + xmax),
		0.5*(ymin + ymax),
		0.5*(zmin + zmax)
	)

	local extents = Vector3.new(xmax - xmin, ymax - ymin, zmax - zmin)
	local minBounds = Vector3.new(xmin, ymin, zmin)
	local maxBounds = Vector3.new(xmax, ymax, zmax)

	local rotation = transform.Rotation
	output = output or { }
	output.origin = transform + rotation:PointToWorldSpace(offset)
	output.extents = extents

	local bounds = output.bounds
	if not bounds then
		bounds = table.create(2)
		output.bounds = bounds
	end
	bounds[1] = minBounds
	bounds[2] = maxBounds

	return output
end

local function drawBox(box, part, cleanup)
	-- draws an OBB gizmo
	if not part then
		part = Instance.new('Part')
		part.Name = '__FrustumBox'
		part.Shape = Enum.PartType.Block
		part.Anchored = true
		part.CanQuery = false
		part.CanTouch = false
		part.CanCollide = false
		part.CastShadow = false
		part.BrickColor = BrickColor.Red()
		part.TopSurface = 0
		part.BottomSurface = 0
		part.Transparency = 0.9
		part.Parent = workspace

		if cleanup then
			cleanup(part)
		end
	end

	part.CFrame = box.origin
	part.Size = box.extents

	return part
end

local function drawEdges(points, indices, lines, cleanup)
	-- draws a frustum given it's points and the indices
	-- that make up its edges
	lines = lines or { }

	local vertCount = #indices
	local edgeCount = math.floor(vertCount*0.5)
	local cornerCount = #points

	local key, index, maxAlpha
	for i = 0, edgeCount - 1, 1 do
		key = i + 1
		index = i*2 + 1

		local p0 = points[indices[index + 0]]
		local p1 = points[indices[index + 1]]

		local line = lines[key]
		if not line or not line.Parent then
			line = Instance.new('Part')
			line.Name = '__FrustumEdge'
			line.Shape = Enum.PartType.Block
			line.Anchored = true
			line.CanQuery = false
			line.CanTouch = false
			line.CanCollide = false
			line.CastShadow = false
			line.TopSurface = 0
			line.BottomSurface = 0
			line.Transparency = 0.5
			line.Parent = workspace
			lines[key] = line

			if cleanup then
				cleanup(line)
			end
		end

		local id = line:GetAttribute('EdgeId')
		if id ~= key then
			line:SetAttribute('EdgeId', id)

			if not maxAlpha then
				maxAlpha = math.floor(((edgeCount-1)*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
			end

			local alpha = math.floor((i*2 - cornerCount*0.5) / cornerCount + 0.5) + 1
			line.Color = Color3.fromHSV(alpha / maxAlpha, 0.9, 0.95)
		end

		local length = (p1 - p0).Magnitude
		line.Size = Vector3.new(0.2, 0.2, length)
		line.CFrame = CFrame.lookAt(p0, p1, Vector3.yAxis) * CFrame.new(0, 0, -length*0.5)
	end

	return lines
end

local function createFakeCameraFromInstance(inst, fieldOfView, nearPlaneZ, viewportSize)
	-- this is just used for me to create a debug instance
	-- of a camera without actually having to supply a camera
	-- so that we can move a block around in studio as a 'fake camera'
	--
	local internal = { }
	internal.Instance = inst
	internal.FieldOfView = fieldOfView or 70
	internal.NearPlaneZ = nearPlaneZ or -0.1
	internal.ViewportSize = viewportSize or workspace.CurrentCamera.ViewportSize

	local fakeCamera = newproxy(true)
	local proxyMetatable = getmetatable(fakeCamera)
	function proxyMetatable:__index(index)
		if internal[index] then
			return internal[index]
		end

		local succ, res = pcall(function ()
			return inst[index]
		end)

		assert(succ, string.format('No property %q for FakeCamera', tostring(index)))
		return res
	end

	return fakeCamera
end


-- EXAMPLE USAGE

-- e.g. just to show you how this all comes together
local function createDebugScene()
	local RunService = game:GetService('RunService')

	-- cleanup
	local maid = Maid.new()

	local lineMaid = Maid.new()
	maid._lines = lineMaid

	-- var(s)
	--
	--   note: these can be changed to modify the frustum
	--         so that we can easily visualise the effect(s)
	--
	local farPlaneZ = 50
	local nearPlaneZ = -0.1
	local fieldOfView = 70
	local viewportSize = workspace.CurrentCamera.ViewportSize -- e.g. Vector2.new(1920, 1080)

	-- create camera from block for debug viewing
	local cameraBlock = Instance.new('Part')
	cameraBlock.Name = 'FakeCamera'
	cameraBlock.Size = Vector3.one*4
	cameraBlock.CFrame = CFrame.new(0, 12, 0)
	cameraBlock.Shape = Enum.PartType.Block
	cameraBlock.Anchored = true
	cameraBlock.CanQuery = false
	cameraBlock.CanTouch = false
	cameraBlock.CanCollide = false
	cameraBlock.TopSurface = 0
	cameraBlock.BottomSurface = 0
	cameraBlock.Transparency = 0.5
	cameraBlock.Parent = workspace
	maid:addTask(cameraBlock)

	cameraBlock:SetAttribute('ShowBox', false)
	cameraBlock:SetAttribute('ShowFrustum', false)

	local showBox, showFrustum
	maid:addTask(
		cameraBlock.AttributeChanged:Connect(function ()
			-- i.e. listen to attr changes so we can toggle the
			--      frustum gizmo(s)
			local attributes = cameraBlock:GetAttributes()
			if showBox and not attributes.ShowBox then
				maid._box = nil
			end
			showBox = attributes.ShowBox

			if showFrustum and not attributes.ShowFrustum then
				lineMaid:cleanup()
			end
			showFrustum = attributes.ShowFrustum
		end)
	)

	local camera, frustum
	camera = createFakeCameraFromInstance(cameraBlock, fieldOfView, nearPlaneZ, viewportSize)

	-- create test sphere
	local someTestSphere = Instance.new('Part')
	someTestSphere.Name = 'SomeTestSphere'
	someTestSphere.Size = Vector3.one*2
	someTestSphere.CFrame = CFrame.new(6, 12, 6)
	someTestSphere.Shape = Enum.PartType.Ball
	someTestSphere.Anchored = true
	someTestSphere.CanQuery = false
	someTestSphere.CanTouch = false
	someTestSphere.CanCollide = false
	someTestSphere.TopSurface = 0
	someTestSphere.BottomSurface = 0
	someTestSphere.Transparency = 0.5
	someTestSphere.Parent = workspace
	maid:addTask(someTestSphere)

	local sphereSize = someTestSphere.Size
	local spherePosition = someTestSphere.Position
	local sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5

	local sphereWatchdog
	sphereWatchdog = someTestSphere:GetPropertyChangedSignal('CFrame'):Connect(function ()
		-- listen to changes so we can view
		-- the changes from the perspective of the sphere
		sphereSize = someTestSphere.Size
		spherePosition = someTestSphere.Position
		sphereRadius = math.min(sphereSize.X, sphereSize.Y, sphereSize.Z)*0.5
	end)
	maid:addTask(sphereWatchdog)

	-- create test obb box
	local someTestBox = Instance.new('Part')
	someTestBox.Name = 'SomeTestBox'
	someTestBox.Size = Vector3.one*4
	someTestBox.CFrame = CFrame.new(-6, 12, 6)
	someTestBox.Shape = Enum.PartType.Block
	someTestBox.Anchored = true
	someTestBox.CanQuery = false
	someTestBox.CanTouch = false
	someTestBox.CanCollide = false
	someTestBox.TopSurface = 0
	someTestBox.BottomSurface = 0
	someTestBox.Transparency = 0.5
	someTestBox.Parent = workspace
	maid:addTask(someTestBox)

	local obbOrigin, obbExtents = getBoundingBox(someTestBox)
	obbExtents *= 0.5

	local boxWatchdog
	boxWatchdog = someTestBox:GetPropertyChangedSignal('CFrame'):Connect(function ()
		-- listen to changes so we can view
		-- the changes from the perspective of the box
		obbOrigin, obbExtents = getBoundingBox(someTestBox)
		obbExtents *= 0.5
	end)
	maid:addTask(boxWatchdog)

	-- runtime
	local runtime, lines, box
	runtime = RunService.Heartbeat:Connect(function (gt, dt)
		-- update frustum
		frustum = buildFrustumFromCamera(camera, farPlaneZ, frustum)

		-- draw frustum edge(s) gizmo
		if showFrustum then
			lines = drawEdges(frustum.points, FRUSTUM_EDGES, lines, function (obj)
				lineMaid:addTask(obj)
			end)
		end

		-- draw frustum box gizmo
		if showBox then
			box = computeFrustumBox(frustum, box)

			drawBox(box, maid._box, function (obj)
				maid._box = obj
			end)
		end

		-- test whether our box/sphere is inside
		-- or outside the viewing frustum
		local boxInFrustum = testFrustumOBB(frustum, obbOrigin, obbExtents)
		someTestBox.BrickColor = boxInFrustum and BrickColor.Green() or BrickColor.Red()

		local sphereInFrustum = testFrustumSphere(frustum, spherePosition, sphereRadius)
		someTestSphere.BrickColor = sphereInFrustum and BrickColor.Green() or BrickColor.Red()
	end)
	maid:addTask(runtime)
end

-- start the scene
createDebugScene()

[Crimson_VID (super_paper)]

I truly appreciate all the assistance you’ve provided so far. However, I’m wondering if there might be a simpler approach to this. I don’t really want a updating display on the camera because I want the picture to be a surprise.

[DevSynaptix]

Mine only updates to show you how it all works, for your case you wouldn’t do that; it’s just so that you can get a better understanding of what the frustum actually is and how to use it.

In your case, the idea would just be to:

1. Player does some action, e.g. clicks a button, to take a picture
2. Broad-phase tests
   • You perform a spatial query to get the relevant objects
   • You do this either by (a) querying some data structure, e.g. an octree; or (b) by using WorldRoot:GetPartBoundsInBox / WorldRoot:GetPartBoundsInRadius
3. Narrow-phase tests
   • You compute the camera frustum (though this could be done earlier and be used in the broad-phase as you could do AABB tests on the Octree and the Frustum)
   • You iterate through the object(s) and find those that are within the frustum of the camera
   • For those that meet this criteria, you clone them and insert them into the ViewportFrame
4. Finalise
   • Finish up your ViewportFrame, e.g. add the skybox using a method described above; and/or do any other required finishing touches (e.g. do you want the character’s captured in the picture to be posed the same as when captured etc)

Though, again, this is just the easiest way to do it if you were worried about lag. If you’re not that concerned with it potentially stuttering when capturing then you could skip the narrow-phase step and just perform the broad-phase step.