How would I go about making the equivalent of Viewport Point to Ray on the server side?

Help and Feedback Scripting Support
,
#1

  1. What do you want to achieve?
    I am creating a puzzle game like Viewfinder that also involves wands and manipulating objects within photographs. In order for players to interact with objects within photographs, I need to raycast as if the photographs’ viewports were a 3d space (though they will always appear 2d from the outside). This is similar to client sided screen point to ray functions. The server knows all information regarding the viewports and their cameras.

  2. What is the issue?
    I began creating a custom raycast function and quickly found out that I didn’t really understand how the point to ray functions worked. I am currently finding the offset of the raycast’s position on the viewport frame from the frame’s center. Then, I calculate the distance from the viewport’s camera attachment necessary for the viewport frame to fit inside a cone with an angle of the camera’s field of view. Finally, I am adding the offset to that point away from the camera attachment to get a direction point. I raycast from the camera attachment towards the direction point. This of course, does not work.

  3. What solutions have you tried so far?
    I have looked up several posts regarding how similar functions to viewport point to ray work, but they didn’t provide an adequate explanation for me.

Here is a video of it not working:

Here is my current code:

for i,v in pairs(viewports) do
	if v.part == ray.Instance and v.face == GetEnumNormalFromRay(ray) then
		local fcf = CFrame.new(lOrigin,ray.Position)*CFrame.new(0,0,-1*(ray.Position-lOrigin).Magnitude)
		local acf = (fcf:ToObjectSpace(IDtoNormalCF(v.part,v.face)*CFrame.Angles(0,math.pi,0))):Inverse()
		--acf (my calculations for this are correct btw) is a cframe offset between
		--     the center of the viewport facing towards the center of the viewport's part
		--     the cframe of the raycast's position with the raycast's direction
						
			local length = getWidthLengthFromFace(v.part,v.face).length
			local width = getWidthLengthFromFace(v.part,v.face).width
			local diagonal = math.sqrt((length/2)^2+(width/2)^2)
			--the diagonal is the distance of the viewport part's face's rectangle's diagonal
						
			local ViewCF = v.attachment.WorldCFrame*CFrame.new(0,0,-1*math.tan(math.rad(90-(v.fov/2)))*diagonal)
			--the camera attachment's cframe moved forward by
			--the distance required to fit a rectangle with the diagonal length
			--in a cone with the viewport's field of view
						
			local rayCF = CFrame.new((v.attachment.WorldCFrame).Position,(ViewCF*acf).Position)*CFrame.new(0,0,-1*((ViewCF*acf).Position-v.attachment.WorldPosition).Magnitude)
			--NOTE: the origin should technically be at the viewport camera's attachment, but I moved to forward for visualization purposes
						
			local NewOrigin = rayCF.Position
			local NewEndPoint = (rayCF*CFrame.new(0,0,-1000)).Position
			nextRay = workspace:Raycast(NewOrigin,NewEndPoint-NewOrigin,raycastParams)
			lOrigin = NewOrigin
			lEnd = NewEndPoint
		end
	end