How to do raytraced reflections?

I spent a WEEK looking for a way to do this, I tried loads of things:

  1. Using viewportFrames
  2. Making a part use tons of Pixels then make raycasts out of them.
  3. Using BoatBomber’s viewport Handler.

I need help pls.

This is not really a clear question to me. What I think you try to accomplish is a realistic looking mirror , correct?

No like, make surface reflections a thing like this:

Control: Multiple Stunning Ray-Traced Effects Raise The Bar For Game  Graphics | GeForce News | NVIDIA

image

You see how the floor reflects objects.

Or maybe a more precise example, EthanTheGrand achieved what I wanted to achieve:

This is what i wanted to achieve, making a raytracing rendering system that reflects surfaces around a reflective part in realtime.

Took me a week to trying to figure out a method of doing this.

Looks to me like a bunch of blurry pixels. Figure out the reflected direction from the camera to the pixel and use that to raycast.

local function GetReflectedRay(camera: Vector3, pixel: Vector3, normal: Vector3)
  local dir = (pixel - camera).Unit
  local reflected = dir - 2 * normal * normal:Dot(dir)
  return Ray.new(pixel, reflected * 100)
end

On Roblox, we have the ability to make surfaces highly reflective. But they don’t actually reflect objects around said surface. I want to make it do exactly that.

Imagine a part with reflectance 1 that reflects objects in realtime by rendering the environment around it every 1/24th of a second, updating objects and reflecting it on the surface.

remember: Viewport Frames are a terrible option

If you look in the comments of the video, you can see that every ‘mirror surface’ raycasts and makes a surfaceGUI with imagelabels. It’s at a low resolution however and even then I can imagine it runs poorly.

The closest thing Roblox has to proper surface reflections are water reflections. Using these, combined with a transparent floor you can get something that kind of resembles a surface reflection.

If you want actual clean reflections, you can get rid of the water distortion by deleting the water textures in your roblox folder, but this will not be shared across clients, so only you can see it.

3 Likes

I get you. I made something like this before: Collection - Roblox

(Press “E” once you’re in the game to switch to the mirror)

This uses surface GUI triangles. The video you showed would be a little simpler

Why do i think its not as simple as that…

but either way I do have a module that converts Udim2 to Vector3, my problem is the accuracy of said module, It sometimes makes the vector3 the same thing for every pixel. So the next 2 questions are:

  1. How is it possible to get a vector3 version of a pixel’s position?

  2. I noticed that shadows are being rendered in the mirror of the video, I guess thats a whole different thing to figure out?

But I do get the basics, such as get the part’s color so that it can be distinguish, but the problem is that it looks pretty 2D from my earlier tries compared to EthanTheGrand’s. So I don’t really know how he made it so 3D and smooth.

This is my curiosity going wild, so I kinda smash my head whenever im trying to figure it out.

and finally, what is the normal variable in that code…

  1. Something like this, assuming your mirror/screen is on the front face of the part
-- x: pixel's X position, 0 for first column
-- y: pixel's Y position, 0 for first row
-- w: number of pixels across
-- h: number of pixels height
local function GetPixelPosition(part, x, y, w, h)
  local size = part.Size
  local scaledX = (x - 0.5) / w * size.X
  local scaledY = (y - 0.5) / h * size.Y
  return part.CFrame * Vector3.new(size.X / 2 - scaledX, size.Y / 2 - scaledY, -size.Z / 2)
end

That would give you the 3D position for the top-left of a given pixel. To get the center just use (x + 0.5) instead of x in the scaledX (same for y).

  1. normal means “the surface normal” i.e. what direction the pixel is facing in world space.

So if your pixel was on the front face of a part that would be normal = part.CFrame:VectorToWorldSpace(Vector3.new(0, 0, -1))

Edit: Added / 2 to the Size.Z in the last line
Edit2: Fixed which way is up

Took a crack at this if you’re still interested!

It gets real laggy real quick :slight_smile:

But the effect is kinda neat. The comments under the video you linked to reveals that the creator is using a 15x15 resolution with blurred image labels as the individual pixels, so you could do something like that to improve its look.

I implemented it as a module, and broke up the major functions. Most of it was just what I posted before. Should be relatively easy to read, but lmk if you have more questions about it.

You create a Mirror object by assigning it to a part and calling Update as frequently as you like:

local Mirror = require(game.ReplicatedStorage:WaitForChild("Mirror"))

local mirror = Mirror.new(workspace.Part, {
	width = 50,
	height = 50,
})

game:GetService("RunService").Stepped:Connect(function()
	mirror:Update()
end)

And the module script - there’s a few TODOs sprinkled around that are left as an exercise for the reader :slight_smile:

local Mirror = {}
Mirror.__index = Mirror

type MirrorOptions = {
	width: number?,  -- number of pixels x resolution (default 10)
	height: number?, -- number of pixels y resolution (default 10)
	cutoff: number?, -- length of individual raycasts (default 100)
	raycastParams: RaycastParams? -- params to use when raycasting (default RaycastParams.new())
}

-- creates a mirror, possibly with some options
function Mirror.new(part: BasePart, options: MirrorOptions?)
	local options: MirrorOptions = options or {}
	
	local self = {
		_part = part,
		_gui = Instance.new("SurfaceGui") :: SurfaceGui,
		_pixels = {} :: {{Frame}},
		_face = Enum.NormalId.Front, -- TODO could make this configurable, would require changes to _GetPixelPosition
		_width = options.width or 10,
		_height = options.height or 10,
		_cutoff = options.cutoff or 100,
		_raycastParams = options.raycastParams or RaycastParams.new()
	}
	
	self._gui.Parent = self._part
	self._gui.Face = self._face
	self._gui.CanvasSize = Vector2.new(self._width, self._height)
	
	self._pixels = table.create(self._width * self._height)
	
	-- initialize pixels table with a bunch of Frames
	for x = 1, self._width do
		local column = table.create(self._height)
		self._pixels[x] = column
		
		for y = 1, self._height do
			-- TODO could be ImageLabels with a blur or something
			local frame = Instance.new("Frame")
			frame.Size = UDim2.fromOffset(1, 1)
			frame.Position = UDim2.fromOffset(x - 1, y - 1)
			frame.BackgroundColor = BrickColor.Random()
			frame.Parent = self._gui
			frame.BorderSizePixel = 0
			
			column[y] = frame
		end
	end
	
	return setmetatable(self, Mirror)
end

-- destroys the gui, that's all
function Mirror:Destroy()
	local self: Mirror = self
	self._gui:Destroy()
end

-- updates all the pixels
function Mirror:Update()
	local self: Mirror = self
	
	-- TODO make this more interesting
	local skyColor = Color3.new(0, 0, 0)

	for y = 1, self._height do
		for x = 1, self._width do
			self:_UpdateOnePixel(x, y, skyColor)
		end
	end
end

function Mirror:_UpdateOnePixel(x: number, y: number, skyColor: Color3)
	local self: Mirror = self

	local frame = self._pixels[x][y]
	local camera = workspace.CurrentCamera
	local pixel = self:_GetPixelPosition(x, y)
	local dir = self:_GetReflectedDir(camera.CFrame.Position, pixel) * self._cutoff
	local result = workspace:Raycast(pixel, dir, self._raycastParams)
	if result then
		frame.BackgroundColor3 = result.Instance.Color
	else
		frame.BackgroundColor3 = skyColor
	end
end

function Mirror:_GetReflectedDir(from: Vector3, onSurface: Vector3)
	local self: Mirror = self
	
	local normal = self._part.CFrame:VectorToWorldSpace(Vector3.fromNormalId(self._face))
	local dir = (onSurface - from).Unit
	return dir - 2 * normal * normal:Dot(dir)
end

-- returns center of a pixel in world space
function Mirror:_GetPixelPosition(x: number, y: number)
	local self: Mirror = self
	
	local size = self._part.Size
	local scaledX = (x - 0.5) / self._width * size.X
	local scaledY = (y - 0.5) / self._height * size.Y
	return self._part.CFrame * Vector3.new(size.X / 2 - scaledX, size.Y / 2 - scaledY, -size.Z / 2)
end

-- hack to type 'Mirror' correctly
type Mirror = typeof(Mirror.new((nil :: any) :: BasePart))

return Mirror
2 Likes

Well Shoot, You did a nice job ngl, but idk about copying this code. I feel like its illegal.

Either way, i’ve recently learnt how to make custom dynamic shadows. But honestly i couldnt thank you more.

can I copy it and take a spin off it?

Damn now, I would love to make this code more performant and etc, then open source it.

The conclusion is, You taught me how to make raytraced reflections, not only that you also gave me a semi proper insight on how raytracing works. Thanks!

Do whatever you want! Copy it, modify it, learn from it—don’t need to credit me, go wild

Inconsequential but this line should just be

self._pixels = table.create(self._width)

Saves some memory I guess

Deletes Whole Post

jk, maybe ill leave this open for a while or something.

I also found a post by the original creator and posted a question if you’re curious

Alright thanks, forgot to mention that this is part of the Voxel Overhaul Project that im working on. Making a lighting system built on top of Voxel with new features.
Here’s the Post:

Still thinking of more features to add btw.

Alright, with a refresh rate of 24 (which is what i added for performance) and quality of 65x65, it can run pretty well. The problem is that it isnt smooth, but that can be fixed if i can make the refresh rate variable, which i can do.

On my crappy laptop, It worked wonderfully, I was able to maintain atleast 50-60 FPS. Again I plan on making the reflection FPS variable. The limit of the FPS of the reflection is 30 FPS which is half of the 60 FPS limit (however if the FPS cap has been bypasssed then the limit will change to half of the player’s current FPS).

Meaning if the player’s FPS becomes 10 FPS, then the FPS of the reflections become 5 FPS.

This is to prevent the roblox engine from constantly calculating the reflections even if the player’s FPS is being murdered.

The cost of smoothness for performance… its fine.

This is just 10% of the bunch of optimizations im gonna implement,

Next step is well… Variable mirror Quality (Reduce the Quality whenever needed, like a youtube Video). Then, Range based rendering, making the mirror stop updating after not being in a certain range, so on and so forth.

I think 65x65-80x80 are ok for any device, are they arent laggy, but that’s if you limit how frequently a new frame is rendered.

the reason why it’s laggy is cause your iteration lookup takes two calls (two indexing) so it is o(n^2) in the space complexity. Just make it a batched array and also you are creating a color3 every update which can stress the gc. It doesn’t help also that you are updating it on stepped lol. change that to heartbeat and i guarantee you will get double performance

ohhh my gahhh, its CoderHusk! the epic programmer. Sure! I’ll try everything you suggested thank you so much!