This thread documents the progress of my raytracing project that I intend to use in my future work. The goal is to have scalable performance that allows both static rendering and realtime performance, with various tradeoffs depending on the use-case.
Original Post (27th January 2025):
After being inspired by RetroRaster (Strongly recommend if you want the most fully-fledged rendering solution outside of Roblox’s rasteriser), I wanted to educate and challenge myself on learning how to create my own rendering solution via raytracing.
Coming into this I had no prior graphics programming knowledge, so it definitely has been an experience for me so far. If you feel inspired to make your own raytracer, please do not follow my guidance as I am merely documenting my experience as I go along and create this system for the first time. I will probably make fundamental mistakes that you shouldn’t follow.
STEP 1: CORRECT OUTPUT
Since this project uses EditableImages, it was important for me to learn how to output correctly. Following a guide I was able to confirm the ability to correctly assign a colour to the intended pixel.
STEP 2: SEEING THE WORLD
Next we make use of game.Workspace:Raycast() in order to grab colour information. We now apply the EditableImage to an ImageLabel, and treat it like a cutout of the camera’s viewport. This gives us a ‘window’ into the world.
STEP 3: CORRECT PERSPECTIVE
While not shown in images, the previous method contained warping towards the edges of the screen. This was due to the FOV being linearly spread based on how far we were from the render’s midpoint. We had to apply the same math that roblox (and many engines) use for ensuring that the render didn’t do this ‘spherical’ warping.
STEP 4: REFLECTIONS
The hallmark of a raytracing engine is the ability to simulate light in a way that rasterisation just can’t. We implement that by bouncing and continuing the ray, then mixing the colours together in order to get reflective surfaces. For performance, some raytracers may apply a render distance, a maximum number of bounces, or both.
STEP 5: INFERRING MATERIAL PROPERTIES
Now that we know the raytracer can navigate the world properly, I then play around with higher resolution renders, allowing greater inspection of how the system is running. In this I also enabled the raytracer to infer reflections from the material it hits, and it currently terminates the ray if the reflection is 0. This is good for now, but this would need to change later on in regards to lighting.
STEP 6: FOG
Since our raytracer has a render distance, we can use that to apply fog to the world. Implementing this was tricky as now that we are recurring our rays based on reflections etc, any slight change in the logic drastically changes the output. After much trial and error, I was able to add fog in a believable manner, even to rays that are reflected multiple times.
STEP 7: LIGHTING, AMBIENT
Despite making all this progress, we have not implemented any lighting yet. This is a very delicate process, and would require much research on my end, but in the meantime multiplying the color of each ray via an ambient color allows us easy manipulation of the color of the environment. It’s worth noting at this point that I have provided an FPS counter in the project representing the FPS of this system, as it is able to run in realtime in lower resolutions. Methods for improved performance will be explored at a later stage.
STEP 8: LIGHTING, SHADOWS (Experimentation)
This step will most likely be rewritten once I have made progress on this step.
I experimented with shadows to see what the world would look like with them. During this I ran into issues of rays missing the parts that the shadow was meant to be visible on, and many recursive issues due to my ray termination condition being based on render distance vs bounce depth. However seeing this to me is very exciting and makes me want to work on the project more.
Looks good, though I wouldn’t really try graphics programming in roblox at the moment, there are too many limitations to actually make something decent, which is just sad, I have been wanting custom shader programming for a while, yet its probably not going to be possible since roblox would be needed to be ported onto the same graphics engine on all platforms.
I am accepting the possibility that realtime graphics may not be viable, but I do however see use-cases in areas such as dynamically generating thumbnails for ingame content (such as shop items or map thumbnails) without resorting to replicating instances within the datamodel then requiring a viewportframe to render that (adding more burden to the client GPU), or more commonly, manually spending magnitudes more time generating said thumbnail content.
Even without that, learning in a language (and an engine) that I am most familiar with allows me to focus purely on creating the system
Personally i’ve been making quite a few ray tracers on roblox. They have 0 use case so far. I get your idea for a potential use case but it’s hardly a thing usable then. Even with a parallel luau setup you would still be looking at sky high cpu costs for very little in return. Thumbnails would by default still look super low quality, flat and either super aliased or pixelated. You would need to spend ages to make a renderer which would look half competent and half usable only for the half usable renderer to provide performance in the minutes per frame on the even most powerful desktop cpus.
Past just playing around with ray tracers on roblox, i really dont recommend them using these things anywhere in production.
Thanks for the concern, but even if 0 usecase on Roblox was to be found, I would still get value out of this from an educational standpoint, and at a stretch legitimising a path for my own engine via CodeGen outside of Roblox.
Hearing that you’ve made your own raytracing systems, I’d love to be able to see what kind of results you came up with. If you’re interested, feel free to DM me.
Just thought I’d share how realtime performance is doing!
After implementing certain optimisations today I have been able to achieve ~100FPS at 160x90 while dancing in-front of two recursively reflecting parts! This is still being run as a single thread, so there will be a lot of performance to gain from implementing a parallel solution once other fundamentals are sorted.
Honestly super cool to see this being done within the Roblox engine! For a long time I believed it was near impossible to achieve systems like this in Roblox which is a 3d engine at its core, especially an entire graphics engine in itself. I don’t know if this is possible due to the addition of editable meshes, or my lack of graphical knowledge , but it’s honestly really amazing to see this come to life. I’m also curious; does this use GUI elements such as frames or viewport frames? I would assume so, but if there’s a different way, it’d be great to know. Overall I’m impressed with this project especially considering you said you don’t have previous graphical knowledge.
More tweaks today to make the performance more scalable! I can now render 288x162 at 60FPS+, which beats the previously posted video on both fronts! We’re still rendering from a single thread too, so this is going to scale reaaaally well.
WOW big improvement here this is incredible. Can I ask what changed? Also, the real time rendering is quite performant all things considered. I think this rendering style could DEFINITELY be used in some niche circumstances, like a game that would take advantage of the “artstyle” of it. Fantastic job!
raycasts are done in software, and is expensive. doing them too many times per frame will lead to poor performance, being more harshly on lower end devices. (when you’re developing roblox games, your first assumption should be that you’re developing for the low end)
fancy stuff such as lights, reflections, shadows and global illumination, requires more raycasts to be performed, lowering performance to something unplayable. even the most primitive implementations is enough to drag down fps by alot.
these problems wouldn’t matter if you’re doing this on a high end system. in fact this is probably what op is using, because i surely don’t expect my i5 7300u to render a huge magnitude of reflections at 1920x1146 pixels
Understandable! I’m not a graphics programmer so im definitely not the most qualified to comment on the viability of these systems, but managing a fairly stable 60 fps is impressive to me considering the rendering hes doing and hox expensive it is.
Anyways, cool project is all I wanted to say! Thanks for the reply
This is just a demonstration of what the renderer is capable of! I rendered the pretty high-res high-reflective 1024x1024 image in 5 seconds using 1 thread. So it’s the same rendering system but because we are no longer required to provide fast response times for gameplay purposes, we can take as long as we like rendering this.
There are more optimisations to be added and soon I will be exploring texture mapping as that is also important in bringing an environment to life.
I’ve been hesitant to implement multi-threaded support for reasons of unfamiliarity but also I do not want to give myself an easy ‘out’ for what could end up becoming poorly optimised code.
, but it’s honestly really amazing to see this come to life. I’m also curious; does this use GUI elements such as frames or viewport frames? I would assume so, but if there’s a different way, it’d be great to know. Overall I’m impressed with this project especially considering you said you don’t have previous graphical knowledge.
