Problem with making shadows

Oh shadows… I’ve made a system that that can raycast and build up a shadow out of tris.

As a performance optimisation I’ve relied on not raycasting thousands of times to build up an image, this is how the system is built up to allow for real time shadows:

  1. Get the vertices of the cube, that means we raycast 8 times
  2. After that depending on where all raycasts hit the script builds up a “2D” shape that represents the shadow out of tris (tris being a polygon made up of 3 vectors basically being a 3D triangle)

After that we get this decently good result:

Oh noes! There’s a complicated bug!

This is pretty self explanatory what’s causing this but the problem lies in the whole system itself, I need guidance of how I could solve the issue. To solve this I’m going to have to be subdividing triangles which is not something I have much of experience on, also that somehow I need to make an algorithm that finds the places to “cut” the triangles at maybe even at multiple places!

Code
local CubeShadowPositions = {Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero} --

local Wedges = {}

local wedge = Instance.new("WedgePart");
wedge.Anchored = true;
wedge.Color = Color3.new(0.231006, 0.231815, 0.234257)
wedge.TopSurface = Enum.SurfaceType.Smooth;
wedge.BottomSurface = Enum.SurfaceType.Smooth;
wedge.Material = Enum.Material.Neon
wedge.CanCollide = false
wedge.CanTouch = false
wedge.CanQuery = false

local rot = Vector3.new()

local function draw3dTriangle(a, b, c, parent)
	local ab, ac, bc = b - a, c - a, c - b;
	local abd, acd, bcd = ab:Dot(ab), ac:Dot(ac), bc:Dot(bc);

	if (abd > acd and abd > bcd) then
		c, a = a, c;
	elseif (acd > bcd and acd > abd) then
		a, b = b, a;
	end

	ab, ac, bc = b - a, c - a, c - b;

	local right = ac:Cross(ab).unit;
	local up = bc:Cross(right).unit;
	local back = bc.unit;

	local height = math.abs(ab:Dot(up));

	local w1 = wedge:Clone();
	w1.Size = Vector3.new(0, height, math.abs(ab:Dot(back)));
	w1.CFrame = CFrame.fromMatrix((a + b)/2, right, up, back);
	w1.Parent = parent;

	local w2 = wedge:Clone();
	w2.Size = Vector3.new(0, height, math.abs(ac:Dot(back)));
	w2.CFrame = CFrame.fromMatrix((a + c)/2, -right, up, -back);
	w2.Parent = parent;

	table.insert(Wedges, w1)
	table.insert(Wedges, w2)


	return w1, w2;
end

game:GetService("RunService").Heartbeat:Connect(function()
	for i = 1, #Wedges do
		Wedges[i]:Destroy()
	end
	table.clear(Wedges)
	
	for i,v in pairs(game.Workspace:GetDescendants()) do
		if v:IsA("BasePart") then
			--print(v.Name, v.Position)
			CubeShadowPositions = {Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero}
			local rayparams = RaycastParams.new()
			rayparams.FilterType = Enum.RaycastFilterType.Exclude
			rayparams.FilterDescendantsInstances = {v}	
			if v.Parent:IsA("Model") then
				rayparams.FilterDescendantsInstances = {v.Parent:GetDescendants()}	

			elseif v.Parent:IsA("Accessory") then
				rayparams.FilterDescendantsInstances = {v.Parent.Parent:GetDescendants()}	
			end
			
			local ray = game.Workspace:Raycast((v.CFrame * CFrame.new(v.Size.X/2, v.Size.Y/2, v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray then
				CubeShadowPositions[5] = ray.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(-v.Size.X/2, v.Size.Y/2, v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[6] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(v.Size.X/2, -v.Size.Y/2, v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[7] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(v.Size.X/2, v.Size.Y/2, -v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[1] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(-v.Size.X/2, -v.Size.Y/2, v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[8] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(v.Size.X/2, -v.Size.Y/2, -v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[2] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(-v.Size.X/2, v.Size.Y/2, -v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[3] = ray2.Position
			end
			local ray2 = game.Workspace:Raycast((v.CFrame * CFrame.new(-v.Size.X/2, -v.Size.Y/2, -v.Size.Z/2)).Position + game.Lighting:GetSunDirection().Unit*1, -game.Lighting:GetSunDirection().Unit*60,rayparams)
			if ray2 then
				CubeShadowPositions[4] = ray2.Position
			end
			if CubeShadowPositions[1].X == 0 and CubeShadowPositions[2].X == 0 and CubeShadowPositions[3].X == 0 and CubeShadowPositions[4].X == 0 and CubeShadowPositions[5].X == 0 and CubeShadowPositions[6].X == 0 and CubeShadowPositions[7].X == 0 and CubeShadowPositions[8].X == 0 then
			else
				
				draw3dTriangle(CubeShadowPositions[1],CubeShadowPositions[2],CubeShadowPositions[3], v)
				draw3dTriangle(CubeShadowPositions[4],CubeShadowPositions[2],CubeShadowPositions[3], v)
				draw3dTriangle(CubeShadowPositions[5],CubeShadowPositions[6],CubeShadowPositions[7], v)
				draw3dTriangle(CubeShadowPositions[8],CubeShadowPositions[6],CubeShadowPositions[7], v)
				draw3dTriangle(CubeShadowPositions[1],CubeShadowPositions[2],CubeShadowPositions[5], v)
				draw3dTriangle(CubeShadowPositions[7],CubeShadowPositions[2],CubeShadowPositions[5], v)
				draw3dTriangle(CubeShadowPositions[4],CubeShadowPositions[3],CubeShadowPositions[8], v)
				draw3dTriangle(CubeShadowPositions[6],CubeShadowPositions[3],CubeShadowPositions[8], v)
				draw3dTriangle(CubeShadowPositions[4],CubeShadowPositions[2],CubeShadowPositions[8], v)
				draw3dTriangle(CubeShadowPositions[7],CubeShadowPositions[2],CubeShadowPositions[8], v)
				draw3dTriangle(CubeShadowPositions[3],CubeShadowPositions[1],CubeShadowPositions[6], v)
				draw3dTriangle(CubeShadowPositions[5],CubeShadowPositions[1],CubeShadowPositions[6], v)
			end
		end
	end
end)

How could I go about making this algorithm? Plus to not crash the players pc?

edit: If there is a better and more lag efficient way of doing this please inform me about that : )

2 Likes

Seems like I’m gonna have to solve this myself…

Well I’ve come up with two good ways to detect if a shadow will generate not as intended.

Corners and edges. If I get the normal unit of the ray hit and compare to all different units of different rays of the part I can safely make an algorithm that can know when something is not right. Basically if all units are not the same the algorithm will have to then find the exact position of where the corner exist

Cliffs and steep faces. To detect this I will just have to have a threshold there if a ray travels x distance further then all other rays it can safely say that the shadow is at a steep edge or cliff.

Look into how mesh decals are made outside of Roblox. It might help. Theres an example somewhere here Introduction to Decal Rendering.

1 Like

Screenshot 2024-05-31 at 16.22.45

After browsing quickly through the text I still don’t really understand how you’re supposed to do this,

I can’t just get the scene depth since Roblox doesn’t support that and even if I choose to make one myself that would require thousands of raycast. The post did although help me understand how others outside of Roblox would do this

Theres two methods: the shader method (needs depth) and the mesh method (requires vertex positions) Im asumming your entire map is just a bunch of cubes, so getting those shouldnt be too hard.

The link also has a unity implementation of it. driven-decals/Runtime/MeshProjection.cs at master · Anatta336/driven-decals · GitHub

New Shadow Projection Plan

Inspired by the system previously discussed, I have a new and improved approach for shadow projection. This plan involves creating a polygon instead of projected boxes, making it more adaptable to different meshes and planes.

Algorithm Steps:

  1. Raycasting:
  • Raycast from each vertex position in the direction opposite to the sun.
  1. Detecting Corners:
  • If one of the ray’s normal values differs from the rest, use the “binary search” algorithm to find the exact point where the corner is.
  1. Sorting Points:
  • Once all points are identified, sort these positions correctly to ensure the shadow shape is accurate.
  1. Polygon Triangulation:
  • Triangulate the polygon and then build the shadow using the draw3DTriangle function.