Placement system issue?

Hello! I am currently working on adding a feature to allow “stackable” objects on all axis for my placement module. What I mean by that is similar to Minecraft’s placement system where you can place objects in the air horizontally without the object being within/too far from the other object. This allows for bridging, roof building and just easier placement overall.

What I want:

What I get:

Currently my system only allows for vertical stacking (y) and that is calculated in the following function:

-- tp = to position, ts = to size, o = model size
local function calculateYPos(tp, ts, o)
    return (tp + ts*0.5) + o*0.5

I tried using the same function to calculate the offset just on another axis but it didn’t end up working as I want.

-- Calculates the stacked positions and returns them
local function calcStack(r)
	local px, py, pz = 0, 0, 0
	local t = mouse.Target

	if mouse.TargetSurface == Enum.NormalId.Top then -- vertical
		px = 0
		py = calculateYPos(t.Position.Y, t.Size.Y, primary.Size.Y)
		pz = 0
	elseif mouse.TargetSurface == Enum.NormalId.Front  then -- non vertical
		py = t.Position.Y + (t.Size.Y*0.5 - primary.Size.Y*0.5) -- sets the object on the same y
		pz = calculateYPos(t.Position.Z, t.Size.Z, primary.Size.Z) -- should calc the z

	return px, py, pz
If you're interested, my main calculate function looks like this:
local function calculateItemLocation() : CFrame
	-- Changes y depending on mouse target
	if stackable and mouse.Target then
		if mouse.Target:IsDescendantOf(placedObjects) or mouse.Target == plot then
			sx, y, sz = calcStack()

	if currentRot then
		cx = primary.Size.X*0.5
		cz = primary.Size.Z*0.5
		cx = primary.Size.Z*0.5
		cz = primary.Size.X*0.5

	x, z = mouse.Hit.X - cx - sx, mouse.Hit.Z - cz - sz

	if moveByGrid then
		-- Calculates the correct position
		local pltCFrame = cframe(plot.CFrame.X, plot.CFrame.Y, plot.CFrame.Z)
		pos = cframe(x, 0, z)
		pos = snapCFrame(pltCFrame:Inverse()*pos)
		finalC = pos*pltCFrame*cframe(cx, 0, cz)
		finalC = cframe(x, y, z)*cframe(cx, 0, cz)

	finalC = bounds(finalC)

	-- Clamps y to a max height above the plot position (assuming removePlotDep = false)
	y = clamp(y, initialY, maxHeight + initialY)

	return finalC*anglesXYZ(0, rot*pi/180, 0)

The reason why this doesn’t work is that I am using NormalId’s which should work except that models can be rotated. When a model is rotated, it wont work. Is there anyway I can make this work regardless of rotation? I am out of ideas so all help is greatly appreciated!