Climbing Problems Depending On Wall Surface

So, I have this climbing mechanic and it’s going very well but the thing is depending if I start on the back, front, left, or right surface of the part it doesn’t work out to well, it works perfectly on the back side of the part but not the front and sides, any ideas what I should do to combat this problem?

Back Surface :

Front Surface :

This is the code when I raycast towards a wall in the first place :

UIS.InputBegan:Connect(function(input, isTyping)
	if isTyping then return end
	if input.KeyCode == Enum.KeyCode.R  and not Climbing and not debounceR then
		debounceR = true
		local climbRay = Ray.new(humrp.Position, humrp.CFrame.lookVector * 1)
		local hit, position, surface = game.Workspace:FindPartOnRayWithIgnoreList(climbRay, {humrp,head})
		local mag = (position - head.Position).Magnitude
		if hit and not Climbing then
			print(hit)
			print(surface)
			Climbing = true
			hum.PlatformStand = true
			print("Climbing Enabled.")
			local attachment = Instance.new("Attachment")
			attachment.Parent = hit
			attachment.WorldCFrame = (head.CFrame * CFrame.new(0,0,-mag))
			attachment.Visible = true
			attachment.Name = "WallClimb"
			head.CFrame = attachment.WorldCFrame
			humrp.Velocity = Vector3.new(0,0,0)
			local BodyVelocity = Instance.new("BodyVelocity")
			BodyVelocity.MaxForce = Vector3.new(5000,5000,5000)
			BodyVelocity.Velocity = Vector3.new(0,0,0)
			BodyVelocity.Parent = humrp
			BodyVelocity.Name = "Climb"
		end
		wait(1)
		debounceR = false
	end