Very Far Objects Being Considered For Culling

My game uses a custom culling system which simply culls objects in or out of view when they are deemed too far from the camera. To make the client stop rendering said objects, I move them as far away from the origin as possible to a specific point. When I turn the camera in certain directions, it is causing the game to lag (see image below). My current workaround is to simply move the objects out of Workspace to prevent them from being rendered, but this is slower than moving the objects using BulkMoveTo. When there are quick changes to the amount of objects my own culling system wants to cull in, it can be pretty slow to change parents of a lot of objects. This issue was nonexistent before March 25th based on player activity, which I believe is the date that culling was enabled for all games (possibly through FFlags?).

This script will generate a bunch of parts as far from the origin as possible, then point the camera in a direction that causes the game to lag.

local Camera = workspace.CurrentCamera
local Folder = Instance.new("Folder")
Folder.Parent = workspace

local Part = Instance.new("Part")
Part.CFrame = CFrame.new(math.huge, math.huge, math.huge)
Part.Anchored = true
Part.Parent = Folder

for i=2,30_000 do
	Part:Clone().Parent = Folder
end

Camera.CameraType = Enum.CameraType.Scriptable
Camera.CFrame = CFrame.Angles(0, math.pi/3, 0)

Here is an image of the amount of time the engine’s culling system is taking per frame:

Here is a place file I created to reproduce the issue:
CullingBugReport.rbxl (55.2 KB)

Expected behavior

I expect the culling system to not consider parts that are outside of the render distance, obstructed by other parts, and far enough to not even show as a pixel. Once again, this issue did not happen before the engine had culling enabled on my game.