Issue Making a 3d dragger

Hello, I’m trying to make a 3D object grabbing system similar to the one in Lumber Tycoon 2 by using DragDetectors. So far, I’ve been working on getting the position right, however I’ve stumbled upon an issue where depending on what’s behind the object I’m grabbing, the object will suddenly go up. On the following script that refers to the custom drag function in the drag detector, you can see that I’m using a combination of the ray itself that the function provides me, with the result of a raycast. I’ve tried multiple solutions such as not using a raycast result at all, however this prevents me from dragging the part behind me. I have run out of ideas on how to approach this, and I’ll gladly accept any help on this matter.

this is a gif of the issue i’m having:

https://gyazo.com/eb6624f68c3d823a975efa0ab620845a

and this is the current custom function i’m using

function drag(ray: Ray) : CFrame
    local player = game:GetService("Players"):FindFirstChild("iniciantinroblox").Character -- i'm using this cuz i still haven't coded a way for the function to know which player is grabbing the object
    local playerPos = player.HumanoidRootPart.Position
    local part = script.Parent.Parent
    local params = RaycastParams.new()
    params.FilterType = Enum.RaycastFilterType.Exclude
    params.FilterDescendantsInstances = {player, part}
    local result = game:GetService("Workspace"):Raycast(ray.Origin, ray.Direction, params)
    local hit
    local increment = Vector3.new(0, 1.5, 0)
    local dist = 10
    if not result then
        hit = ray.Origin + ray.Direction
    else
        
        local t_dist = (result.Position - playerPos).Magnitude
        if t_dist < dist then
            dist = t_dist
            
        end    
        hit = result.Position--ray.Origin + ray.Direction
    end
    
    local dir = CFrame.lookAt(playerPos, hit).LookVector.Unit
    local newCf = CFrame.new((playerPos + dir * dist + increment))
    
    return newCf
    
end

I think it has something to do with your increment vector.