The first video linked shows my gun system, and what happens when I try to use CFrame.Angles() to rotate my gun. The second video is a reference of what I want my system to look like
Here is my script:

local Offset = Vector3.new(math.rad(0), math.rad(0), math.rad(0))
local currentX = mouse.X
local currentY = mouse.Y
local X = 0
local Y = 0
local sens = 10
mouse.Move:Connect(function()
X = math.clamp(X - math.rad(cam.CFrame.LookVector.X - currentX) * sens, -0.25, 0.25)
Y = math.clamp(Y - math.rad(cam.CFrame.LookVector.Y - currentY) * sens, -0.25, 0.25)
Offset = Vector3.new(X, Y, 0)
currentX = cam.CFrame.LookVector.X
currentY = cam.CFrame.LookVector.Y
end)
run.RenderStepped:Connect(function()
gun:SetPrimaryPartCFrame(cam.CFrame * CFrame.Angles(Offset.Y, Offset.X, Offset.Z))
end)

(I am aware that the Y offset comes first in my CFrame.Angles, this is intentional because the gun was not rotating in the proper direction)

Have you tried CFrame.fromOrientation or changing the CFrame angle order?

CFrame.Angles(5,0,0)*CFrame.Angles(0,9,0)

Usually adding Z value to CFrame.Angles makes it weird because of the order, Z is applied first which messes up the X and Y rotation

CFrame.Angles ( number rx, number ry, number rz )
Equivalent to fromEulerAnglesXYZ

CFrame.fromEulerAnglesXYZ ( number rx, number ry, number rz )
Creates a rotated CFrame using angles (rx, ry, rz) in radians. Rotations are applied in Z, Y, X order.

Yes, I did change the order of the CFrame, but thatâ€™s not my issue. If you watch the 2 videos I sent above, there is a video of what my system looks like, and what I want it to look like

My reference video uses the exact script that I posted, including the CFrameâ€™s new order

Looking at it it seems to be similar to a mouse delta sway script except that it â€śstays in placeâ€ť with a certain angle limit.

First step is I would not use lookvector as its not based on mouse movement or delta rather on the current position of the camera.

I would probably start by adding a CFrame.Angles factor after the goal CFrame to make it stick in place rather than return back to its original position at center of screen.

LookVector is a vector, not an angle. Iâ€™m pretty sure thatâ€™s a big part of your problem. If you want to use XYZ rotations despite the many flaws, you can use CFrame.toEulerAnglesXYZ() to get the X Y Z angles of the CFrame.

XYZ rotations donâ€™t contain enough data to accurately describe rotation. You can run into little flaws like axes flipping under certain circumstances.

It can take a little bit to learn, but the preferred method is quaternions or, my personal favorite, rotation matrices. You can keep going the way youâ€™re doing it but youâ€™ll want to learn quaternions or rotation matrices at some point.

I was going to link you to a good resource for matrices that would have been simple enough to educate a 5 year old, but unfortunately Roblox deleted it. Your best bet is to do some googling and find a guide you can understand. If you want me to spend some time writing an easy-to-understand tutorial on it, send me a DM and Iâ€™ll get to it over the weekend when I have time.

Like I mentioned though, until you run into problems with XYZ rotation, itâ€™s fine to keep going until youâ€™re ready to learn one of these other methods.