LinearVelocity "stuttering" when going down slopes

Hello! Recently, I created a slope sliding system, but I’ve encountered an issue where gaining too much speed causes an issue where a weird jitter happens when gaining too much momentum. I did try removing the collision from the player but that didn’t help with the issue.

Currently, the system works by putting a force that is (hopefully) perfectly parallel with the slope angle. I’ve never worked with physics a lot so physics based issues are new to me.

if not updateVelocity then
	local SurfaceTangent = DownwardsRaycast.Normal:Cross(Vector3.yAxis)
	local SlopesDirection = -(SurfaceTangent:Cross(DownwardsRaycast.Normal).Unit)

	PushForce.VectorVelocity = SlopesDirection * HRP.AssemblyLinearVelocity.Magnitude
	PushForce.Enabled = true
	updateVelocity = true

	--Hum.WalkSpeed = 0
	--Hum.AutoRotate = false
	--Hum:SetStateEnabled(Enum.HumanoidStateType.Jumping, false)
else
	local AngularBoost = math.lerp(BASE_SPEED, MAX_SPEED, DownwardsRaycast.Normal:Dot(Vector3.yAxis))
	PushForce.VectorVelocity = lastVelocity:Lerp(lastVelocity * AngularBoost, DeltaTime * 2)
end

Well I found the issue, my checks above the update velocity part were turning it off