Bullet going random direction up close

I use raycasting to make my bullets work, when am a decent space from a part/dummy it will hit the surface, if am too close the bullet will start going in a different direction like straight up and avoid the part/dummy.

this is my cast script which am not sure if its this that is causing the problem

		function module.cast(origin, endposition, velocity, damage)

					local Bullet = game.ReplicatedStorage.Misc.Projectiles.Default:Clone()
					Bullet.Parent = game.Workspace
					Bullet.Anchored = true
					Bullet.CanCollide = false
					Bullet.CFrame = CFrame.new(origin, endposition)

				local Loop

				Loop = game:GetService("RunService").RenderStepped:Connect(function(dt)
						Bullet.CFrame *= CFrame.new(0, 0, -velocity * (dt * 60))
						local Hit = workspace:Raycast(Bullet.Position, Bullet.CFrame.LookVector * velocity * 1.5)

						if Hit then
								if Hit.Instance.Parent:FindFirstChild("Humanoid") and damage ~= nil then
										print(Hit.Instance)
										if Hit.Instance.Name == "Head" then
												damage:FireServer(Hit.Instance.Parent, 20)
										else
												damage:FireServer(Hit.Instance.Parent, 10)
										end
								else
										Loop:Disconnect()
										Bullet:Destroy()
								end
						end

						if (Bullet.Position - origin).magnitude > 5000 then
								Bullet:Destroy()
								Loop:Disconnect()
						end
						
				end)
		end