LinearVelocity not working well with Raycasting collisions

Hello, I am trying to implement a very simple ball that bounces around and spins with slight gravity. I have all of the code functional, but since I am currently updating the positions manually, the balls have choppy movement in-game.

To fix this, I tried implementing LinearVelocity which was pretty straightforward. There is a big issue that I cannot get an accurate value for the velocity of the ball. When updating manually, I can use deltaTime and the speed of the ball for this calculation to work perfectly, but since physics steps are more fine than render steps it leads to bad collisions constantly, and the ball will travel through walls inevitably…

I have not tried calculating the velocity with a smoothed average, which seems like an obvious yet expensive fix, but I was wondering if there was a way this is traditionally handled?

Here is the code that controls my balls. The two commented lines of code were my attempt at making the LinearVelocity control the positioning (this code is placed in a PreSimulation connection):

local ballPart : BasePart = temariBall.part

-- local linearVelocity : LinearVelocity = ballPart:FindFirstChild("LinearVelocity")

local angularVelocity : AngularVelocity = ballPart:FindFirstChild("AngularVelocity")



-- gravity
temariBall.velocity -= Vector3.new(0, 0.1, 0) * deltaSeconds

-- normalize velocity value
temariBall.velocity = temariBall.velocity.Unit




local ballRealVelocity = temariBall.velocity * temariBall.speed * deltaSeconds

local updatedPosition = ballPart.Position + ballRealVelocity



local raycastParams = RaycastParams.new()

raycastParams.FilterType = Enum.RaycastFilterType.Exclude

raycastParams.FilterDescendantsInstances = { ballPart }

raycastParams.RespectCanCollide = true


local raycastResult = workspace:Raycast(updatedPosition, temariBall.velocity*ballPart.Size.X*0.5, raycastParams)

if raycastResult then -- the ball should reflect

	local incident = temariBall.velocity

	local normal = raycastResult.Normal


	-- reflection	
	temariBall.velocity = incident - normal:Dot(incident) * 2 * normal

	-- speed growth factor with maximum speed
	temariBall.speed = math.min(temariBall.speed * 1.1, MaximumSpeed)


	-- angular velocity approximation (it is the vector orthogonal to both the normal and the unit velocity projected onto the normal)

	local projectedVelocity = normal:Dot(temariBall.velocity) * normal - temariBall.velocity

	angularVelocity.AngularVelocity = projectedVelocity.Unit:Cross(normal) * projectedVelocity.Magnitude * math.map(temariBall.speed, InitialSpeed, MaximumSpeed, 1, 12.57)

	return
end

-- manually update position of the ball

-- linearVelocity.VectorVelocity = temariBall.velocity * temariBall.speed

ballPart.Position = updatedPosition