How to make a server sided part collide with a client sided character

The green floating part following the player within the video depicts the server character (this is non collide-able btw and yes this is 30 ping), which is the player visible to everyone else. The ball halts upon contacting the server-sided character. However for the client player, it appears as though the ball has stopped on its own. The ball is server-owned (:SetNetworkOwner(nil)) for security purposes. Is there any way to make this look better via Lag Compensation, Extrapolation (using velocity and trajectory) or another way keeping the network owner nil? (This can be applied on the ball or player)

So what if the server sided ball hits the client character first? The ball will simply just push the player out the way not effecting the balls trajectory or velocity.

What if this is the other way around and the player does have ownership of the ball? The ball will collide with the client character and ignore the server character (which is what Iā€™m looking for) but for other players it will stop by its self which is just as bad + its a big security issue.