Help! Ball has weird physics when touched

When I just walk over/touch my ball normally (nothing else), It starts spinning weirdly and has these weird physics bugs. Is there anything I can do to fix this?

Heres a video of how I want it to look like:
https://gyazo.com/775c4aca8df522939197d2bcdddbf64d

Heres a video of how it acts in my game:
https://gyazo.com/33d316306b709d4715c9cae817df4452

(You can even see at the end of the video it just starts spinning wildly)