How can I make AlignOrientation more rigid feeling?

I wrote a VR hands and tool system that uses AlignOrientation and AlignRotation to handle server-side positioning of hands and tools. Currently, it works, but the aligns like to freak out sometimes, and when they are calm, they feel sluggish/unresponsive.

(when I say freak out, I mean the hands will start to rapidly move about before settling down into a usable state.)

Ive tried to enable Rigidity on all of the aligns, but this just causes more of that freaking out behavior.

Video of the behavior I’d like to remove:

CFrame them instead on the server and perform magnitude checks periodically to see if the hands are positioned correctly on the client. AlignOrientations don’t work too well when done on the server

How about using WeldConstraint instead?