I forgot to consider that you said that BodyGyro aims to align along the Y axis first, and I just realised that it wouldn’t do that with any simple vector-based algorithm. A simple vector based algorithm may take overly-long paths, but not curved (relative to the surface of a sphere) paths.
I did get the vector based algorithm working, and try and test it out in your place, but I gave up due to my internet getting slower and slower to the point where just starting studio took 3 or 4 minutes.
Alright, I think I’ve figured out the algorithm; this one is vector-based, has curved paths (particularly favouring correcting pitch before yaw, as observed), and requires all of the properties associated with BodyGyro with nothing more and nothing less.
For each axis, find the direction (+1 or -1) to rotate in that minimises distance to reach the goal for that axis.
Multiple that distance by P, capping it at the MaxTorque for that axis.
Then multiply by the direction.
Once you’ve done that for all axes, oppose the applied force and all current rotation with the drag.
I really don’t feel like testing this since in practice the CFrame and Vector math involved is painfully nonsensical, but it should have the same quirks as BodyGyro; increasingly severe errors towards poles due to vector math, and curved paths.
While nlerping the lookVectors of the CFrames may get you the shortest path on the sphere of possible lookVectors, it does not get you the shortest path in rotation space. The visualizations you showed aren’t keeping track of z-spin, which has to be factored into the calculations for the shortest path.
I made a demo that shows BodyGyro vs. Slerp, which gives the actual shortest path. Blue is BodyGyro, red is Slerp. The difference between the two is minimal, given appropriate damping and stiffness coefficients. It’s best viewed in Edit mode.
Edit: I’m exaggerating a bit when I say “minimal”; as the angles become larger, the difference becomes more significant. I’m not really sure why BodyGyro deviates in this way–perhaps it’s because of the singularity with 180 degree angles?
The only glitch I have seen is when you force it to rotate exactly 180 degrees away from the initial position, like @AllYourBlox mentioned. Once I saw it freeze near the starting position, although usually it does fine. For other rotations, it works perfectly.
If BodyGyro doesn’t work how you need it to, I would recommend using a Torque instance instead. There will be no black box behavior to worry about, you can just compute your desired rotation transform, convert to axis angle format, and set the applied torque directly with your own control code. This is still preferred over setting Velocity or RotVelocity directly.