Issue with Network Ownership and Collision Groups, object behaves fine in studio, flings me in game

Unanchored ladder in the same collision group as character and with network ownership set to client is flinging character at random. Works fine on studio, only happens in-game. Any idea what could be causing it, any thoughts on how to fix it? Repo place is uncopylocked if you want to try it out and check the code:
https://www.roblox.com/games/17164972665/repoplace
Screenshot of what mostly matters:

Update: A bug report has been filled for this, it was a recent physics update.

The solution that worked for me was this:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.