Network ownership being lost randomly, anyway to fix?

Hey, so I have this capsule object that I set it’s network owner when the player is added, because I have CharacterAutoLoads off for what I’m trying to achieve and for some reason it seems at random times the network ownership is being lost, I was having this issue before but I just found out that it was network ownership