This is kinda weird and honestly pretty annoying.
When I set the privacy to “No one,” everything works fine for me. But at least it should allow the friends I invite to join, right? Instead, they can’t join at all and it just shows “user has left the experience,”