This is intended behaviour as setting the orientation of StarterGui only affects new players (as stated in the documentation), you should instead run this code:
When a Playerjoins the game, if they are using a mobile device, this property will determine the device’s starting orientation. - StarterGui | Roblox Creator Documentation
Changing this property will not change the ScreenOrientation for Players already in the game. To change the orientation for an existing player use their PlayerGui.ScreenOrientation property. (StarterGui | Roblox Creator Documentation)