Completely forgot about this, but after looking into this more I can confirm that the folder does not save when restarting studio, publishing, or saving to roblox. The only circumstance I can get it to stay is when saving in a local file and opening that local file. I believe this post describes my issue quite well.
https://devforum.roblox.com/t/folder-inside-game-gets-deleted-when-restarting-studio/2208906