Last night I stayed up until 2am to grind out a final overhaul of a system in World // Zero. I can’t remember if I submitted the draft or not, but the code was complete and I went to bed.
Place Link: https://www.roblox.com/games/2727072708/Dev-Place
Thismorning I woke up and was working on my other computer, doing some GUI changes. I adjusted the size of some guis, and uploaded new icons to replace old ones. We had a power outage 30 minutes later.
I booted my PC back up and noticed none of my gui changes had saved. This is odd because team create is enabled, and these are not script changes. I then noticed my scripts from last night weren’t changed, so I figured I forgot to submit the draft on my laptop. I went to my laptop and the drafts are empty. So this work has been erased.
I have had 2 of our builders tell me they’ve been experience build progress getting reverted in the past week, and have turned off team create to avoid the issue.
tl;dr Team Create / drafts are currently not in a reliable state and should be disabled until Roblox can verify developers time is not being wiped.
We are going to try doing manual merges of our scripts with offline-places for the time being. This has a major impact on our ability to operate & scale a game studio. Roblox needs to have working collaboration tools, or switch to supporting proven tools such as GitHub out of the box. Team create has been having issues deleting people’s work for years and I thought it was finally patched.
12 Likes
Do you have rough timestamps for when each of these happened, past the 2 AM work?
Also, how much building progress was lost? < 5 minutes? 30 minutes? An hour? And how was the loss discovered? Successfully closing Studio & re-opening to find the work missing? After reopening from a crash?
~10:05am PST was the power outage that wiped 30 minutes of GUI work.
The 2 builders have been losing progress over the past week. Sunday night / Monday morning was a time it happened.
The loss was discovered because it was as if I had never made any changes at all. I might as well have slept from 10pm - 10am and the place would’ve looked identical
There shouldn’t have been any service outages that affected the TC session at 10 AM today. Can you provide a log file from that session? We would need to correlate it with our Team Create server logs & see what went wrong.
For the Sunday / Monday loss, that is somewhat expected in the sense that we had a large service outage during that time, but this should have triggered a disconnect in Team Create that prompted the builders to save their work. It would also be helpful if we could get log files from those sessions to check the Team Create server logs there as well.
As for the Drafts issue, drafts are entirely local and should not be affected by any service problems. They are backed up to the local disk every time you make a change, so it is interesting that even still hours of work were lost. Can you provide the log file from that session as well so we can check what went wrong on the TC server & your Studio client?
Unsure if this is related to this post but yesterday (approx. 5PM - 9PM EST) my team create was also failing to save or publish any changes. I could save/publish my packages but draft changes were also lost as well upon restarting studio. If it weren’t for packages, I would have lost hours of work if I wasn’t paying attention. Games that weren’t running TC had save working. I had to disable and re-enable my TC for it to finally work again which was strange. There was no changes to my work environment.
log_1C56D_1.txt (16.1 KB)
Thanks! To clarify, were there any notifications in Studio that something went wrong? E.g. the connection indicator in the top-right turned red, a warning popup, etc.
Top-right was green the entire session (I was constantly checking if it was my own internet). I believe when I was trying to restart studio, an error window popped up saying something along the lines of (im paraphrasing since I don’t remember the exact lines) “This place was unable to save. You would lose any changes if you exit now” with the option to press yes no and cancel.
1 Like
Here’s a log from the 10:05 crash (timestamp 10:04am)
log_6316F_1.txt (10.3 KB)
It looks like there was a service outage there too prior to the power outage, as there are multiple save failures at 5 minutes apart each:
1584118125.52905,24cc,6 Error: Server Save Failure: HTTP (500): Internal Server Error
1584118438.18530,16b0,6 Error: Server Save Failure: HTTP (500): Internal Server Error
1584118750.10315,2128,6 Error: Server Save Failure: HTTP (500): Internal Server Error
1584119067.59521,24cc,6 Error: Server Save Failure: HTTP (500): Internal Server Error
Clearly we don’t do enough to communicate/rectify that the server cannot save during a service outage, and have immediately begun work to address this. Until we are able to ship improvements, you can keep an eye on the output to see if the server is failing to save (in which case, save a local backup of the place and start working on that instead).
If we could get logs for your session where you developed scripts until 2AM, it would help us determine whether the issue was that they were committed & experienced the same problem as above, or were not committed & may be the same issue as this bug report.
5 Likes
Note that in my bug report I could edit other scripts at the same time without issue which appears to be slightly different from what is happening here. My issue also began some time before these other, more serious issues started happening as well (AFAIK from when Collaborative Editing was first enabled). Glad to hear this is being worked on
1 Like
Here are the logs from the computer that had the draft, taken at 1:35am:
log_77713_1.txt (24.6 KB)
log_3D766_1.txt (13.0 KB)
1 Like
I know this might not be the right place to reply, sorry about that.
But I wanted to tell you that I too have been having this problem since yesterday, since the servers were having problems.
I get the same error message after trying to publish, now I can’t turn Team Create back on after turning it off (without saving progress) as it keeps saying Save Failed. Yes I am still getting the error even after several hours later.
1 Like
It looks like the drafts were never committed, so they should not have been affected by the Team Create server failing to save.
If it’s similar to the issue EmeraldSlash reported, then it could be they were never checked out to Drafts at all, but to double check, could you PM me the contents of:
%localappdata%\Roblox\Sandbox\2727072708\4402987
?
1 Like
The folder is empty, I couldn’t go that far
1 Like
This is probably unrelated, but publishing to a team create from a local file doesn’t save to the team create either. Instead, it closes the local file, opens up team create and doesn’t actually publish.
3 Likes
I think this is related. I was trying to publish the place and turn off TC so I could edit it on another PC, but I’m unable to make it publish. I have to use a USB stick to transfer the place file between computers right now.
After the first publish fails, I can’t try again because the publish dialog has a bug.
1 Like
We had to disable tram create again as it is disconnecting us every 10 minutes or so.
3 Likes
Sorry to hear that, this has happened to me once before, but I still don’t know the cause of the glitch.