What response data does RenderingTest want?

The RenderingTest object can be parented to TestService and will send data to localhost:8001 upon a TestService:Run() call. It sends a POST request to /api/v1/submit_test containing lots of rendering information, including a screenshot, CPU/GPU info, performance info, tickets, etc.

What response data is necessary to “let it know” that this test failed/succeeded? The Developer Hub article states that you should pass the strings "passed", "failed", or "golden":

However this is no longer accurate and passing these strings now causes the following to happen:

What response data should I return to tell Studio if the test passed or failed?

3 Likes

RenderingTests are not a publicly supported feature. We started building that system in 2018 to detect rendering bugs, and have been gradually improving the system and using it for more things internally. Some day, it may be ready for public use.

Unfortunately, there was internal miscommunication about this feature that resulted in usage instructions being posted on the dev hub. I’m sorry if you’ve built technology on this, but we don’t have the bandwidth for proper support at the moment. The protocol may continue to work in its current state for the near future, but as we continue to iterate on our internal system it may break for you.

We’ll update the dev hub with a note warning that this API should not be relied upon. If you have uses for this technology, please let us know what they are in a feature request so we can understand the demand and work on proper public support sooner.

6 Likes

It wasn’t so much a miscommunication as it was me just personally documenting it without oversight because I thought it was interesting. I still had a wiki volunteer mentality where I’d just document niche stuff.

The DevHub’s documentation has always been sort of a wild west, a mishmash of community contributed documentation and official documentation. It seems work is finally being done on this front though.

7 Likes