As a Roblox developer, it is currently extremely difficult to run custom A/B tests and view analytics about how these tests affect our experiences. Right now, we have to rely on third party analytics software in order to create segmented groups to experiment with changes and compare metrics between these groups such as engagement and retention.
Many third party analytics providers offer features like custom user segments (e.g. a segment for an experiment targeting new players, returning players, etc.), remote configs, and analytic dashboards with breakdowns for different test groups. Roblox already provides developers with first party analytics dashboards, but little options for improving our analytics. There’s no way to view custom breakdowns, so it’s not possible to create an A/B test and compare metrics for control groups and test groups without moving your analytics to a third party provider that offers these segment breakdowns.
It would be extremely beneficial for developers to have a built in feature for running A/B experiments that target improvements in our metrics, allowing us to segment players and compare these segments in Roblox’s analytics dashboards.