Analytics [Similar Experience Benchmarks & Broader Access]

Updates Announcements
#1

Hi Creators,

Our vision for analytics is to empower every creator to take action to grow their experience. Today, we’re launching two new updates that push us towards that goal:

  1. Similar experience benchmarks
  2. Analytics access for all 10+ daily active user experiences

Your input is a crucial part of achieving that goal. If you’re interested in giving feedback on our upcoming analytics products, please fill out this quick form and we’ll get back to you.

Let’s dive into this update.

Similar experience benchmarks

In October, we launched the ability to benchmark your metrics against experiences overall on Roblox. Today, we’re updating these benchmarks to show how your metrics compare with similar experiences (e.g., genre).

How do similar experience benchmarks work?

When you visit an analytics page, you’ll now see similar experience benchmarks in the 50th - 90th percentile range for per user metrics.

For example, for day 1 retention, the 50th to 90th percentile range for similar experiences might be 12.4% - 24.1%. This means that:

  • 50% of the similar experiences have a day 1 retention of 12.4% or lower.
  • 10% of similar experiences have a day 1 retention of 24.1% or higher.

Similar experience benchmarks are based on our internal genre data (used in our home and discovery pages) and other metrics. These genres include Obby/Platform, RPG, Fighting, Survival/Horror, and more.

1
11238×620 55.9 KB

Which metrics have similar experience benchmarks?

We show similar experience benchmarks for per user metrics including:

  • Retention: D1, D7, and D30 retention
  • Engagement: Average session time
  • Monetization: Payer conversion rate, ARPPU, ARPDAU

For other metrics (e.g., daily active users) we still show the closest top experience benchmark:

2
21238×622 16.1 KB

Analytics access for 10+ daily active user experiences

All experiences with 10+ daily active users over the past 7 days now have analytics access (we don’t remove access once granted). We hope to expand access to all experiences by the end of this month.

  1. Visit your creator dashboard.
  2. Select one of your experiences.
  3. In the left nav, select an analytics page (e.g., Overview, Retention, Monetization).

Please let us know if you have any questions or concerns regarding this update.
Thank you.

107 Likes
Analytics: View Personalized Insights for Your Experience
Analytics: Recommendations Qualified Play Through Rate and Similar Experiences Benchmarks
#2

This topic was automatically opened after 10 minutes.

#3

Creating broader references and defining those references by relative percentile empowers the developer to be more insightful of competing games, or for their own. I really like this change, since it allows developers such as myself to know if the game is steady, volatile, or growing.

In other words, the broad ranges really help provide a much more informative baseline that helps the developer create quarterly key performance indicators for updates and releases.

I do wish some other stats like DAU also had broad benchmarks so that the developer can create assumptions about how well they are acquiring new players.

12 Likes
#4

Adding additional lines for “similar experiences” is fine, but removing the existing benchmark line for all experiences feels like a major regression. It is a useful line, and removing data in favour of “other” data is unnnecessary – more metrics please, not trading one for another!

I think that the route to go here is to allow us to show the old benchmark line in addition, to compare against everything, as it’s a great way of knowing “where” your game is (e.g. 500th-1000th), rather than an arbitrary percentage range.

Lastly, in regards to the line itself, tell us what genre Roblox actually believes our game to be! This comparison doesn’t mean much of anything if I don’t know what I’m being compared against.

Great, no-brainer change in regards to the 10+ DAU access.

17 Likes
#5

I disagree. The existing benchmark line is mostly misleading and/or uninformative to the developer. The previous benchmark line only told the developers about how their game health compares exclusively to top experiences.

By providing a range, the developer now has access to both that information, as well as information regarding less-performant stats of other games. Knowing where you lie between (or outside) the two can create strong assumptions, whereas before the developer could only make moderately accurate guesses.

9 Likes
#6

I think this will be very useful to come!

Will there be any more ways to add more ways to compare? For example, maybe more genre tags? I have always felt they were too limited.

2 Likes
#7

This is awesome! What’s the chance we can get 6 month and 12 month retention rates? Our studio wants to use these as KPIs but they’re not tracked right now.

This sort of information will help us build long multi-year games that last for a decade, not just a month.

Anyway, great update! Analytics are so important!

2 Likes
#8

Id love to see analytics, atleast some, for each place of a universe… in order to properly assess how one place is doing compared to another. As of now, we have not one single metric available on the place level, only the experience level.

2 Likes
#9

This is absolutely game changing. Excited to use this!!

4 Likes
#10

This is exciting news for Roblox developers! The addition of similar experience benchmarks will provide valuable insights into how their metrics stack up against other experiences in the same genre. It’s great to see Roblox continuing to empower creators with analytics tools to help them grow their experiences. And with analytics access now available for all experiences with 10+ daily active users, even more developers can benefit from these insights. Looking forward to seeing how this update helps drive the success of Roblox experiences!

image
image680×510 126 KB

5 Likes
#11

Yep, you explained why we made this change well :slight_smile:

5 Likes
#12

Thank you! We haven’t thought about 6-12 month retention rates. We have been discussing showing retention by weekly cohorts. If you’d like to preview our upcoming releases and give us feedback, please fill out this quick form.

6 Likes
#13

Thanks! We do plan to add a place dropdown so you can see analytics for specific places in the coming months.

4 Likes
#14

Thanks for your praise - will definitely motivate our team! :rocket::rocket::rocket:

3 Likes
#15

How does Roblox knows the genre a game is? For example, I don’t see Survival as a genre to choose from when setting a genre for the game:

image

Is it based on the description or name? I don’t understand.

Currently my game is set to Adventure, but it is a survival game. (I imagine is considered to be survival by Roblox too since the website/app has recommended the game on the Survival sort last year)

1 Like
#16

it is based on our internal genre data that you see used on our home and discovery pages.

1 Like
#17

Is there a way to know which genre your experience is being compared with? I’m just assuming that the genre is correct, but it would be nice to know for sure.

1 Like
#18

Frustrating response, perhaps I wasn’t clear.

For my game, for day 1, it exceeds both the 50th percentile, and the 90th percentile of more than half of the graphs. Great – I’m doing well in my genre, but I now can no longer compare how my game is doing in regards to ALL genres on the platform for those metrics.

I do not have an issue with the addition of new metrics, but it was not a sensible decision to remove the old benchmark line; particularly in the case of games that do not have strongly performing games in their genres. Again, if you’re going to show a line correlating with “similar” genres, tell me what genre I’m in!

2 Likes
#19

A little off topic but if genres are being tracked like this internally, could some sort of genre sort return to the Explore page? Being able to see games of a genre that aren’t necessarily part of the featured sort like how we used to with the manually selected genres was a great feature for finding newer titles that is sorely missed.

#20

Doesn’t help until we can have genre sorting back. Why are genres internal? What even qualifies an experience under a certain “internal genre”?

Every time I see an update talking about employing genres, it feels like a smack in the face. Genres were axed in 2017, I made a feature request the very same day that has garnered exceptional support as well as other feature requests of the same nature or in a different take (e.g. experience tags), we were promised them back at RDC, and still nothing.

It’s not a bad update but this one thing irks me to no end, and I’m entirely dissatisfied that we continue to have no replacement nearly 6 years later that would empower us not only for analytics but also for discovery. Developer-driven sorts - NOT staff-curated sorts - would go hand-in-hand with the ability to compare to similar experiences while actively knowing that the data is at least considerably accurate. It feels like staff are moving towards locking in discovery that benefits only the top elite developers by having systems based around these internal sorts and it worries me to no end that staff are suddenly not going to want to change these systems to support better genres.

Why can we not first fix this cripplingly broken discovery first? Why is it so hard to give us genre sorts back or give us an adequate replacement? Enough of this staff-curated discovery, it’s so harmful towards developers who want to break into certain niches without being overshadowed by others who simply have larger capital or standing on a higher irrelevant sort. A good number of players won’t even scroll all the way down to some obscure sorts. Please empower our ability to cater better. The analytics are worthless to me without knowing that I’m reaching the audiences I want to as best as possible without the input of staff who may misinterpret my audience.

20 Likes