Analytics Benchmarks Are The Same Across Different Genres Of Games

Bug Reports Website Bugs
#1

I have talked to and seen the stats of numerous developers who have games of varying genres, I believe that analytics is displaying the same “similar experiences” benchmarks across games that are not of the same genre

Reproduction Steps:
View the analytic benchmarks for the games included in the DM to bug-files
Expected Behaviour:
“Similar Experiences” benchmarks should at least somewhat reflect the genre the game is in, which the following games’ benchmarks do not

This has a significant impact on developers because as far as I am aware benchmarks across different KPIs all observe the same “benchmark category” (for the lack of a better term). Developers rely on benchmarks to visualize how their games are performing compared to their peers and it is vital that we get accurate data.

I can appreciate that genre is not the only factor that Similar Experience benchmarks are computed with, but to have so many different types of games share the same benchmarks renders the entire thing useless as a reference point to see how your experience is comparing against your peers.

A private message is associated with this bug report

#2

This is just an acknowledgment announcement!

We’ve filed a ticket into our internal database for this issue, and we will update you when we have further information!

Thanks for the report!

#3

Analytics uses the genre that has been assigned to the Experience and if none is assigned then fallbacks to a default.

Not all Experiences have been assigned a genre, but I’m not sure what is the criteria for being assigned a genre, but generally larger Experiences would have an assigned genre.

So i think this is twofold:

  1. Have the Experiences that you viewed actually been assigned a genre? or is it an incorrect genre?

I suspect this is the actual issue but would need a list of Experience IDs to confirm

  1. Is the genre based benchmarks broken?

I just spot checked on some larger Experiences and they have different looking benchmarks.

2 Likes
#4

Thanks for the reply! I’ve attached a list of game IDs in the private message I sent to bug-reports.

Here is the link again: https://devforum.roblox.com/t/analytics-benchmarks-are-the-same-across-different-genres-of-games/2889165

#5

Experience genres are labeled by human evaluators that is what powers the Similar Experiences list. This is different than the genre that you select in Studio.

We checked each Experience and they do not have a human evaluated genre and hence are being placed in the default genre which is being used for benchmarks.

#6

How does one get their experience evaluated for a genre? Is there a way to submit it for review, or is it based on how large the experience is/how many visits it has? It would be interesting to be able to view that statistic on your experience configurations, too. I have two experiences that are the exact same but with different maps, and yet upon looking, they each have different “Similar Experience” benchmarks in their analytics, which makes me think either one of them was incorrectly assigned a genre, or one just doesn’t have one, yet. I suppose the reason why I’m interested in all this is that I am wondering if this affects experience discovery, since my experience has been having intense algorithm issues lately. Thanks

3 Likes