Analytics: Recommendations Qualified Play Through Rate and Similar Experiences Benchmarks

The labels are:

  1. Similar experiences
  2. Genre (if we don’t have 50+ similar experiences for your game)
  3. All experiences (if your game’s genre hasn’t been classified yet internally)
8 Likes

The benchmark range visuals on the charts don’t negatively impact recommendations. I’m curious why you think the range is unreachable - feel free to DM me a link to your game and I’ll take a look.

8 Likes

Not even current users?? So small games are not even able to see something like that?? We were able to see it before, why are we not able to anymore???

5 Likes

Very helpful - thank you. Do you know what it’d take for a game to get classified by genre internally? Mine has quite a few CCU and has had them for weeks now. Could I DM you a place ID?

6 Likes

Correction, you should see the table of current players.

That you don’t see it is a bug and we will reply back when it is fixed.

8 Likes

Super awesome to get analytics data to get a better insight into how the algorithm is viewing my games. Keep up the great work :slight_smile:

9 Likes

The issue has been fixed! You should be able to see your concurrent users again. Thanks for reporting :saluting_face:

7 Likes

Yes, exactly that.

The funny is that the experience that i have the new overview dashboard available is not my post popular experience right now.

But thanks for the reply

7 Likes

I am not interested in benchmarking “similar experiences”, especially when the definition of “similar” or “genre” is opaque and changing. Other factors may make a big difference - for example, why should I care about comparison with experiences that are primarily PC-focused, while my game targets mobiles? The “All experiences” benchmark is good enough for me to see a trend or progress, can I just switch to it?
Also, I think the only useful stats are the total playtime and the number of concurrent players. DAU, MAU, Retention include hundreds of alts, random clicking players who did not even wait for the game to load, or their device crashed etc. The Number of sessions is useless, if players’ internet goes down, or they rejoin for some reason, or we kick afk players with an autoclicker.

6 Likes

Did you even read the post? It sounds like they have the solution to your problem.

5 Likes

I have a few questions about experience discovery. How does the “category” your game is put in in relation to being similar to other experiences get chosen? Does a Roblox mod come and determine what category you are in? Or is it somehow dynamically chosen? Does that category have a big impact on suggestions? I think it would be good for us to be able to see what “category” our game has been placed into would be helpful.

I’m asking this because I have two games where, despite having very similar stats, one gets tens of thousands of home recommendations per day and the other gets a few hundred. I don’t really understand what could cause such a massive difference, except maybe the two games were put into two different “recommendation categories”. I have no idea, really. Thanks

5 Likes

Don’t want to create a separate thread about this yet, but I wonder if anyone else had a significant change in their engagement stats after being “recategorized”. To me it seems like whatever category was chosen for my game was a missmatch, since the game has been getting much more uninterested players since benchmark ranges changed.

image

There’s no annotation for the shift, but those are buggy. Audience analytics are hard to read and split, but “Most recent” date (whatever that means) show’s 31.6% of the audience being <9, when it was 27% “30 days ago”. My game is not very fitting for <9 and it is too late to use Content Rating to cut children off, because it would be unfair to those who spent a lot of time in the Experience.

I don’t like using dramatic phrases, but this is kind of killing the game :melting_face:

Then again, this could be a coincidence… and I should’ve just made a simulator instead :smiley:

2 Likes

I’ve been wondering the exact same thing in my above post. I think it would be great if Roblox showed us these “categories” that are assigned to help improve suggestions. The original post here talks about how you need to be marked as similar with 50 experiences to “qualify for certain benchmarks” and that if you don’t, their “internal genre data” is used. I have to wonder if any of this affects discovery, how this “internal genre” is decided, etc. All of this would be useful information from Roblox.

1 Like

Does this in any way address the bug where if you update your description it can be dropped from search results/recommendations completely? I posted about this bug affecting discovery back in June, my experience dropped from ALL search results for it’s keywords where it had been steadily growing in rank from sponsored ads.

MY GAME HAS BEEN UNSEARCHABLE FOR OVER 2 MONTHS, this has basically killed all motivation to continue to work on my game, or run ads, because users will never find the game thereafter. There’s no apparent fix because the search team apparently doesn’t understand what hidden blacklists they may have in their system. I even managed to get a Roblox staff member to play my game (they liked it :)), but no solution, just a 'wait till some future patch, maybe".

How are people expected to ‘grow’ and get recommendations if their experience literally can’t be searched?

2 Likes

This, please. My game used to be the 8th result for “army” and then suddenly disappeared from that search term entirely and the amount of new players coming to my game dropped massively, divided by 5 or 6. I actually emailed Roblox asking about the “blacklists” you mention and they claimed it didn’t exist. I am not so sure I believe it.

Hello @signal_zzz ! Nice to meet you

I have a question not sure if you could help me out, but I am struggling to get my games to even be recommended in the “Home-Recommended” category. Check this:

image

Despite the game having good metrics, It has N/A Home recommendations, and for such reason 0% in Qualified Play TR. It has only been up for a few days, wondering if this is a process that takes longer? I am at 3000 daily active users.

I made anothe game 1 month ago which I have been sponsoring since then, not with the same metrics but with a decent ad spent and it got 0 home recommendations and N/A too

Let me know if you have any insights, I am a bit worried it will not get to home recommendations for some reason. Here is the game if you want to check it out: Mining RNG 🎲 [LAUNCH] - Roblox

Best,
Mutrix

3 Likes

I’ve tested about with this recently because I lost my good thumbnail that was bringing in 30k dau was doing well until I clicked delete by accident and noticed my qualified play through rate for recommendations go down to around 50th, looked at the thumbnail on the chart that showed me where the line was starting to go back up again and changed it to that one now my play through rate for recommendations has gone back up to 90th again concurrent is going back slowly. This is how it works for me test about with the thumbnails and a good icon leave it a few days and see what the analytic is for qualified play through rate recommendations. The thumbnails or icons I like the best don’t actually do that well sometimes the one I’m using now I’m not too keen on but it brings in more players so testing this out currently. This might sound silly but what I’m confused about is what exactly does 50th and 90th mean nothing I type in on google/devforum seems to bring any info to what 50th and 90th actually means is it the 50th or 90th place on the home page or something else ?