(Other signals are within benchmark range)
Check out my post above, I have the same dire problem. This is an issue arising from complications with reserved servers, which you seem to use on Wizard Wars. It’s only counting things when you are in that highly temporary lobby public server. Roblox seems to be willing to fix this critical problem with games like ours, it’s very important for us!
I KNEW IT, did you know referredbyplayerId is also 0 when trying to join a friend in a reserved server, can’t do reward logic for invites because of that
Does that mean that our games were pushed under because of that? I’m gonna be so mad if that’s the case
Yes I have noticed that. Invites seem to act like standard joins, although that’s better than the more common and worse case where they try to directly join a friend and are told they “aren’t allowed to join the experience”, Error Code 524. It’s been quite brutal.
I explicitly asked twice in my replies if there was a similar social co-play metric in the algorithm before the update. But, I think they aren’t permitted to say. Which is understandable. My question was “acknowledged”, but not answered yet. I personally hope so, because then that means it can only get better for us! Best of luck to us.
Bro if this damaged our discoverability we should get compensated by Roblox, this is wild
My game had reserved servers for about 2 years now I’m pretty sure and I assume that this signal is relied on not just slightly, so this is really frustrating
I mean, they lost out on money too if so. It’s a shared profit scheme. It’s in everyone’s interest to see this corrected and for discovery to work as well as it can. But I’m not one to be incensed myself, although yes it could’ve been hurting us for a long time. I don’t want to spam up the thread anymore, so send me a PM if you had anything else you want me to answer.
Idk if this is right by the way, my game has been dropping sharply from recommendations the past months and the signals data is pretty consistent even improving over time!
- The benchmark
Is something wrong here?
@skpm_plz Is this expected to happen for whatever reason or is it due to something else out of my control?
Hi. This change in algorithm severely dropped my game’s recommendations over the past 20 days, to the point where we came down to 500 CCU on the peak weekend time versus being 2,000 CCU previously. We were actively pushing updates during this period. Also the game was grouped with much bigger titles such as Dead Rails, Rivals, etc in similar experience benchmarks. I noticed that just today our grouping was changed back to shooter genre as it was before this update. So I just want to understand did the change happen because our player numbers dropped so much or because the Roblox staff changed something in the algorithm again these recent days?
@skpm_plz you seem like the person to ask this, thank you for your time
Hello, this is exactly the same situation that is going on with us. We have some insanely variable benchmarks, and our home recommendations have dropped to the point that we went from 1k ccu to 100 in just about two weeks. All of this is in spite of adding updates that have improved virtually every benchmark, even the ones that are supposedly now used by the algorithm.
We were also compared to games like Dead Rails even though our game is a 2 Player Obby (which could make sense under some criteria, i don’t know if its accurate or not).
In some benchmarks, the comparation data seems to have dropped to the before levels, when the algorithm was actually giving us more impressions. However, i believe its going to go back up to the disproportionately high benchmark again, since in the Average Playtime graph one can see that it has gone back up after dropping.
The change from Similarity to Genre Based Benchmarks and then back to Similarity is due to some data issues at the platform level for that date.
We fell back to Genre because Similarity was not available
Sorry for the trouble
Hey @royee354 sorry for the late reply, I have a lot of threads going on and missed this one.
Could you link this experience? It’s difficult to tell if the beginning (Jan) was coming off an algorithm exploration phase and what affected it. However, based on these screenshots it looks fairly stable since the new algorithm was released (end March).
In general, impressions can fluctuate based on multiple factors, including your own metrics, platform metrics (i.e. more users on weekends, fewer on weekdays), and other experiences’ metrics.
Experiences are placed on Home next to others. If others have improved metrics / more experiences have wide appeal they might now rank higher than yours. If a user decides to play one of those, they won’t scroll to see yours (even if we have recommended it to them). So it’s not strictly that your metrics improving guarantees more impressions (though it usually occurs)
Sorry for the late reply @4_ykk, I’m seeing that this experience got a spike of impressions on 4/15 (I think I’m seeing an improvement in your monetization right before). I’m also seeing another larger spike in the past couple hours on your thumbnail personalization charts.
At this point I’d recommend just monitoring your metrics / working on those that are under benchmark. In particular it seems like there’s a step in your onboarding funnel that has a big dropoff but you probably already know that - this funnel seems pretty well set up btw
“No benchmark experiences” - I think it doesn’t always show up until there’s enough data, but maybe @imagineerrx can help explain it more.
Does this mean that the similarity benchmark will be the one kept?
My benchmarks were also rolled back from “Benchmark for experiences with similar players” to “RPG genre benchmark” Which appears to have severely affected the performance of my active Sponsored Ad(s) and potentially impacted my Home Recommendation impressions negatively.
Though I haven’t managed to resurrect the game just yet – this comes at a time when I’m preparing to release a large update.
Thank you for replying! I hope the algorithm didn’t mess me up, it feels like it did and maybe partly because of the fact I’m using reserved servers, maybe it messes up some of the signals. Some features I found already break due to the useage of reserves serves such as ReferredByPlayerId being 0 even if you were directed to an experience by your friend that’s in a reserved server. So I wouldn’t be surprised if more things are affected by it.
Hi @LootRPG , benchmarks are purely for informational purposes and do not affect the algorithm / impressions. They’re just there to show you what people who are playing your game are also playing.
Ths issue we’re describing is that this information was missing for some period of time (only for analytics reporting purposes), so during that time, we weren’t able to show the similar experiences data.
Thanks for explaining - would you be able to file dev forum bugs for these issues you’ve found?
That will help us to ensure that the correct teams are tracking them.
