While I fully support and appreciate improvements in the recommendation algorithm, the recent changes have been an objective step backwards.
I’m going to avoid talking about home, as that boils down to a largely subjective argument of if it’s worthwhile to promote smaller experiences with worse engagement in the long run.
To focus on one issue, genre searching, I don’t think there is any data roblox can find or provide to suggest the recent changes have been positive.
When searching a genre, e.g. Obby or Tycoon, Roblox should promote the games with the best engagement, retention or monetisation. Not CTR.
If a user is searching for an Obby, they are going to find an Obby to join, so sorting by CTR is both pointless and incentivises foul practices such as constantly changing icons to trick users into thinking a game is new. This practice has increased since the recent algorithm changes and results in poor quality engagement.
Genre searches should return to the old algorithm as not only is it fair on developers who work hard to improve their core metrics, but it also provides objectively better engagement for Roblox as a whole.
Impressions are down to almost 1/3 what they were 2 months ago.
Part of me wonders how this is affecting Roblox’s bottom line. To my knowledge, all the top experiences have taken a pretty substantial hit… despite having a very high ARPPU. As a profit driven platform, why would you try to kill your biggest & most consistent revenue sources?
My theory is Relevance is now the key metric - which explains why my games page and recommendations are suddenly flooded with brand new games with clickbait-esque icons and titles. It feels like YouTube.
I shouldn’t have to guess. This was obviously a big change. More clarification would go a long way.
I recently launched my game, Farmstead: Farmstead [BETA] - Roblox. About a week after a somewhat quiet release, Farmstead was put on the ‘up and coming’ sort. I reached 10,000 concurrent players shortly after, with fantastic revenue, engagement, and retention metrics. The only metric that was performing poorly was payer conversion. I second @OutlookG’s statement about payer conversion, and I would love to know why this is considered instead of a combination of ARPPU and APRDAU.
Just 6 weeks after hitting 10,000 concurrent players, Farmstead is now averaging about 600 concurrent players. In the last 4 weeks, I have seen my ‘Home’ impressions drop from 30,000,000 to 4,000,000, roughly halving each week.
I am not trying to blame the ‘algorithm’ for the shortcomings of my game; there are definitely areas I can improve, but seeing exposure drop so drastically in such a short period of time while having 30+ minute session times, 90+ percentile ARPPU and ARPDAU, 90+ percentile D7 retention, is incredibly disheartening.
I sincerely hope retention, engagement, and monetization (with a greater focus on ARPDAU and ARPPU rather than payer conversion) become the key signals to discovery. “Impressions to plays” as a key signal only encourages “clickable” games and does not correlate with overall user experience, platform revenue, engagement, or any other useful metric to developers or the Roblox platform as a whole.
Disappointing, predictable & generic reply which ignores the points brought up by many developers on this post.
In regards to “what happened recently” – increasing the spread of players throughout games, provided players are finding a more “suitable” experience (e.g. one they will spend more time in) is entirely reasonable. What is not entirely reasonable is making such a large change, and not making any announcement whatsoever. This impact would have been obvious to those working on this change, and it should not have gotten to the point of this post needing to be made about it. Roblox is, as usual, oblivious & ignorant to the fact that people base their livelihoods upon the earnings of these games.
In regards to “improving transparency & communication”, this is the first proper statement that I’ve seen from the S&D team, it’s already extremely lacking when it comes to transparency & communication, and makes no commitments other than “we’ll do better”. Is it a stretch to simply detail the plans there? Does it involve announcements when large changes are being made that affect X% of creators? There needs to be some proper conditions for communication, and it can’t just be “we’ll do better”.
Additionally, the “insights” product you mention has already been brought up here in this post multiple times. You’ve got folks posting here who have insights far above the benchmarks. Their games are performing significantly worse than they were before. They have insights which outperform other games, and those games have 10x their players. These insights are fundamentally not useful if they are not connected to the behaviour of the algorithm.
Lastly, I suppose it was very much to be expected that you would state to work on retention, engagement, monetisation & “relevance”. Once again, this is already known information that has been brought up in this thread – there are games here that outperform other games in all of those areas, in matching genres, that have less players than them.
I’m sorry to be so pessimistic, but this is just what I expected from Roblox when I heard a reply was coming today. Some changes that I propose that I believe would remedy some of these issues
A single, algorithm score rating on the analytics page for your game – currently, you can double your monetisation, or even session length, and see no impact in new visitors. It’s not clear how long the algorithm takes to update, or why this is, and seeing a single “combined” score to know when you’ve done something that will have a positive impact is key.
An explanation of the surge behaviour that many new games seem to have been experiencing, to describe this, many new games at launch are experiencing tens of thousands of concurrent players, and then dropping down in to the low hundreds after a month. Can this behaviour be explained? Additionally, if the concept is that the algorithm sends large numbers of players to determine metrics surrounding the game, once the algorithm has “made a decision”, is it then possible for games to actually improve their metrics & be sent larger numbers of new visitors? e.g. I have increased my game’s metrics significantly, and have pretty much only gone down.
A commitment to notifications about algorithm changes when you expect for there to be a large impact on the creator community at large – a change of this size should obviously be included
An open channel between folks on the S&D team to shed light on issues with particular games, that are doing well in terms of all of their metrics, that don’t have a clear reason to be doing poorly. There’s simply no good reason why this shouldn’t be possible. The Analytics team have done incredibly well by starting a Guilded server, and being clear & open in communication, and the S&D team should be doing the same.
It looks like the algorithm is trying to hide bigger games and promote random games with no players and with clickbait thumbnails and titles. Which looks like a pretty dumb move by Roblox (if that’s what is actually happening) because from what I know Roblox is trying to appeal to older audience and everything that I see in the discovery page is obbys and games like “Punch X simulator” which at least in my opinion won’t help Roblox.
Respectfully, have you folks read the replies above? Many of us that have been negatively impacted by these experiments are above the average for the KPIs. There are games with worse KPIs being recommended over games with better KPIs.
Honestly, I could write an entire wall of criticism here for the non-answers that we’re consistently given, but I think Brian1KB’s reply, korky5000’s reply, BuildIntoGame’s reply and OutlookG’s reply all sum up what I have to say.
We ask for genuine dialogue yet we’re still given vague & closed-off answers while being told “we promise to do better”. We’ve been told this for years with no actual meaningful change.
As I mentioned previously:
Developers need open communication about these experiments. We are not getting this. Our businesses are being negatively impacted by the arbitrary unannounced surprise changes that Roblox has been constantly applying to S&D. If we cannot reliably run a business on this platform, we will move to a competitor.
While I thank you for the response, it doesn’t address any of our issues & dodges our questions.
This is the 90-day new user acquisition for Specter, a game with > 300M visits. Around May you can see that the users from the home page (top blue bars) got a drastic cut of the overall percentage. We went from 2k-4k new users from the home page a day to less than 1000, and this drop happened in a few days. This didn’t recover back past 1000 until the 20th of June. We made no changes that would have hurt the game like this. In fact, we released an update at the beginning of May that should’ve boosted our stats.
It’s always seemed to us that Specter is designed in a way that the algorithm does not like. We aren’t a P2W simulator, and the algorithm doesn’t like that. We have a lower conversion rate than most games partially due to the genre (horror). We have a generally older audience than a lot of games.
Yet for being one of the few games on the platform that is catering to a slightly older audience, we are constantly punished. Teleport failures are almost certainly affecting our session time and retention which hurts our algorithm rankings. Almost all of our analytics show us in the top 3%-6% of “similar experiences”, yet we receive essentially no promotion from the algorithm. Luckily we are a very social game and get a lot of new users from friends joining.
Developing on this platform is such a chore, we want to make ambitious projects. We want to create games for older audiences. Yet every feature that lets us do this actively works against us based on the stats. The algorithm seems to just favor whatever gets clicks the fastest, and that’s usually neon-colored clickbait. We’ve had to adjust our icons to be brighter to match the rest of the website, even though it’s a horror game. It honestly makes no sense, but there is no genre sorting, so this is the only option if we want to get players interested.
Simply, make like 100 games a year. That’s what I do They all suck but with this new algorithm I am sustaining like 100 players on every one of them and it adds up. All these old developers are one hit wonders and now are mad that roblox doesn’t want to reccomend the same games from years ago even if they are good because people get bored not of the game but roblox as a whole if its the same games OVER and over to put it simply.
While I am grateful I even have 100 playing on my game, it is demoralizing to see games have games of my own and games of others with far worse stats being pushed by the algorithm much more. Until about just a week ago, the game did not have a single bad insight. This makes me honestly feel like the algorithm is just selecting games by complete random and throwing them out there.
If you ever wonder why spam games are a problem on Roblox, this is why, and trust me they have become very tempting to make. Spam games are awarded while other games that have had lots of hard work put in to them, with good stats are punished.
I don’t want to attack you personally, but I believe this is terrible advice to be giving people.
The terrible thing about it is that I’m pretty sure it works… Low effort games shouldn’t get rewarded over high effort games. That simply isn’t fair.
But I guess this is the current state of things, similar to the mobile gaming industry. Best we can do is hope Roblox adjusts their algorithm for the better.
Algorithm changes, exclusively, have taken my game from getting 250,000 visits a month for a few years straight to suddenly dropping to only 50,000 visits a month. Hoping for a fix, or even a reversion to these updates Roblox has been making.
Considering how many people are affected negatively by these updates I hate to say this but I am actually a big fan of the algorithm changes.
A few years ago I released a complete kit for building games themed around public transport. You could just grab it and you had a fully playable game with buses and trams.
I’m not gonna lie, a bunch of pretty well-made games emerged based on the kit. But I’ve also seen a few very low-effort, cash-grab ones too. And because of Roblox’s wacky algorithms, they gained a lot of recognition that they definitely did not deserve.
Today, when I look up the relevant keywords I struggle to even find them. And when I do find them, they seem to have only a fraction of the playercount that they used to have.
Interestingly enough, my own game about public transport seems to be entirely unaffected by these changes. And it’s back to being the top of it’s genre!
I know the situation is pretty complicated, but if I was you I would change a lot of stuff of the game, mainly because roblox is probably not going to change anything.
So I really like your game but I see one thing that the algorithm might not like: Monetization: Ok so this is quite complicated, but , one way you could make way more money could be adding lootboxes, I know no one likes them but you may not have a choice, or maybe packs of cosmetics, I would personally change the store and do the second one.
So many good posts here with great data. Somehow I missed this topic, but to summarize; everyone seeing a drop is more than likely the victim of the system spamming of bots to game the system. Right now it is too easy bot visits, likes, dislikes, etc. All this is probably the root if the issues presented here. The search algorithm is being crushed by the spammers.
They all suck but with this new algorithm I am sustaining like 100 players on every one of them and it adds up. All these old developers are one hit wonders and now are mad that roblox doesn’t want to reccomend the same games from years ago even if they are good because people get bored not of the game but roblox as a whole if its the same games OVER and over to put it simply.
