Testing Improvements to the “Recommended For You” Algorithms

[starrrydays]

Hi Creators,

As shared at RDC, we are committed to being transparent about meaningful updates that could affect the discovery of your experiences. Today, we’d like to share an update on our recent efforts to improve the algorithms behind the “Recommended For You” sort on Home.

What are we testing

We’ve previously shared how user behaviors impact which experiences are shown in the “Recommended For You” sort on Home. For instance, we rank experiences based on engagement (e.g. qualified play through rate, D1/D7 retention and play time) and monetization (e.g. payer conversion and robux spent), as these are proven indicators of user value and long-term retention.

We’re excited to share our progress in identifying additional user behaviors that indicate long-term retention and help more creations get the opportunity to succeed on Roblox.

We’ve found that the frequency with which users return to play, spend, and interact with friends in your experiences are strong indicators, so we’re now testing these in our recommendations.

This will allow us to rank experiences and updates with a broader definition of success.

What this means for you

Keep building and updating experiences that users want to come back to. Our focus on long-term user retention hasn’t changed. You can also continue to track how your experience is doing in Creator Analytics.

As we test, we want to give you a heads up that some experiences may see fluctuations in their “Recommended For You” impressions. You can continue to monitor impressions from various sources including Home Recommendations in your analytics dashboards.

In early 2025, upon finalizing the results of these tests, we will incorporate these additional user behaviors into our overall recommendation algorithm and add them to your Creator Analytics dashboards so you can clearly see how your experience is doing and where it can be improved.

Please stay tuned for additional announcements in the new year and let us know if you have any questions.

Thank you,
Roblox Discovery Team

---

FAQs

Thank you for your questions. Below we’ve answered some of the most common ones.

What happens to good experiences that are meant to be deeply engaged vs. frequently in bursts?

• We’re testing additional user behaviors that are strong indicators of long-term retention. We are not replacing or overriding the user behaviors that are already proven indicators of long-term retention.

  As a recap, we rank experiences based on engagement (e.g. qualified play through rate, D1/D7 retention and play time) and monetization (e.g. payer conversion and robux spent), as these are proven indicators of user value and long-term retention. In addition we’re now testing the addition of the frequency with which users return to play, spend, and interact with friends.

  One user behavior alone is unlikely to disproportionately impact an experience’s discovery. As an example, while we value robux spent, completely free experiences that engage users in other ways can still rank well. Our intent is to rank experiences and updates with a broader definition of success.

How does Roblox account for frequent crash rates or other low-quality issues that may superficially increase the frequency of use?

• Frequent crashes mean a poor user experience, which doesn’t help the discovery of an experience; it hurts it. Our systems already account for this and will continue to do so.

Do the recommendation algorithms show monetizing experiences to paying users only?

• We do not funnel paying users to monetizing experiences only or non-paying users to non-monetizing experiences. We aim to match experiences that users enjoy playing and keep returning to.

What does “interacting with friends” mean?

• Co-playing, such as users joining an experience because their friend is playing or chatting with their friends within the experience are two examples of user behaviors in this category.

[Capoyoace (beebs)]

Great to hear I’m going to be seeing some experiences I want to play.
These epic updates are all dropping around the time when my experience releases, which is great

[Noble_Draconian]

@starrrydays

e.g. payer conversion and robux spent)

Are there any short-term plans to improve how the algorithm reacts to improvements in user spending in a given game? Currently, if an experience historically has had bad monetization but then receives an uptick in spending due to updates being released that address that problem, the algorithm continues to funnel non-paying users into it despite the changes / improvements made to monetization. This makes it extremely difficult to convert a non-monetized experience into a monetized one. You’re quite literally better off publishing the same game to an entirely new universe to “reset” the algorithm’s view on it. This is horrible for business and cannot be the intended dev experience.

@starrrydays

We’ve found that the frequency with which users return to play, spend, and interact with friends in your experiences are strong indicators

Isn’t this going to be a net-negative for content-heavy experiences? If the algorithm is punishing games for not pumping out updates every few days, that’s going to incentivize low quality / slop releases, further exacerbating the challenge of competing with idle / clickbait games.

[Noble_Draconian]

@Noble_Draconian

Are there any short-term plans to improve how the algorithm reacts to improvements in user spending in a given game? Currently, if an experience historically has had bad monetization but then receives an uptick in spending due to updates being released that address that problem, the algorithm continues to funnel non-paying users into it despite the changes / improvements made to monetization.

cc @TheAmazeman, IIRC you’ve spoken about this in the past.

[jackjenningsdev]

My game struggles with a low D7 retention since our gameplay is complicated and pretty niche, but those users that are retained tend to play for hours every day.

While we’re working to improve onboarding for all users, would this change benefit my game’s discovery by considering the high engagement of those users that are retained?

[TeamDreams123 (sillygoober)]

Glad to see roblox committing to what they promised us. Although I don’t think this is going to change much as the algorithms are just about what you play…

Then again, I see this kind of stuff and really like this update:

For your information, I don’t play games like these.

[BladianMC]

Could you elaborate what is meant by interacting with friends?

Is it time spend playing together, is it joining, like what specifically is it?

[jackjenningsdev]

@Noble_Draconian

Isn’t this going to be a net-negative for content-heavy experiences? If the algorithm is punishing games for not pumping out updates every few days, that’s going to incentivize low quality

As I read it, this should be a positive change.

Consider an extreme example:

Game A has lots of content, like a campaign or story mode. A player joins, engages with a story, and comes back every day for a week to keep progressing in the story.

Game B has very little content, but releases a minor update every week. A player joins, plays two days in a row, but quickly runs out of stuff to do; they only come back again a week later, when a minor update is released.

Under the current system, the algorithm considers these both equally engaging, even though common sense tells us Game A is better. The new system should recognize that the frequency of visits to Game A means it’s a better game.

@starrrydays Correct me if anything I’ve said is wrong, but as I’m reading this thread, above is how the new algorithm should work?

One hope I have, especially considering that the new algorithm will consider frequency of spending, is that this doesn’t penalize games that don’t implement developer products. My game Oath of Office relies on a few high-value gamepasses for monetization, not repeated-purchase developer products. This strategy is objectively working very well - we have one of the highest ARPDAUs on the platform - but players will typically buy the gamepasses they want in a single purchase, so while our payer conversion rate and revenue are quite high, the actual frequency of transactions is low.

Avoiding constant microtransactions while maintaining a high revenue is objectively a good thing - for the business and the player - so I hope we aren’t penalized for doing this in an algorithm that considers frequency of purchases.

[Noble_Draconian]

@jackjenningsdev

The new system should recognize that the frequency of visits to Game A means it’s a better game.

In the longer term, the algorithm would punish game A because it has a longer release cycle (e.g. 3 weeks instead of 1 week). Barring replayability, players would frequent game B more as it introduces new releases more often. The algorithm would see this, and boost game B.

[caio_balassoni (coai)]

To be honest, having the algorithm prioritize experiences that do frequent updates is really damaging to games that have a lot of content to explore(specifically narrative-heavy ones such as Alone), but are only supposed to be fully played through once or twice therefore not receiving many updates, if any at all. Isn’t this going to degrade even further the quality of games on roblox, while incentivizing low quality cash grab games that get updated every other day? I feel like Roblox should be doing the opposite here. Hopefully they will tweak this system before pushing it to everyone in 2025.

[Offpath]

This sounds an amazing change if my interpretation is correct. My questions revolve around what exactly does “frequency” mean here?

• I would assume we’re trying to target higher aggregate playtime right? So a game where a player enjoys five separate 10-minute sessions is a great example
• But if there’s a game with a high crash rate – and that results in a high “frequency” of play – is that going to benefit from this change?
• Or worse, a developer who sets up a teleport network A → B → A in order to juice frequency. Is that going to exploit the system too?

[Noble_Draconian]

@Offpath

I would assume we’re trying to target higher aggregate playtime right? So a game where a player enjoys five separate 10-minute sessions is a great example

What about a game with one 50 minute session, vs 5 10 minute sessions? Are we now incentivizing games with shortform / less content over games with longer form content?

[jackjenningsdev]

I hope this change would see those games as being equal. Which they should be - because it would incentivize both longer, single-play sessions; but also casual, short-form, but highly re-playable games.

“Daily/weekly/monthly engagement hours per unique user” could actually be a pretty good statistic that rewards good games, whether they’re one-off, 3 hour long story sessions, or 5-minutes-a-day casual experience.

[Real_Daytern (Daytern)]

If the algorithm considers multiple factors and rates games on a profound level, it will definitly deliver better results, that’s why I welcome such updates.

As always with these changes there might happen some unusual traffic, but I really think it will be a positive one to find new games.

Another important metric is the memories the experience gave them, so glad that will be considered too within the amount of times the players come back.

[starrrydays]

Hi all,

Thanks for your questions — we’ve answered some of the most common ones in the body of the post. Hope this helps!

[Iliuminatis (・ロ・)]

@starrrydays

Frequent crashes mean a poor user experience, which doesn’t help the discovery of an experience; it hurts it. Our systems already account for this and will continue to do so.

This sounds like a feature that will be exploited by purposely using underperforming old devices to attack competitors games.

[Ovon30650]

@starrrydays

• One user behavior alone is unlikely to disproportionately impact an experience’s discovery. As an example, while we value robux spent, completely free experiences that engage users in other ways can still rank well. Our intent is to rank experiences and updates with a broader definition of success.

Great point my game has huge session time(18 mins) but less retention. There should be indicators on the algorithm for more than just one funnel of “good game” should be different metric combinations that give it more recomdaitons.

[EtNowis]

@Noble_Draconian

Currently, if an experience historically has had bad monetization but then receives an uptick in spending due to updates being released that address that problem, the algorithm continues to funnel non-paying users into it despite the changes / improvements made to monetization.

I’m shocked to hear about these. Where can I get these information or where it is discussed about please?

[Noble_Draconian]

Mostly conversations with other devs, there isn’t a centralized forum thread.