[Update] March 15, 2024
Hi Creators,
We have heard loud and clear that you want to learn more about how Roblox’s Discovery system (including Home, Matchmaking, Search, Discover, and Notifications) is working along with our vision for the future so you have a framework to understand the product decisions we are making. Today, we are excited to kick off a series of posts, with updates on what we are testing and why, what we have shipped, and the impact of these solutions for the creator community.
This first post focuses on high-level topics such as our Discovery mission and principles, where we are today, how the current Discovery system works, what it means for you, and a preview of what is coming. In future updates, we will cover other products related to Discovery, such as Ads and Creator Analytics, as well as go into more depth and share case studies to inform actions you could take.
Mission and Principles
Our mission for Discovery is threefold:
-
To connect every user with the best creation and community for them.
-
To connect every creation with the most relevant audience.
-
To keep the platform fresh, vibrant, and exciting with new creations and updates.
The first two aspects amplify each other: finding the right audience for every creation encourages you to continue investing in or developing new experiences which, at scale, provides a higher variety of great content for our users, and over time brings a larger and broader audience to the platform. The last aspect amplifies the first two, creating a healthy ecosystem. Today, Discovery primarily focuses on experience recommendations, but has also expanded to include social connections, and will include other creations (such as Avatars) in the future.
We design and continue improving our Discovery system according to three principles:
-
Merit-based Discovery: Every creation should reach an audience based on its quality (which often correlates with how much creators invest in it, such as updates, events, etc.) and determined by how much people love it (measured by how much time and money they spend on it). This allows you, our creators, to focus on building instead of finding an audience.
-
Agency: You should have agency to reach and describe a creation to an audience. Agency includes the ability to communicate directly and proactively with users, such as sending notifications or buying ads to acquire new audiences. Agency also means you can understand, via insights, forums, and analytics, how to improve a creation to earn more recommendation impressions, based on its performance relative to the rest of the community’s creations.
-
Predictability and Actionability: This means that you can understand the “why” behind changes to the Discovery system so you can take clear action to benefit from the changes. This will be a journey together as we consider many factors such as technology changes, our growing and diverse user base, and the ever evolving creation ecosystem. We aim to communicate upcoming changes and associated impact proactively.
As an extension of the above mission and creator-related principles, we also consider the following product values to help steer our roadmap:
-
Seamless and Immersive: We are lowering the barrier between Discovery and experiences, e.g. server pre-warming allows a user to nearly instantaneously join an experience. Roblox emulates the physical world, and we believe Discovery should also be increasingly immersive and provide more visual information for a user to make a decision. Recently, we announced that we are testing 16:9 images as a step in this direction.
-
Fast and Flexible: Unlike the physical world, a user in Roblox can immediately teleport anywhere, and our system can react to user interest quickly and automatically, at any time during the lifecycle of a creation. We also want this system to scale along with new content types, formats, and metadata, which will give you and Roblox users more opportunities to connect to each other.
-
Personalized and Broad: Given every user has a diverse set of interests, friends, and is in a different state of their journey on Roblox, we are building state-of-the-art algorithms and UI that personalize Discovery for every user. Meanwhile, we seek to connect every great creation with an audience so Discovery indexes and ranks every creation across Roblox. A recent example is the merge of recommendations with friend activity, which increased user engagement, as well as the number of unique experiences that are recommended.
Where We Are Today
We have come a long way together. In 2019, while our community had 1.5 billion experiences and 70 million monthly active users, an experience needed to be in the top 1,000 most popular to get recommended on Home. Now, nearly every experience can be recommended. Year over year (Q4 2023 vs Q4 2022), we have seen:
-
A 56% increase in unique experiences recommended
-
A 61% increase in new experiences appearing in the top 150 within 90 days of launching
-
A 9% increase in time spent from users’ sessions initiated from recommendations
We have not accomplished our mission yet, but we are making progress towards our goal to enable a diverse range of creations to succeed and find their audience. We are constantly and pleasantly surprised by the types of creations taking off as a result of these changes.
How Discovery Works
Discovery serves more than 7.5B impressions per day, presenting a massive opportunity for creations to find an audience. We optimize Discovery for three main purposes: Engagement, Monetization, and Ecosystem Health. The first two quantify how much people love a creation by spending time or money on it.
-
Engagement includes multiple dimensions about how people spend their time, such as depth (play through rate), length and frequency (session length and frequency), stickiness (retention), socialness (friend inviting and co-experiencing) and in the future degree of joy (user satisfaction such as thumbs up rate).
-
Monetization focuses on whether users are getting enough value so that they want to spend money to further improve their experience (payer conversion) and how much they are willing to spend (revenue per paying user).
-
Ecosystem Health ensures we are building a vibrant multi-sided marketplace. We want to give every great creation the chance of succeeding, help new creations find an audience, and reward creations that bring new users to the platform as well as existing creations that are continuing investment with updates, events, and more.
We will continue improving transparency and recommendations for all these three main purposes, through tools such as Creator Analytics.
We currently have several product surfaces for Discovery: Home, Matchmaking, Search, Discover, and Notifications. We will cover the mechanics of each surface in this post, and introduce updates on other products related to Discovery (Ads and Creator Analytics), in future posts.
Home
Home is a user’s personalized view of Roblox, which includes Continue Playing, Friends List, Recommended for You, Sponsored, and People You May Know, and recommends the best content and community for them with the current focus on experiences and connections, but in the future it will expand to include other types of creations.
Today, Home has >90% of traffic on Roblox. It is powered by a multi-stage algorithm that starts with all eligible experiences on Roblox, generates candidates based on the three purposes of Discovery (engagement, monetization and ecosystem health), and culminates into a ranked list of content that we believe users will most enjoy. For a detailed overview, please see our talk from RDC. Note: We now show the Home recommendation conversion rate graph in Creator Analytics.
Matchmaking
Matchmaking enables users to join experiences on the platform, by tapping the play button to join the perfect server. We have been focused on improving scale, specifically on our ability to handle the large traffic spikes that experiences are reaching on Roblox. Over the last year, we have had experiences reach 1m+ concurrents, while still enabling friends to play together. In addition, we have significantly improved join times; we rolled out pre-warming in late October across the platform, and now ~70% of joins that would have had to wait for the server to start are instant.
Our focus for 2024 is on the matchmaking algorithm, which currently uses many signals across latency, language, age, and social to find the best match for the user. In the coming year, we plan to enhance our core matchmaking algorithm to be adaptive to the type of experience, and in addition, we are exploring ways for creators to have more control over signals that are most important for matchmaking decisions.
Search
Search aims to be a companion (easy to find and use), concierge (understands user intent; safe and trustworthy to use), and a rescuer (helps when recommendations aren’t quite what the user is looking for). Similar to Home, it is evolving towards a more universal tool for all creations and connections at Roblox.
Historically, Search has primarily focused on relevance based on exact search queries and limited metadata such as titles. However, while we continue to improve search quality (enabling more opportunities for our community’s creations to show up), we are increasing efforts to understand user intent. For example, recently we launched semantic search for all of our officially supported languages to enable users to find experiences through natural language queries, such as food games or avatar editors.
Notifications
Notifications elevate timely and actionable information to users. Historically, we have focused on building and scaling social notifications, such as friend requests and invitations. Recently we opened this system for creators to engage with users directly while they are away. Milestones, high scores, and other key moments can be delivered to users as personalized notifications.
We are also actively building systems to prevent spam and ensure that notifications meet a high quality standard before they are delivered. We provide two distribution channels for notifications: push notifications and notification stream, the latter of which will be the primary destination for users to receive notifications from experiences. We are actively developing and testing other potential channels and creator-facing tooling, to further amplify reach.
Discover
Today, Discover has <5% of traffic on Roblox. We are committed to dramatically evolving it to better serve our community and provide more diverse recommendations to our users. Discover is also curated as a zeitgeist of experiences. We think there is a bigger opportunity in this space and hence we are working on something more powerful and dynamic for the future. We will share our plans in upcoming updates soon.
What It Means For You
We often receive questions about ways to find and grow an audience, and the high-level answer is always the same: build a great experience that users love, tell Roblox about it (via high-quality metadata and media), experiment with finding your audience, and nurture it. For example, here is a typical journey for a new experience to obtain a matched audience, which also outlines what a creator can do: Build → Pass Moderation → Grow Initial Audience → Get Assessed → Iterate → Get Rewarded.
-
Build: Creator builds a great experience by emphasizing craftsmanship, quality, performance, device compatibility, localization, and good game design. Deploy this experience to Roblox along with high-quality metadata and media.
-
Pass Moderation: Roblox’s Trust and Safety systems moderate the assets shown within experiences and verify that it’s safe to publish on the platform.
-
Grow Initial Audience: Creator finds initial audience via on-platform Ads, community, and friends, and off-platform community building and influencers, etc.
Example: LifeTogether developers bought some Ads to get an initial pool of users in August 2023.
-
Get Assessed: Roblox’s algorithm gives an initial set of impressions to understand how the experience resonates across a wide variety of users. Then the impression rate normalizes.
Example: This small Ads buy from August 2023 gave the algorithm enough signal to start ranking LifeTogether and building up impressions.
-
Iterate: Creator continues to polish the experience to improve engagement. They tend to do these four things that work well:
-
Great meta-data/media: Using high-quality meta-data and media (such as exploring with better tiles) to better attract an audience;
-
Know the audience: Use Creator Analytics (and off-platform analytics) to understand engagement, progression and monetization trends;
-
Launch thoughtful updates: Launch thoughtful content updates and automatic translations that continue delighting the audience;
-
Improve Monetization and Retention: Introduce new products and price points with dev products, gamepasses and/or subscriptions. Use Creator Analytics to identify potential ways to improve user experience.
Example: Months after LifeTogether launched, especially after their 11th update, they blew up on TikTok+YouTube during Christmas.
- Get Rewarded: Roblox’s algorithm sees the spike in audience engagement and increases impressions on Roblox.
-
Example: With as many as 14 updates, LifeTogether continues growing to become a breakout experience on Roblox.
Example: Similarly, Rainbow Friends had a release of Chapter 2 on June 2nd, 2023, which triggered a sharp increase in impressions from recommendations.
We acknowledge that Step 3 - Grow Initial Audience - is not yet ideal, which we plan to address. For example, we will experiment with automatically awarding initial impressions to explore new experiences so that we can understand and match them with the audience better. We are also working on giving you more control of this process, e.g. a beta mode that postpones this algorithm exploration mentioned above until you are ready, and sharing more insights via Creator Analytics on Step 5, so that you can take action to reach churning/churned users.
In addition, the inputs to our Discovery system not only include user activity and connections, but also the rich information we know about an experience - including those provided by you. We suggest building high-quality metadata and assets (titles, descriptions, thumbnails, videos, etc.) so that we can properly find the matching audience for your creation.
Last but not least, as our Discovery system also strives for ecosystem health, diversity and the incremental value of a creation would matter to the platform more and more. Hence you may also want to consider external factors, such as the evolving tastes of new user bases and the introduction of underserved or category defining experiences.
We know there are still many questions, such as why impressions would spike, what might cause drops in impressions, or what happens to paid access, etc. We look forward to diving into all these topics more and how you can take action accordingly in future posts.
What’s Next?
Thanks for reading so far.
We recently shipped a Creator Analytics update to give you more transparency on your home recommendations conversion rate. We will discuss more on upcoming changes as well as deep dive on case studies, through this series of posts, RDC (such as Growth Q&A), Creator AMAs, Creator Roadmap, and forums in the coming months. We hope they will give you more perspectives on what we focus on accomplishing - which ultimately is about making all of you successful.
We appreciate you coming along on this journey. Remember that we are building this together so please continue to share your feedback and any further details you want us to go over in future posts.
Thank you!
@rbnflx, @been_made, @karunamat9ta, @Christ0pherus, @Signal_zzz, @xBigBroGregx, and the rest of the Discovery and Creator team
