Thumbnail personalization now remembers your existing winning thumbnails

Updates Announcements
1

Hi Creators,

Today, we’re excited to share an update to thumbnail personalization that lets you test new thumbnails while still sending most traffic on your existing winning thumbnail.

Our experiments showed a +0.19% increase in Qualified Play Through Rate and 21% fewer impressions going to a non-winning thumbnail.

How we’re improving thumbnail personalization

Previously, starting a new thumbnail test would reset the system and split all traffic across thumbnails evenly. This could potentially impact your QPTR in the short term while the system finds a new winner.

Now, when you start a new test that includes your existing winning thumbnail:

  • Your existing winning thumbnail will still receive the most traffic.
  • New thumbnails get enough impressions to measure QPTR and then traffic automatically adjusts within hours based on performance.

This approach ensures new thumbnails get enough exposure for effective testing without sacrificing your current winners’ performance. You’ll also know sooner how new thumbnails are performing.

In the example below, suppose you started testing your existing winning thumbnail and 4 new thumbnails at 7 PM. Personalization will now still send most traffic to your existing winner while testing the 4 new thumbnails (right) instead of splitting traffic evenly (left).

2591×1386 414 KB

Before today’s update, new thumbnail tests split traffic evenly (left). Now, your winning thumbnail keeps most traffic while new ones get tested (right).

What you should do

Whenever possible, we recommend that you always include your highest QPTR existing thumbnail when starting a new thumbnail test.

This way, we’ll continue showing most users your existing winner while sending a small percentage of traffic to test your new thumbnails. The total number of uploaded thumbnails remains at 5.
For more best practices, check out our documentation.

Let us know if you have any questions or feedback in the comments below.

Thank you!

156 Likes
2

This topic was automatically opened after 10 minutes.

3

It would also be nice to see thumbnail performance in real time or atleast become slightly faster, as a bad thumbnail could ruin your days qptr because you can’t see instant performance

42 Likes
4

This works better due to the ability to get more accurate and less tampered experiments with thumbnails. Glad our observations will be properly maintained!

While it may sound irrelevant, it’s something that could be considered for experience attention; could it be possible for experiences with video thumbnails to have those videos play when the mouse is hovering on an experience’s icon? Similar to having a video bit play when a mouse hovers on a YT video thumbnail (AKA Keep Hovering to Play).

Example: https://www.youtube.com/watch?v=KQU-_AvrDyI
(0:19)

12 Likes
5

Great call and I’m glad you brought this up. We’re actively working on a video-based Discovery solution. Recently, we started a pilot for creators to submit real gameplay videos for use in Discovery. Over the next couple of months we will be testing these videos on game details pages, hover states on Home, and other surfaces. More on this to come later this summer :grinning:

19 Likes
6

I think it could be useful to see other metrics alongside QTR. Players may click on one thumbnail more, but do they spend as much time in game? Do they spend as much robux?

I ask because my game has been experimenting with some blender-render thumbnails alongside in-game screenshots. The blender thumbnail gets a higher click through rate, so is shown the most. One time however I ran an ad that showed the various thumbnails, and I found that while the blender thumbnail also has a higher CTR in ad form, it ended up earning less play time and robux overall. This makes me wonder if there are cases where selecting the thumbnail with the highest QPTR may end up being a suboptimal choice.

11 Likes
7

can we have good discoverability please?
stuff like user ads, actually being able to search for genres, more flexible recommended section and other QOL things? (regarding in-platform trailers, the trailer should be added to the thumbnail gallery view to actually be watchable and not rely on exorbitant requirements)

8 Likes
8

Honestly my main concern is not being able to find my own games when searching them unless doing so with quotation marks “”

Bug or feature, it’s been present for years and hurts small developers more than anything. It’s difficult to grow a game when interested players can’t even find it though search!

16 Likes
9

Maybe this will make it more easier to see which thumbnail is better.

7 Likes
10

I’ll definitely try this out. I’ve got a game that needs some new thumbnail testing.

Since I don’t have access to the feature request section, I thought this might be a good place to suggest something. I was thinking it would be really helpful if you guys considered adding the AI assistant to help with thumbnail personalization.

I have a game called Blast Wars which, metrics-wise, is performing twice as well as the benchmarks on the home recommendation charts in my genre. However, because my QPTR is low and impressions are limited, I’m not getting the exposure I should be, despite having strong engagement metrics.

An AI assistant. even just a text-based one. that suggests thumbnail ideas based on the player subsets which are most likely to engage with your game (based off similar experiences) would be an insanely useful tool. It would save us a lot of time spent eyeballing and randomly experimenting with different strategies.

Because honestly, what works for one game’s thumbnail often doesn’t work for another. I’ve even had cases where what looked like sloppy thumbnails performed amazingly well on semi-serious games, which made no sense, but I just rolled with it anyway :joy:.

5 Likes
11

Great feature! I wonder when this will come out for game Icons since it will be useful if we can get our returning user’s qPTR up!

6 Likes
12

I feel like Roblox should inform players about the quotation mark feature because I bet most of them aren’t even using them when searching a game resulting in the feature being utterly useless to begin with, haha

6 Likes
13

I agree with @Abroxus, the system of specific search by title works in games page, yet there is no indicators that it works. I tell people about this feature, and even they are surprised this exists.

We could either have a search bar sign that says it, or a filter feature similar to marketplace and creator marketplace that makes it a feature by default optional via bool-value (on-off box).

On: Search by title automatically done without quotations

Off: Search by title manually completed by user

4 Likes