Automatic Translation Experiment Results And Next Steps

Updates Announcements
1

Hi Creators,

A few months ago, we announced that we planned to run an automatic translation experiment to fill in missing translations in experiences, with the goal of improving playtime and retention for players around the world. We’ve now run the experiment and are excited to share some positive results for experiences that are not presently translated, and what we want to do next for the community.

Our goal with this experiment was to determine if automated translation is of a high enough quality to drive more engagement and more positive player interaction. To do this, we experimented for two weeks in late June – automatically translating missing content (in-experience strings, experience names and descriptions) for a small percentage of users in supported languages. Existing manual translations were not affected, and all translations/experiences reverted to their original state when the experiment finished.

Experiment Results

Providing translations for experiences without them significantly improved search, discovery, and engagement. The numbers show the average impact on individual experiences. There are a few key observations to share.

Translations drive more users to experiences

Translating experience names and descriptions leads to more users finding and playing an experience. In fact, experiences saw an average increase in qualified play-through rate of +0.87%.

Qualified play-through rate measures engaging plays (not accidental clicks or quick bounces) after seeing an experience in recommendations.

Users play translated experiences longer, especially those who prefer the Japanese and Korean languages

This reflects how translating in-experience content impacts player engagement after they join an experience:

  • Experiences saw an average increase in overall playtime of +0.87%
    • New-to-experience users saw an increased playtime of +1.06%
    • New-to-experience Korean and Japanese speaking users saw an increased playtime of +4%
  • Also, experiences saw an overall payer conversion rate average increase of +0.95%
    • And for new-to-experience users, an increase of +1.16%

Playtime is the total time users spend in an experience, while payer conversion measures the percent of users who engage with paid features or content. You’ll notice that there is an even greater impact on users who are new to an experience.

Next Steps

Given the positive impact of localization on key metrics, we highly encourage creators to make use of our automatic translation tools, so that all experiences are accessible to any player.

In the next few weeks, we plan to introduce a player account setting that enables users to see translations in all experiences they play. This setting will have no effect on any experiences that are already fully translated by the creator. However, for experiences that are missing translations or not translated at all, when this setting is enabled, automatic translation will translate any strings the player encounters. We will never override a manual translation if it exists.

As always, we are continuously working to improve the quality of automatic translations, and to make Roblox feel native for users around the world. Please let us know if you have any questions.

Thank you!

87 Likes
2

This topic was automatically opened after 10 minutes.

4

This is very useful, I’m not sure how well the translation works since converting the strings back to english using google translate makes them significantly different than what they started with. If the system improves this is a really good feature though.

10 Likes
15

All of the statistics shared here are really small numbers, do you think you could share averages/medians/a realistic example to compare against? 0.87% is huge when the baseline starts at 1%, but 4% playtime increase does not sound very significant. Having a baseline to look at with these stats makes them more meaningful.

13 Likes
16

Well, I doubt it would make sense to talk about it in a Development Discussion where Roblox only has their moderation exist that does their job while those that publish and establish valuable informative statements could highly be more sufficient to approach towards making something be likely have more of a chance to be resolved like an actual post by Roblox to speak forward with the claims and problems that have occured overall so far in our platform if you see where I’m going.

(too much yapping and someone will start nitpicking flaws so im ending it right there)

8 Likes
17

Would it also be possible to open some kind of feedback system for people who decide to use this automatic translation feature, so they can give feedback on the quality of the translations it produces?

5 Likes
18

This probably won’t see much use once the experiment is over due to how many countries are banning ROBLOX for safety issues. It does look promising though, and I hope it can be used by the few nations who don’t ban the corporation.

14 Likes
20

I second this, I think before adopting this it’s fair to get proper data since from personal experience automatic translation tends to do more harm than good.
As an example “Blade Ball” in portuguese translates to “Ball of Blades” or “Volleyball Legends” translating to “Legends of Volleyball” which hurts international branding quite a bit when there are discrepancies with what the title is meant to be. (Think making shirts or other merch with the experience’s name and having people confused on why it’s so drastically different from what they’re accostumed to)

A good way to illustrate this is like if you’re watching a video guide about an experience you like made by a channel that speaks another language and the closed captions translates something important in the experience into something different from what’s displayed on Roblox. Which from what I’ve experienced and seen can be an issue as well.

All in all I think this can be a step in the right direction as long as there’s more specifics on the experiment conducted and possibly personal feedback as @pracsiHD has suggested.

5 Likes
21

This is better answered by comparing these stats against the same kind of delta stats from games that were manually translated.

I just wanted to understand what these %s look like on top of a typical baseline.

5 Likes
22

Hey @PeZsmistic! Thanks for your feedback - for some more detail/clarification:

The baseline for this experiment was experiences in their current state (where they might be missing translations). The numbers we gave represent the increase compared to this baseline - so what we’re saying is that on average, a fully translated experience sees 4% more playtime from Japanese and Korean players compared to when it’s missing translations.

That number is actually quite large by our usual experiment standards! Playtime uplift compounds beyond the two week window (which is how long we ran the experiment for), so if users return over the course of several months, there ends up being much more upside.

7 Likes
23

Hey @pracsiHD - we’re definitely going to be collecting feedback on translations. We have a translation feedback tool today in the experience menu that allows players to select text in the experience and give us feedback on bad/missing translations. We store all of this info and use it to track quality and continuously improve our machine translation system.

Screenshot 2025-08-13 at 3.46.33 PM
Screenshot 2025-08-13 at 3.46.33 PM1656×1116 79.9 KB

Moving forward, we also have plans to display translation feedback directly to creators in Creator Hub so you can get get an idea of sentiment from players in different languages.

7 Likes
24

Please bring back the “Translate to original language” button…

5 Likes
25

Thank you Roblox for a translation update! T
It’s so good! :heart:

4 Likes
26

I’m new to hearing about this, I think it’s a great idea though!
Does this use a translation program that was built by people at Roblox or is it through using google translate/another program not made by Roblox?
Just curious

If it is made by people at Roblox, is the translation process for translating two languages, where neither are english, work similar to:
Translating language 1 to english then taking the english translation and translating it into language 2.
Or is it something else? I imagine it may already be difficult with the different sentence structures between english and another language, perhaps doing this method could help?
Set up every language with how to translate to english and reverse, making it easier than taking every language and figuring out the translation with every other language.

Having a base language to translate to then translating the base into the other language could make proper translations simpler and easier. If I’m wrong on this I’d be interested in hearing the proper process

2 Likes
27

I prefer quality of translations to quantity of translations and players.

This will get weird for custom names/nametags, and there are contexts where things should not be translated, e.g a Japanese-themed area where hiragana is part of the aesthetic.

Numbers aren’t everything.

Will this still respect auto-localize property so it can be disabled in certain places?

Of course I can see how this benefits players of less represented languages.

3 Likes
28

This is like the 10th time you announce automatic translation. Yes we know.

4 Likes
29

I see this experiment as a great opportunity to analyse the effect of translating everything to players, but it isn’t the most accurate way to measure the actual quality of the translations.

While broader language support can increase engagement — since players recognize their own language — it doesn’t reveal how well the content is translated.
I notice this especially in the official Roblox documentation, where AI translations may encourage non-speakers to read an article in the first place. However, until the AI improves, relying on a different approach could yield better results, as the risk of only partially understanding content — or misunderstanding it entirely — still remains.

As someone who speaks more than five languages, I can confidently say that in-experience translations need more refinement, as I know locals wouldn’t understand certain parts.

Either way, I’m curious to see you experimenting further and watch the quality of this automatic system improve over time.

2 Likes
30

Quoting from the previous thread in case someone tunes in now and is about to ask the same question I was going to:

(I am assuming this policy hasn’t changed.)

2 Likes
31

The chat translation barely even works, unfortunately. I’ve already mentioned this on another post. Someone will send a message in the chat in a foreign language, and there will be no translation, let alone an option TO translate it.

1 Like
32

Is the percentage increase in qualified play-through rate additive or multiplicative? If it is multiplicative, wouldn’t that not be a very significant increase?

Editing to add I have the same question for the other percentage based stats like conversion rate as well