[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement

Resources Community Resources
1
Summary of the issue.

As stated in the official Roblox document, in order for your game to be playable for ≤ 16 ages, you need all of these things:

  • First published to 16+ users.

  • 250 unique plays by the 16+ users. This will be evaluated on length of playtime, account age, play history, and if the user has recently spent somewhere on Roblox to ensure the engagement is genuine.

These two rules create a paradoxical problem for games that are designed for kids, but are forced to cater to 16+ users before they can deal with younger ages.

How is this related to machine learning?

The paradox problem that I stated above can actually be solved by personalizing the games. Most game developers may encounter a form of personalization such as language localization and device-specific UIs. However, for this specific age-related problem, you will require gameplay personalization, which is not exactly an easy problem for game developers to solve. After all, how can you determine what core gameplay that you need to change just to adapt to different age groups?

This is where machine learning comes in.

Machine learning allows you to predict the behaviours of individual players for a given input data. These predictions can significantly increase your chances of achieving 500 16+ user unique plays and get to your younger audience faster.

What can I do with machine learning exactly?

As we previously discussed, it can help you extend players’ session length by learning from players’ data such as:

  • How much they played in the current and overall sessions.

  • How much in-game currency they spent in the current and overall sessions.

  • How much in-game items they acquired in the current and overall sessions.

We can also use machine learning (and reinforcement learning) to create AIs that adapts to players, indirectly extending session length. For example, self-learning sword-fighting AIs that could temporarily act as placeholders for missing players, ensuring that the existing players do not churn as much as possible. When combined with Roblox’s text generation API, you can create AIs that feels like real players, further reducing player churn. Try not to confuse this with “botting”, where “botting” often involves third-parties using external AIs for item gathering to sell items in the black markets. The in-game AIs that are stated here are aimed for extending existing players’ playtime more than gathering items to sell for black markets or create fake engagement metrics through the creation of fake players.

If you want to also monetization issue as required by Roblox, your games’ shops can use recommendation systems powered by machine learning.

Does it really work?

I will approach this in multiple ways: big player examples and research papers.

Have you ever wondered how YouTube managed to make you feel like you don’t have to search through thousands of videos that you may never watch and potentially leave their platform? They used machine learning and deep learning in their recommendation system.

There are quite a number of research papers that used machine learning for personalizing games, showing high accuracy and good performance. Here are some of them:

Aren't machine learning models invasive?

To be honest, that depends on what data you feed on and what are your goals are.

Though, if you want to be ethical about it, then I can say this: The more that the player have fun, the longer the players’ session length.

That’s it. Spending money isn’t one of them. In fact, for players to spend money in your game, they must spend time in your game. While I did say people can use recommendation system for monetization, recommendation system doesn’t always have to involve real money and instead you can use in-game currency. Players basically have two choice with this route: buy in-game currency with real money or they have to grind in your games.

Aren't these machine learning models required to comply with personal data protection laws?

The answer is “depends on the country”.

Most machine learning models tries to learn patterns from input data you give. Once trained, you can remove the input data forever, leaving the models with learned patterns. Because these models learns based on patterns, different inputs generally produce different predictions. This tells not much of players’ information, and instead tells what combination of factors that produces that result.

While some jurisdictions may classify derived patterns as personal data, the model itself contains no “Personally Identifiable Information”.

How can I start implementing machine learning models?

To be honest, your choices are generally based on budget, how many data that a single model you want to train on and so on.

  • Using DataPredict, which is a general machine learning, deep learning and reinforcement learning built in pure Lua developed for over 3 years. It has over 136K views and 265 likes, making it a top 40 of most viewed community resource on Roblox DevForum. You can run these models on the server and the client since it was built in pure Lua. I even got high-value project tutorials that could help addressing with this issue.

  • Using externally hosted machine learning models, which gives you the ability to train models that are capable of handling large datasets and centralized global training.

For people who are tight in budgets, DataPredict will be a superior choice since you will not have to pay for external API costs. Just use Roblox’s servers and clients with the added benefit that you have a bigger pool of compute resources and minimal latency when compared to using third-party providers.

However, the core process of implementing machine learning models are always the same:

  • Baseline Pretraining: Models are initialized using designer-provided or historical baseline data to avoid cold-start instability.
  • Incremental Online Learning: Models are configured for controlled online updates using small learning steps, sufficient statistics, or optimizer constraints to adapt safely to evolving player behavior.
  • Partial Rollout: Only a subset of players (typically 20–30%) are exposed to the adaptive system initially.
  • Metric Monitoring And Kill Switch: Engagement, retention, and revenue metrics are monitored continuously. If degradation is detected, the system is disabled and diagnosed before further rollout.

You can contact me directly in DataPredict’s Roblox DevForum Post for further consultations regarding implementing machine learning in games or the DataPredict itself. I’m am often active in this platform.

If you still don’t get the use cases of machine learning in games… Well… Here’s the “Too Long, Didn’t Read” version. This is basically the “time-to-leave” prediction model, where you can hook events (like currency multiplier or limited-time items) during prediction to extend playtime when the player is about to leave within 5 seconds.

Q & A:

Question: What can I use machine learning for in games?

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #2 by eatabler0ck

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #3 by MYOriginsWorkshop

Question: How to use machine learning tech in simple pet simulator games?

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #11 by eatabler0ck

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #12 by MYOriginsWorkshop

Question: How hard is it to learn DataPredict?

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #13 by prO005modeer

[GUIDE] Passing The 250-Players Requirement: Machine Learning Strategies For Retention And Engagement - #14 by MYOriginsWorkshop

24 Likes
2

Lemme give a scenario:

Say I’m a new developer who made Pet Simulator (pretend it doesn’t exist yet). How would I apply “machine learning” to change my game when the art direction, depth, and gameplay loop are made for younger children and don’t appeal to a 16+ audience?

6 Likes
3

Giving me hard questions already? Excellent.

The main issue is actually on engagement, not the art, depth and gameplay itself. We need those players to stay long enough to count as a “qualified user”.

Sure, you can’t change your current art direction or depth through machine learning, but you can change how gameplay loop so that it feels appropriate to their age groups. Not to mention, even some adults will still prefer kids’ aesthetics that are commonly found in puzzle games.

These are the examples how your gameplay loop can personalize based on their age groups.

Dynamic Difficulty Adjustment

Generally, we can expect children and adults exhibit different “staying” behaviours.

  • Children: Prefer easy and repetitive gameplay as they will get constant dopamine hits.

  • Adults: Prefer a balanced and unique gameplay as this is where most adults’ flow state lives.

Now, with these in mind, we can create a machine learning model where it generates these properties:

  • How hard is it to catch that pet.

  • The frequency at which that pet should spawn.

  • Pets’ statistics like health, attack and speed.

Personalized Grind Adaptation

Even in here, the children’s and adults’ gameplay varies significantly:

  • Children: Items must be easy to collect, a high rarity item will likely frustrate children into leaving the game permanently.

  • Adults: High rarity item are often associated with power or high value, leading them to likely stay more longer.

In here, the machine learning models can control:

  • How often these items are dropped.

  • The locations where these items should be dropped.

  • How time-limited or purchase-limited is the item before disappearing over the long term.

Personalized Shop Adaptation

If your game also involves shops that the players can purchase stuff from, you can expect:

  • Children: Prefer easy purchases.

  • Adults: Prefer frictions when purchasing these items, as they prefer a more challenging gameplay.

As a result, we can create a machine learning model that can control:

  • The items’ price.

  • The limit on how many items that the players’ can buy within certain duration.

  • How time-limited or purchase-limited is the item before disappearing over the long term.

Personalized Event Adaptation

Let’s say your game have a number of random time-limited events that will be permanently stay in your game. The machine learning models can be used to maximize playtime through:

  • Summoning those time-limited events at the perfect moment before they leave.

  • How often should certain events should be spawned.

  • How hard these events should be.

  • The duration of these random time-limited events.

As you can see, while your art and depth may be for children, you can do significant adjustments to the gameplay loop so that it doesn’t feel like a “kids” game after all. Also, you will be surprised on how adults are actually neutral about these art direction and depth as they prioritize high-quality arts and gameplays more than the art theme itself. I mean, we often see adult game YouTubers often play in a variety of art styles, even if it looked cartoonish.

Also, speaking of “cartoons”, this reminds of adults that also watch cartoons, where cartoons are often associated as “for kids”. Case in point: “Rick And Morty”, “Adventure Time”, “The Simpsons” and so on. This further solidify the proof that the art direction may not be as influential as you think.

That’s all for today!

1 Like
4

All of this is very speculative, coupled with the fact that changing core gameplay for age differences is basically just making one game for 16+ and one for kids.

7 Likes
5

Well, I’m just placing the solution out there. It is better than having no solution.

Even if it was speculative, I can guarantee you it would be at least better than random chance, given the nature of machine learning trying to learn from data. This will also compound the more longer you train and the more data you give.

Even if you argue that you can manually tune the gameplay… How can you detect these different players’ age based on their behaviour? Or better yet, how can you get an optimal adjustment for individual players based on players’ behaviour? In other words, you are essentially going blind. Machine learning would at least perform data-driven learning and make predictions based on data.

Also with DataPredict, I have designed a lot of these machine learning models to be capable of real-time adaptation in live servers instead of waiting for huge batches of data, making the improvements more instant than manual tuning.

6

I’m just bumping this so that people don’t get confused with the previous broken published guide that got moderated.

7

Lmao have you never played/seen Pet Simulator gameplay? The loop is just buy pets → get money → get better pets → get more money. You don’t “catch” pets.

While true in theory, one of the biggest appeals to Pet Simulator are the “huge” pets, which despite being so incredibly rare (and even expensive), kids will spend hours trying to get. These “huge” pets are also the most powerful in the game.

Again, this doesn’t help the issue of getting new players. The title, thumbnail, and icon are designed for kids. How do you get over the initial hurdle of getting 16+ players to join your game?

1 Like
8

Eh. Even if you mocked me not knowing about pet simulator gameplay… You can’t really deny the value of it, no? I also wanted to point out that some games might not have pet simulators being that simple. So, I just make a lot of generalizations. That’s why I don’t limit the use cases of machine learning just to avoid being laughed at by you.

Eh, I forgot to mention you can adjust the machine learning models to adapt to whatever your game mechanic you have. It is just really that hard to cover all of them since you can literally apply machine learning to a lot of places. I mean DataPredict has like over 100 models just to try to cover these use cases.

Well, for starters, calling your game “Powered By Machine Learning / AI” would certainly make adults curious about it, no? After all, machine learning and AI is a great marketing hook against… a lot of competitors not powered by machine learning or AI.

9

yes because gamers LOVE AI and would love to play a game blatantly saying they use AI

2 Likes
10

You do realize this is different to AI generating games right…? The gamers are specifically backlashing on that.

If you’re using AIs to improve their gameplay experience… It is not something for gamers to backlash at.

Do you really want to argue that an interactable NPCs that uses LLMs reduce their gameplay experience?

Do you really want to argue that a game that adapts to their playstyle through DataPredict so that they have fun playing your games is something negative to their gameplay experience?

Even if the gamers don’t ask for games that adapts to you through machine learning or AIs, they still want to have a good game experience. Not that hard to understand.

To be honest, you’re getting immature and close-minded at this point. Not a good attitude in a game designer. Good game designers are always open to feedback and find tools that are useful, so that they can make their games fun and have long playtime.

1 Like
11

Lemme back up a bit. This still doesn’t address my original question: How do I use this kind of technology in a simple game? (Specifically, “Pet Simulator”)

Pet Simulator gameplay/game loop

You start with one pet. You then walk around a map with coins, where you wait at each coin for the pet to “destroy it”, giving you money.

With said money, you can buy eggs to get a random pet that could be stronger or weaker–it’s luck based. You then use your new pets to break coins faster, getting money faster.

After a certain point, you can unlock more of the map with higher-HP coins that require stronger pets. The loop continues: get money → hopefully get a better pet → get money → unlock new area → get more money

The fun is in the game’s simplicity.

I can’t deny that a “smart” NPC with unique dialog is cool, nor can I deny that an enemy adapting to a player’s strategies is cool. But most games don’t need those, or are deliberately designed to not need those. NPCs convey lore and information. Enemies have attack patterns to play around.

I don’t understand how this tech is useful for most games, especially for simpler games where AI/machine learning elements add more bloat and confusion.

2 Likes
12

Thanks!

Now I really can work with this! To be honest, there are a lot of things that can be used for machine learning in this statement.

This one is pretty easy. Imagine that players are clustered around in one area, and you’re relying on these coins for spawning at random locations.

With machine learning, you can determine the clusters’ center of these players and spawn these coins based on how far they are from the center of these clusters. This maximizes the likelihood that one of the players will be able to get that coins. In other words, less coins lying all around that could cost servers’ and clients’ performance just to display or run the scripts for them.

Granted, you can just spawn with using an offset from players’ position if you want to go non-ML path. However, when it comes ML, its main aim to choose optimal locations based on mathematical properties, which could translate to improved game experience.

Oof. If you put luck-based, I cannot build a system around that.

HOWEVER. Have you ever… considered making that pet’s stats based on player’s current level and game statistics? This will not prevent them from buying from future pets, but if they want an upgrade, they have to buy a new one to match the player’s new level and game statistics. This basically makes all pets personalized to that player and potentially extend the player’s playtime.

Yeah… You’re not the only game developer here who has that problem. People don’t understand it since DataPredict was first released at January 2023. So this is why this post exists… To exactly tell people all the possibilities through education.

13

Although I see this type of game designing to be very useful, there are still some unclarities that I would like to resolve:

  1. How much learning experience is it actually required to implement AI into my own game? Would it take days or months to learn if I tried consistently learning DataPredict?

  2. How long would it take for the AI to actually learn how to perform a task really well? Would I need to “train” my AI for many weeks or months to be finally good at performing one or multiple tasks?

  3. As you may know, there is a common problem with AI’s sometimes having hallucinations about a topic. This causes AI to confidently giving wrong answers. Is DataPredict likely to come to this type of problem? If so, in what ways can you resolve it?

If you need any examples to clarify any confusion, feel free to ask me!

14

DataPredict follows scikit-learn’s API, but with a lot more customization and flexibility. Generally, python users find scikit-learn API easy to use, but I made it a bit more easier a little bit more for real-time model adaptation. You can learn the basic API here.

The API is as simple as this:

-- Basic model construction.

local LinearRegression = DataPredict.Models.LinearRegression.new({maximumNumberOfIterations = 30})

local featureMatrix = { -- Input dataset.

{0, 1},
{1, 1}

}

local labelVector = {  -- Output dataset.

{1},
{0},

}

local costArray = LinearRegression:train(featureMatrix, labelVector)

local predictedLabelVector = LinearRegression:predict(featureMatrix)

-- Saving and loading model parameters for storing trained models.

local ModelParameters = LinearRegression:getModelParameters()

LinearRegression:setModelParameters(ModelParameters)

I estimate… A day that you can learn DataPredict’s basic API. For power users, maybe 1-2 days.

Dependent on what models you are using + how much data you gather. For classical machine learning, like linear regression as shown above, it can converge to a solution with very few data with minimal computational resources. This means you can train them on live servers while the players play!

Though… If you go for reinforcement learning (a.k.a. self-learning AIs), they’re quite known for being very data-hungry. So I’d rather train them offline to get basic performance before rolling out to players.

This one, is only applicable to text generation LLMs. Machine learning generally tries to fit patterns to data when trained, making the ML models trying to output based on patterns. When compared to LLMs, LLMs always produces probabilistic output that leads to hallucinations.

You might see similar “hallucination” behaviour in reinforcement learning, but that is mostly because of how you design the reward signals. And yes, these RL models are quite known for outsmarting the “reward designer” by learning behaviours that maximizes the reward.

1 Like
15

Thank you so much! Appreaciate the help!

1 Like
16

People got mad at gta6 because they referred to the bot logic as “ai”

17

More like people kept confusing generative AIs with machine learning…

2 Likes
18

I don’t think gtas ai is even machine learning I think it is just a complex algorithm

19

Just classical decision trees most likely.

Though, games are playing around with a more advanced methods just like Arc Raiders game…

1 Like
20

I don’t know if you are familiar with the SCPF (serious roleplay) groups on Roblox, but I don’t really think players would like having bots or NPCs in a chaotic game where riots and containment breaches happen frequently. Like, I don’t think a bot or an NPC could recontain those anomalies, as it requires some skills to do.