The recent update forcing users to verify their age with either AI age estimation technology or a government ID is ineffective at its proposed goal of child safety. Not only that, but it can actually be MORE dangerous to child safety than if it didn’t exist at all. I’ll do my best to explain why this is, and some possible solutions.
- What do you want to achieve? Keep it simple and clear!
What I, and everybody else, want to achieve is a platform that is safe for children. A platform that users aged 17 and under can enjoy and interact with without the risk of being preyed upon by child predators or being unwillingly exposed to sexually explicit experiences, avatars, and other media.
- What is the issue? Include screenshots / videos if possible!
The recent update forcing age verification and restricting users’ ability to chat to their relative age groups seemingly aims to promote child safety. However, due to the technology being flawed and estimating users as older or younger than their actual age group, the craftiness of some child predators in tricking the AI, and the harm the update does to platform-wide communication, I believe it should be removed or made optional.
Firstly, AI age estimation technology is flawed. Below is a screenshot from discord where my 18-year-old friend tried the age estimation technology and it estimated that she was 13.
This is just a case of it happening to someone I know, imagine how often it’s happening to others. On top of being frustrating for users with no malicious intentions, I want you to think of how dangerous this can be. Lets say a child predator tries the age estimation and, due to having a younger-looking face, it estimates that they are 13 or younger. The technology has just given that child predator access to the exact age group that they want to prey upon. Inversely, what if a minor who looks older than they really are uses the age estimation and it estimates they are 18+? They are now unwillingly exposed to an age group that may contain child predators or other adult users with malicious intentions.
Secondly, child predators are going to be relentless in their attempts to fool the age estimation into giving them access to young victims. It’s extremely naive to assume that they’ll just give up if the AI correctly estimates them as an adult. There are child predators who have children of their own or work in a profession regularly interacting with children, and they may use one of these children to trick the AI age estimation technology. Even if they don’t have access to children, they could use generative AI to generate an image or video of a child that doesn’t exist and use THAT to fool the technology. They will absolutely brute force their way into the age range they want.
There is also the possibility of data breaches. Personal user data is held for up to 30 days, which seems highly unnecessary for something that takes the AI a matter of minutes to process. No data is ever 100% safe, no matter how heavily it’s guarded, and if a data leak happens, THOUSANDS of pictures of children will be leaked and almost certainly used for malicious purposes by child predators.
Lastly, even if this technology may prevent some child predators from being able to interact with potential young victims, it does a great deal of harm to the majority of the userbase who DON’T have any malicious intentions. One game I, and many other users, play a great deal of is Dandy’s World, a game that relies heavily on communication with your teammates. Callouts such as telling your teammates to get to safety, notifying them that there are dangerous enemies near them, or calling out valuable items are common and absolutely necessary to making progress in the game and staying alive. With this update, users are now going to regularly be unable to communicate in any way with their teammates in Dandy’s World and other heavily teamwork/communication-focused games, which will lead to a worse user experience and these games possibly losing players through no fault of their own, which is unfair to the players and the developers of these games.
- What solutions have you tried so far? Did you look for solutions on the Creator Hub?
The implementation of solutions is up to the ROBLOX team, not me. But I do have a few proposed solutions.
You guys seem pretty dead set on using AI for moderation/child safety purposes. One solution I propose is to have AI scan in-game chat in real time and any time something suspicious is occurring in chat - such as a user trying to lure another user off-platform to discord or other social media, or an inappropriate/sexually explicit discussion happening - have the AI temporarily restrict the offending player’s chat access while the messages are forwarded to a HUMAN moderator, who can then determine whether or not further moderation action is required. Much less inconvenient to players than segregating them based on age.
Alternatively, if you insist on using this AI age estimation technology, have it be optional. Allow parents to use parental control setting to restrict their child’s chat access to users of their own age range, or disable their chat entirely if they think that’s what’s best. This will put the choice into the parents’ hands rather than forcing every user on the entire platform to abide by an update that they are almost unanimously unhappy with.
It’s clear that this update was an extremely poor decision. Your market cap has dropped by about 11% in a matter of 2 days and appears to still be dropping, nearly the entire community is unhappy, and as outlined above, this does not do nearly as much for child safety as you claim it does and in some cases is actively DANGEROUS to children on your platform. Please make the smart decision and revert this update, or at the very least simply make it optional for the parents to decide. I want roblox to be as safe as possible but this is not the way to do it.