Limiting AI Moderation to 1 day bans

Feature Requests Website Features
, , , ,
#1

As a Roblox developer, it is currently too hard to trust uploading any asset on Roblox without worrying of a false moderation

If Roblox is able to address this issue, it would improve my development experience because I have gotten a lot of false moderations in the past year that wouldn’t make any sense.

Recently, there’s been a lot of issues with the Roblox Moderation taking action against innocent items & creators.
To my knowledge, ever since Roblox was known to migrate to a fully automated moderation, there’s been a lot of issues, where creators got banned for 3 days, or even terminated due to a false moderation action that happened on their account. UGC creators and other developers had their assets randomly deleted and their accounts terminated over these asset deletions. This is very concerning as somebody whose income is currently reliant on Roblox, because I can be banned any day for no reason just because the “AI Moderation” detected some asset as Inappropriate.

There are a lot of known UGC creators whose creations were featured at RDC or “Today’s Picks” and later on their creations got deleted.

Many of these deletions also happen because of mass reports/DSA reports, which the AI somehow detects as “inappropriate”.

Twitter posts Regarding this issue

EthRonArt’s Account deleted
vamonoz’ Account deleted
Lukas’ Account deleted
FireAtacck’s items deleted
ADbros’ item deleted
Reverse_Polarity’s item deleted
blushannie’s Account deleted
Viewported’s Account deleted
noxyra’s Account deleted
RobloxRTC’s Account deleted
Schlep’s Account deleted

Even one of my friends has had such an issue, where his account got terminated, and wasn’t unbanned until I posted this tweet.
My friend’s account getting deleted!!

Please limit the AI moderation to warnings and 1 day bans, and add some sort of protection for top developers against false AI moderation. Possibly flag their assets as potentially inappropriate and let the human moderation team decide if the items are inappropriate or no.

AI moderation should only act as a precaution system to ban people for “breaking the ToS” until a human moderation is able to fully review the situation and decide if the user should be moderated or no.

I have also tried removing an asset off the catalog for the past few months, and I haven’t been able to get rid of a “warning action” taken against my account through support, even though I was told I shouldn’t receive any moderation for that item after the request. Please don’t allow the AI to moderate people whos’ income is reliant on Roblox. AI moderation will never do a perfect job and will always have false positives.

One of Roblox’s most popular games which was part of a few events, Dress To Impress, even had it’s icon DELETED for no reason.
image

EDIT: I just noticed my alternative account got banned for creating a SUPPORT CENTER GAME???
I have to keep up with the removal of inboxes, creating a support game and I get banned for it?

image
image883×645 80.3 KB

59 Likes
#2

Very interesting take on this.

Would love for at least something to be done about ridiculous bans due to automated moderation, rather than waiting for an appeal that takes the entire ban duration to even be approved!

11 Likes
#3

I think they should be able to do all bans except terminate. Roblox does not realize, but terminating someone could send their life into hell (if their main income is from Roblox). The termination process should ALWAYS have to go through a human first.

3 days later: moderation should be able to put accounts on hold. Allowing no one to access them until a human views it. This could be used instead of termination, then when the human moderates it, they decide to ban or uphold the account.

13 Likes
#4

The fact that account terminations can be automated and also be completely false positive is actually pretty scary.

Account termination should never be automated, people should never rely on fully automated systems to make such big and risky decisions.

19 Likes
#5

Funny things, I never get banned for a problem of uploading assets.

But yeah, I have see too much getting ban and account deleted for nothing.

4 Likes
#6

Limiting AI moderation must be limited, surely agree, but…

reasonable limits on moderation should be equal and apply to everyone regardless of their status. The idea that more successful money-making developers get special treatment over small or hobbyist developers reeks of elitism and points to some of the problems people suspect of moderation biases today in ignoring the glaring and brazingly simultaneous TOS-breaking and money-making games we see on the homepage. I as a small hobbyist should feel safe from both bad actors as well as abusive moderation from the company that hosts the platform.

In the past two years, or so, or at least since Roblox became a public company, moderation has gone off the rails in extreme and unpredictable directions. Like myself, I collect Ro-nation groups and base their quality on how rare the name of the group. Then in December of 2023, Moderation, out of the blue, gave me a 1 day ban and locked the group without giving me clear, direct explanation as to why aside a vague accusation without pointing to any evidence nor any chance to fix whatever moderation thought was wrong and every appeal was ignored.
Why were my very reasonably appeals over what could just amount to a small oversight be ignored meanwhile I see the same brazen and obvious TOS breakers almost every. single. day. on the homepage, even in the sponsored section??? I can only speculate, but it has to be that some developers are treated more equally than others.

2 Likes
#7

I agree with developers treated equally should be a thing, however if Roblox is currently unable to provided that, I would expect moderation to atleast not ban users as fast if they are for example, somebody like reversepolarity, asimo, minitoon, etc. users that were on the platform for a while and Roblox trust, should atleast be known not to break Roblox ToS, so the ai moderation should only ban such people in case of a real ToS Violation.

I would be more than happy knowing everyone is treated the same, and knowing that the AI moderation would just flag stuff as inappropriate or potentially rule breaking, until a real moderator can look into it, however, I think before Roblox gets to a point where that’s spread among millions of users, they could atleast try with some of the more known figures on Roblox. It’s legit so weird seeing so many tweets from reversepolarity that one of his items got deleted, after being showcased in today’s picks. same with other creators. I’m all in for equal treatment here, but for the beginning I’d love to see them atleast try their best, by testing how this would work on people that the Roblox team is trusting with not breaking the rules. And first of all I would be happy to see no more terminations happening from automated moderation. It’s stupid to get your only income get deleted by a robot that won’t achieve a perfect level of accuracy. It’s scary and concerning.

2 Likes
#8

I’m absolutely terrified updating assets to my main Roblox account. I have to use alternate accounts simply because of the false positive moderation.

5 Likes
#9

Yes please!
Also, can we please have appeals for AI moderation be handled by humans?

8 Likes
#10

I would recommend creating automated moderation for 1 day then an request could be sent to a moderator at ROBLOX to check the reason and set a reasonable ban time.

While, there might be CONS of this such as overwork it could be reduced by a certain time like less then 7 days to check the ban?

My view of this is people would not like it but this is just a suggestion.

1 Like
#11

This definitely should be addressed. I’ve seen multiple games/items that have randomly been deleted or changed, such as Experience Icons and Thumbnails, even some of my own assets have been falsely moderated.

The AI Moderation is becoming a bigger and bigger problem for developers. Let’s just hope something is done sooner rather than later.

1 Like
#12

yes, please.
there have been way too many false terminations from ai moderation.
if anything, ai shoulldn’t have permissions to ban at all and only signal to moderators that “hey, this needs to be checked”

5 Likes
#13

Please, add this. I almost got my account deleted.

3 Likes
#14

Moderation on this platform is completely backwards. Developers get handed terminations by poorly coded AI moderators managed by some guy in mumbai, yet bans related to hacking/cheating have to be reviewed on a case-by-case basis with a zero false positives policy and a clear and defined system of progression for the consequences. Fire whoever’s in charge of platform trust&safety, they are doing an absolutely horrible job.

3 Likes
#15

Definitely something that shouldn’t cause permanent, irreversible bans.

Earlier last month, Roblox’s automatic AI moderation for avatars falsely flagged anyone wearing Moon Seeing Stars, a limited hat uploaded by Roblox.

About 15 users were terminated, no warnings, nothing, and only about half of them had it fixed after going through appeals. There are two owners which have been inactive for a year and will not qualify for the one month you have to get any moderation concerns for your account resolved. All because Roblox was automatically moderating their outfits and assuming it has something to do with “moon man”.

Below is not even half of the users affected. Many of them were wearing a UGC addon (this is very common with limiteds), and Roblox later on auto-banned owners for just wearing the hat. One owner lost 4 million robux from this!.. and who knows what else of value.

image
image1156×932 212 KB

image
image1410×536 106 KB

image

4 Likes
#16

It’s honestly wild how broken the moderation system has become. At this point, it is quite obvious that there is some AI in play. no human could consistently mess up this badly. The current state is just unacceptable for a platform this big.

4 Likes
#17

That moon man/mac tonight ban reason is actually absurd considering Roblox themselves has a “moon man”/mac tonight hat…

Maybe they should delete their own asset before terminating others for wearing it.

2 Likes
#18

I was about to note that the people in that image aren’t wearing the original Roblox Limited, but a UGC variant.

I would like to see actual incontrovertible evidence that someone was banned for wearing a Roblox made asset, and a limited one at that.

Here’s the Marketplace Search for “Moon Seeing Stars”, a lot of UGC variants which could also be used as unwitting traps by both the creators of those items and people who purchased those items to find themselves unexpectedly banned one day.

Screenshot 2024-10-13 at 4.46.13 PM
Screenshot 2024-10-13 at 4.46.13 PM1920×810 70.7 KB

Screenshot 2024-10-13 at 4.46.21 PM
Screenshot 2024-10-13 at 4.46.21 PM1920×782 83.6 KB

Screenshot 2024-10-13 at 4.46.33 PM
Screenshot 2024-10-13 at 4.46.33 PM1920×787 76.8 KB

Being banned for Hate Speech for wearing a slightly different version of a Roblox made asset is worrying and beyond the pale.

Obviously there are UGC creators making varrients of Moon Seeing Stars because,
1. The Limited is popular and expensive, and
2, since Roblox made it, despite its controversy in what it’s referencing, the McDonalds Mac Tonight commercial and the dark meme that spawned from it, then it’s probably OK to make their own version.

If Roblox wanted to adequately deal with the situation surrounding “Moon man”, they should rescind the Moon Seeing Stars hat, give refunds out to anyone who owns one, and issue a statement why they took it down and removing all varient UGC moon seeing stars assets.

1 Like
#19

All of those users owned and wore the original MSS by Roblox, here’s a few UAIDs you can verify with: 234424180, 236111993, 313704854.

The UGC in question were black and pink shades meant to be worn with MSS as a UGC add-on, but Roblox automatically moderated many of these users that were wearing it with MSS.

Shortly after, Roblox started banning anyone who even owned the item. Here’s a couple of users that are currently banned that wore nothing except for MSS:

A couple of users were likely given understandable moderation (even if I find a termination harsh), as one of them got unbanned, but then set their avatar to something questionable, while the other one is terminated and wearing a “Mac Tonight” inspired shirt.

Moon Man isn’t necessarily an inappropriate character, but the icon has been given negative connotations due to 4chan, and it probably shouldn’t be on Roblox regardless.

Hopefully it wouldn’t get to the point of removing such a classic Roblox item though, and refunding people would be tricky since there is no proper valuation controlled by Roblox, only Rolimon’s, since RAP means nothing.

1 Like
#20

This is now inexcusable. That’s a major cause for concern.
Roblox Moderation is insane for banning people who own an asset they made.

2 Likes