As a Roblox developer, it is currently too hard to moderate in-game creations made by players interacting with e.g. building tools.
It’s trivial to make all manner of inappropriate shapes with almost any kind of in-game creation tool, many players do not interact with built-in UGC blocking and reporting tools, and at scale it is impossible to have enough manual moderators to keep up. It feels like the days are numbered for building games on Roblox without a solution for moderating creations made by players in-game.
I want to be able to give a Roblox API a Model instance or a table of instances, and have Roblox render them out from several viewpoints to pass to an AI that gives me a score of how “inappropriate” the contents are. The score is ideally bounded, like a percentage for how certain the API is that the content is not allowed, for example. Ideally, Roblox can use transparency tricks internally to defeat common workarounds such as putting inappropriate stuff inside of boxes.
I would use this inappropriateness score to determine if a player’s builds should be automatically secretly hidden for all other users until they’re no longer inappropriate; to block the user from saving builds that are inappropriate; to use as a heuristic for determining if user reports are credible and how to handle that user in terms of automated bans and ban duration; or to use to flag high-risk players to moderators who are in-game.
It is important to me that:
I can choose how to handle the player in the context of my game.
I can provide arbitrary instance trees, primarily models and baseparts.
I can re-invoke the API every couple of minutes, per player.
This AMA answer alludes to some work done in this area, so this request is more of a token gesture. I wanted to make sure our requirements for something like this were clear.
This would actually be a very nice thing to have, considering as i have already decided to manually moderate all creations in my game before theyre published for others to see.
This would allow me to skip that entirely and have players builds be published for other to see faster. But if they would make this a thing im assuming they would put some kind of limit on the amount of instances you can provide it. which could limit my ability to use it due to content in my game often having tens of millions of triangles, or roughly 400,000 instances
UGC was the biggest mistake by Roblox. At this point developers should make their own character customisation system using hand picked curated UGC content.
As someone who owns a image-based roleplay game with pixel art (roughly 500 ccu), I’d absolutely love tools like this and agree with everything in this feature request, down to the implementation details.
People have asked for the ability to make their own sprites ingame for ages; the answer has always been “no” because it’s a giant liability to us to allow that, and manual approval of all that material being made in real time is infeasible to do by humans.
If Roblox allowed us to check for moderation violations on player-made sprites, I’d finally be able to allow our players to make their own content, without having to worry so much about bad actors ruining it for everyone.