Use a Neural Network to Quickly Moderate Off Topic Posts

Currently, off topic posts can remain on the developer forum for a long time before being unlisted. This can be a big problem, especially if there are inappropriate content on the post.

One solution to this is to train a neural network to help moderate the forums. This should not replace the current moderation team, but instead help act as a first line of defense against off topic posts.

A system can work like this:

  1. A post is submitted onto the Developer Forum. With the author’s consent, the post is scanned by the neural network.
  2. If the post is determined off topic, the topic is remembered, but is allowed onto the devforum. This is to prevent false positives.
  3. If a reader flags the post, the bot mod checks the database to see if it has been marked off topic by the scanner. If it has, the post is immediately locked and hidden and will be put for further human mod review.

Once the neural network is accurate enough, Roblox may just skip step 3 and send the post to human review if it is marked off topic by the AI.

What are your thoughts on this? Do you think this is a good idea? What improvements to the system should be made to my current propsal?

Thanks,
@Erie20

3 Likes

I don’t think there’s a way to do this. Neural networks can’t process the meaning of human words. If there was an NSFW post, for instance, the network wouldn’t be able to process what that string of words meant. If a neural network could determine the meaning of words, however, this would be great.

4 Likes

The neural network can find patterns in the words, it might be able to determine if a text is off topic. For example, if it sees a post with just a bunch of Ls and Os, it can learn to know that it is off topic. It may not understand that those letters make LOLOLOLOL and that it is not related to development, but it can find patterns in text.

I see where you’re going, but it would be VERY basic. Pretty much just LOLOLO replies, which probably make up less than 5% of all off-topic posts. Knowing that, it would be easier to just not bother with a neural network.

Uhh, I don’t think that’s as easy as it sounds. We’re talking about communication here. This would require a supercomputer Roblox nor Discourse has.

Bots like Alexa and Siri are highly advanced AI that require huge supercomputers to work, and even they probably can’t moderate the forums.

3 Likes

This would definitely be an ambitious project. It would be pretty surprising to see this in action if devrel get around to this. However, it is unlikely that they will due to how ambitious it is, it being difficult if not impossible to get context in order for the network to flag a post as off topic. It’s for the same reason machine translations usually suck because you can’t just feed them context.

Would definitely be interesting to see though

3 Likes

Neural networks may also be able to moderate more complex things. Neural networks find patterns in posts, so they could possibly find a pattern in off topic posts. However, since I’ve never tried training a neural network, I don’t know for sure if it would work.

I don’t think a supercomputer is required. It doesn’t need to communicate like a human, it just needs to find patterns, but like I said above I am not sure if that will exactly happen.

Yeah, I think that is the main problem with my idea. Because there is currently no proof of concept Roblox probably would not implement a Neural network to moderate the forums.

Thank you for your replies!

1 Like

Feature request should focus on problems, not on proposed solutions, ref: How to post a Feature Request

The problem you want to focus on here is that you believe there are too many off-topic posts. You can mention a “neural network for off-topic post detection” as a side-note in your post.

3 Likes