ReportService or Report APIs in Open Cloud

Feature Requests Engine Features
, ,
1

As a Roblox developer, it is currently too hard to moderate UGC assets created by third parties. In my experience, users are able to utilize the full breadth of content created on Roblox including: decals, audio, and avatar items. This means we have a uniquely difficult challenge of moderating anything that is public on Roblox.

As it stands, there is no way to report content on behalf of a user on your experience. This means if users find offending content, such as an inappropriate decal or a sound, they have no ability to report this content if a player in the server is not associated with it. Whilst some workarounds exist, like displaying the assetId of a given asset so users can copy & paste it into a web browser, this is not always viable due to the quantity of assets we display. Ideally, users could simply click a “Report” button next to any asset in our UI, which would submit the report with proper context.

We have internally created a system for creating reports on behalf of our moderators using a specialized proxy that directly sends data to https://apis.roblox.com/abuse-reporting/v2/abuse-report, but this would be unscalable for our entire player base due to rate limits and anti-botting measures on this endpoint.

I’m requesting that a ReportService or some kind of Open Cloud access is made available to abuse-reporting so we can reasonably serve hundreds, or even thousands of reports from our players. Bulk operations that can accept multiple assets at once would be especially valuable.

If Roblox is able to address this issue, it would improve my development experience because we can empower our thousands of users to help Roblox catch inappropriate content faster across the platform, and reduce pressure on our moderators to manually report a massive amount of offending content that bypasses Roblox’s filters.

TLDR:
Our experience allows users to use UGC from the catalog, meaning they can both use and find malicious content that bypasses Roblox moderation. There is no method for developers to report content on the behalf of users, meaning inappropriate content often goes unreported. We need a ReportService or Open Cloud support for allowing asset reporting within experiences to resolve this issue.

21 Likes
2

In my personal opinion I believe it would be a giant mistake to open a public endpoint for reporting “hundreds, or even thousands” reports, especially with current moderation being flooded as-is.
Bad actors could easily spam false reports on everything.

This would be the best solution, having an PromtReport(assetId) in the Player datamodel or something, where a report modal would pop-up for the player to fill in, and report on their own.

4 Likes
3

Definitely can agree with that!

Having a method to report an assetId as a client side prompt would also be a great way to solve this problem.

5 Likes
4

This is a great idea! This can be used in a chat filter to automatically report any bad content.

5

I agree with this proposal for a ReportService, but it misses a much more sought-after use-case among developers, which is giving players the ability to report in-game user-made content like text, drawings or buildings, without forcing developers to hire third-party moderators whose powers are limited to just the scope of the game.

Currently, developers who make use of global user-generated elements are left in a dilemma about whether they should prioritize the safety of the platform as a whole, or the safety of their own game.

If a developer chooses to moderate their own game (either by themselves or by hiring a team of moderators), they will be able to remove harmful content and punish bad actors in their own game, but those bad actors will continue to be able to use other parts of the platform with impunity, since Roblox’s official moderation team was never made aware of their bad conduct. This is bad for the whole platform because a person who offends in one game is likely to offend in many more.

If a developer chooses to go the other route and leave moderation up to Roblox and the official report system, then the content of those bad actors will continue to stay in the game even if Roblox chooses to punish them, since Roblox moderators don’t have magical moderation tools that adapt to every game’s unique data storage design. This is bad for the game since it requires the developer to actively find and remove harmful content, which becomes nearly impossible once the game attains a large enough playerbase.

To solve this issue, I propose that the hypothetical ReportService should include a method for reporting in-game user-made content, that takes a parameter for the reporting player’s ID, a parameter for the offending player’s ID, a parameter for the offending content (text or model or maybe even a screenshot), and a parameter that uniquely identifies the content to the server, like so:

ReportService:PromptInGameUserContentReport(reporter: Player, offenderId: number, content: any, contentIdentifier: string)

The reporting player would then be shown a confirmation prompt, upon which the report would be received by Roblox’s moderation team.

If and when Roblox moderates said content, a callback function would be fired on a random gameserver with the given contentIdentifier, giving the game an opportunity to remove the offending content from itself, in similar fashion to how MarketplaceService.ProcessReceipt works:

ReportService.ProcessUserContentRemoval = function(contentIdentifier: string)
    ContentStore:RemoveAsync(contentIdentifier)
    return true
end

Obviously this introduces the possibility of abuse vectors that have been abused in the past, and for that reason the method would likely have to be restricted to ID verified or maybe even only top creators; but it would take a huge load off those developers’ backs while making the platform safer for everybody.

1 Like