Automate RtbF Processing with Webhook and Open Cloud

Repeating what I said on the previous post about this, webhooks aren’t really an adequate solution to automate GDPR removals. I should be able to write lua code on the creator dashboard (or in some special service in studio) that Roblox can run whenever a GDPR request is received. That way this code can delete whatever is necessary for that specific game without the creator having to do anything manual after writing the code.

I shouldn’t have to do all these steps just to complete GDPR requests.

25 Likes

I’ve wanted to respect these requests… but with how much clutter they generate in my inbox and how inefficient these requests have been to fulfill, I eventually just started archiving them. Is there a persistent record of what we should be wiping if we haven’t complied?

I can’t imagine I’m the only developer who just rolls their eyes and ignores these. Is some regulatory board across the sea going to audit me because I didn’t delete Timmy’s sandcastle? I don’t know how I can reasonably prove an erased user’s save data is negligible because:

  1. It’s scoped to a live experience server only.
  2. It never gets inspected outside of a Roblox server, only in-experience when a user with their ID shows up in a live server.
  3. It cannot be cross referenced in any meaningful way.

In any case, I’m willing to comply now that it’s easier to do so, but I need to know how I can process requests I’ve ignored.

18 Likes

I know this isn’t the ideal place to ramble about this, but it should have been a much higher priority from the get-go. Doing the bare minimum to bounce this problem over to us isn’t very pro-developer.

Way I see it, there were 2-3 easy ways to make this compliance reasonable in the first place:

  1. Response Rules — Have us input one or more format strings that consume UserIds and output keys that should be deleted from our DataStores. This would be compliant with 99.9% of requests where the data is never touched outside the context of in-experience save data.

  2. A compliance callback on the DataStoreService — Developers implement a callback function on the DataStoreService which gets invoked with requests to delete specified UserIds from our records. An RCCService instance could be spun up to process these requests, or it could just be fed to a random running server.

  3. Automatic DataStore Key Deletion — Get heuristics about DataStore keys and how they correlate to the UserId of players who join/leave an experience. If there’s a significant degree of confidence the key is associated solely with individual users, add it as a format key to the response rules system mentioned above.

24 Likes

Forgot to mention before: this is not a proper use of Guilded or Discord. Discord blocked requests from Roblox servers because people were using it for logging. Telling us to use like that Discord after being blocked is a really bad idea. The proper solution is to have a web server to accept the webhooks already. You are going to need to host this either way somewhere, which the article doesn’t cover.

15 Likes

Been using this feature ever since webhooks was released with a custom bot using node, and it has completely eliminated the need for me do this process manually.
One thing that would be nice is to disable the GDPR messages for certain games or make them less annoying now that its fully automated for me.
Maybe a call back that the bot has the option to tell roblox that the data has been deleted and then stops roblox sending the GDPR message.

6 Likes

Quick suggestion to the “In house solution”, let us register what database names we use (Users/Cash/${UserID}), and then just delete the datastores with that structure. It’s instant, it’s useful, and it won’t require us to run a program / discord bot on an external server.

9 Likes

On GlobalDataStore:SetAsync, the second parameter is a list of UserIDs associated with that data

Does Roblox not automatically delete data from DataStores when they receive a gdpr request from one of those user IDs?

Realistically there is no way people will put the effort in to set up up their own servers just to handle gdpr requests

3 Likes

Great Update but i don’t understand everything and i’m stuck at the last step…
There should be a a way to write lua code somewhere or in game to know when we have to delete something…
Why don’t you use the new key metadata with the player userid…

2 Likes

Please prioritise querying DataStore keys by UserId. We have half of the fully in-house solution already existing but further DataStore tooling just doesn’t exist. Out of all the potential future listing and writing methods that were said to come around, none of that has been added.

4 Likes

Since Roblox does maintain a lot of servers for games, I would have loved to see something akin to GitHub Actions — if that is what that ‘Universe Scripts’ was going to be.

Ideally it would be something with a budget that scales depending on the recent player-count, similar to MemoryStores.

2 Likes

Wouldn’t this reflect a breach of the spirit of GDPR? You’d be passing data (UserId and Id’s of games they played) to 3rd parties running unrelated games.

2 Likes

I appreciate the attempt, but as others have stated this is a very convoluted method that requires developing external tools.

It seems like a tall ask to expect me to either manually go through the outdated and scam filled message inbox and waste my time or build and host an external server with external tools to delete this data, further compromising my experience’s privacy.

Why not let us specify the locations in datastores where user data is stored and have Roblox automate the deletion when a GDPR request is submitted?

4 Likes

Hi Creators, we really appreciate your feedback. We looked through each of them, and noted all the suggestions and asks to prioritize in our future releases. In the meantime, we have some answers to your top questions.

Data Store already supports user ID tagging, why aren’t these data deleted automatically?
An overwhelming amount of data is not tagged, and any historical data introduced prior to the tagging feature is also not tagged. Deleting tagged data would only solve a minority of Data Store use cases today.

How come Roblox is not offering a fully in-house solution?
A fully in-house solution is still in the works. This guide showcases what creators can accomplish today using currently available tools, and does not take away from our long term commitment.

Does this solution abuse third party webhook policies?
We respect and follow all third party application’s terms of service. Contrary to webhooks generated in game using HttpService, Roblox webhooks closely follow rate limits set by third party apps, and is designed to not spam or overwhelm the webhook receiver.

10 Likes

If the user has erased their Roblox account, then they usually wouldn’t be able to access that button. Regardless, it’s all about the user experience :)

2 Likes

They could still contact you to remove their data via an email or discord ticket

2 Likes

Yes, but the point is that this is still a flawed user experience (plus identity verification when the account is deleted is difficult) and something that can be improved.

I’m aware of how to make this work currently, I do not need you to suggest workarounds for me.

1 Like

This is a nice upgrade; however, still not the BEST for MY case. I understand how everything works and how we’re supposed to delete their data, okay I’m fine with that, but there’s not much (for me at least) to make the process less tiring. I could simple do normal datastores for my game, but I’m too iffy when it comes to datastores having the possibly to fail (obviously), and that’s why I use a different system for my game.

I use OrderedDataStore/DataStore for my game, so I can save multiple versions of the player’s data by creating a new key to the OrderedDataStore using os.time() - so technically, I’d have a copy of the player’s data every time they played the game - counting with the auto save that saves a copy of their data every 3-5 minutes. Now imagine, if I get a request to delete their data, what if they have over 500 saves of data? it would take a good while to erase all the saved keys on the OrderedDataStore AND removing all these saves in the main DataStore. :pensive:

I had the idea of getting all the game’s datastores and filtering by the player’s id, but that would ALSO take ages since I use a datastore for each player…

local DATASTORE_KEY = "DATA:%d"
local DataStoreKey = DATASTORE_KEY:format(Player.UserId)
local DataStore = DataStoreService:GetDataStore(DataStoreKey)
local OrderedDataStore = DataStoreService:GetOrderedDataStore(DataStoreKey)

This is been the safest method of data saving that I’ve been using for ALL my games, given if any errors occurred I could simply force them to return to an old version of their data (which never happened).

This is how I save the data:

local Key = os.time()
DataStore:SetAsync(Key, Data)
OrderedDataStore:SetAsync(Key, Key)

And this is how I load the data:

local Default = GetDefaultData()
local Data = {}
local MostRecentKeyPage = Retry(function()
	return OrderedDataStore:GetSortedAsync(false, 1):GetCurrentPage()[1]
end).Data
	
if MostRecentKeyPage then
	local Request = Retry(function()
		return DataStore:GetAsync(MostRecentKeyPage.value)
	end)
	if Request.Success then
		Data = Request.Data
	else
		local Message = "Error while loading %s's data! Error: %s"
		Message = Message:format(Player.Name, tostring(Request.Error))
		Debug(true, Message)
		if Player and Player.Parent then
			Player:Kick("Couldn't load your data! Please rejoin.")
		end
		return
	end
else
	Data = Default
end
3 Likes

I appreciate the addition to automate Right of Erasure requests, but I am having issues. As someone who is completely new to Guilded/Discord bots, API requests and webhooks, I’ve been stuck on the tutorial for a long time and I don’t know what it is asking me to do on step “Adding Scripts”.

1 Like

Something sorely missing from the documentation is specifying exactly who receives this webhook. For group-owned experiences, is it the owner of the group? This feature feels fitting to at least have a role permission toggle-able for specific users who have the webhook set up to receive.

If it by is group owner this poses a bit of a security concern for those of us who develop games for a variety of clients who have control over the group owner accounts. We now need to communicate a secret key with them for the webhook, and unless we’re wanting to perform X amount of HMAC checks, we likely would communicate the same key to each client… this is less than ideal. Having this configurable on a singular user and ensuring that they’re the recipient of the webhook would be the best scenario.

If a staff member could clarify or modify the documentation to clarity who receives the webhook this would be good to know as currently it is missing. The documentation I have found is a staff member on a forum post who mentions “owners” receive it — but what does that mean? Owners being plural raises more questions as only 1 user can own a group or game.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.