WebhookProxy | Discord webhooks go brrrrrrr

GOODEVER · October 3, 2021, 4:49pm

Thank you for this, can you also show us how to set up a proxy server with a VPS ourselves

EDIT: grammar

lewisakura · October 4, 2021, 3:47pm

To host it yourself:

Basic Setup

This setup just exposes the proxy publicly with no reverse proxy. Recommended to start with only, but you should probably use the correct setup as soon as possible.

Install Node.js on your server. This can be done through a package manager or through nvm. The minimum requirement is v16.
Install git. This is usually a default tool nowadays, but just grab it off of your package manager if you don’t have it.
Install Redis or its Windows equivalent Memurai (please note Memurai is paid software and you should probably just go put Redis in a Docker container instead on Windows, however for testing purposes Memurai will work fine). You need at least v6.2 due to the commands used, though v7 and above is preferable.
Install pm2 and yarn (npm i -g pm2 yarn)
Run git clone https://github.com/LewisTehMinerz/webhook-proxy to clone the proxy.
Enter the resultant webhook-proxy folder.
Copy the .example.json files to the same name, just without .example (e.g., config.example.json → config.json).
Modify the files as you need, primarily config.json.
Run yarn && yarn build. This will install the necessary dependencies and build the project.
Run pm2 start dist/index.js --name=webhook-proxy. This will start the app under the name webhook-proxy in pm2.
- If you wish to run this on startup, run pm2 startup, follow the instructions there, and then run pm2 save.
You should be good to go! Future updates just require a simple yarn update.

The Correct Setup

This setup involves using a reverse proxy instead of exposing the server directly and using a cluster instead of a single process. This is recommended (and the correct way), but a bit more complicated. ~~This is how I actually run the proxy.~~ This is how I used to run the proxy. I’m now using Cloudflare Tunnel, however this is still the recommended way of doing things that keeps it simple.

Install nginx. This will be our front-facing web server.
Update your configuration and set trustProxy to true.
Create a new site in nginx with the following configuration:

server {
    listen 80;

    server_name <domain name>;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://127.0.0.1:<port>;
        proxy_redirect off;
    }
}

(it is recommended you enable SSL and HTTP2 but this is out of scope for this setup)

Reload nginx and pm2 restart webhook-proxy. You should be good to go.
(Optional) Depending on your load requirements, you may want to cluster WebhookProxy to deal with a large amount of Roblox servers. If you have >50 servers sending webhook requests frequently, you may need to scale. To do so, run pm2 delete webhook-proxy, and then run pm2 start dist/index.js --name=webhook-proxy -i 1. This will run it in a clustered mode instead of the standard fork mode.
- From here, you can now scale the proxy up and down as you need to by doing pm2 scale webhook-proxy <worker count>. This is good for games that are growing that need to send a lot of webhook requests as you can just put on more workers as needed.
- Please note that the benefits of clustering come from having multiple CPU cores. If you do not have more than one core on your server, this will not benefit you and will most likely reduce performance from the overhead of clustering and the workers fighting each other for resources.

Enabling Queues

A new feature of the proxy is the queue system. This requires some extra (but simple) setup.

Install RabbitMQ.
Edit your configuration to enable queues, and point to your RabbitMQ installation (it should just be the default value that I’ve provided, but in case you’ve changed anything you can set it here).
Restart the proxy (pm2 restart webhook-proxy).
Start the queue processor with pm2 start dist/queueProcessor.js --name=webhook-proxy-processor.
- Run pm2 save as well if necessary.
You should be good to go! Try adding /queue onto your webhook requests!

With the newer updates of the proxy, you can now just run yarn update and, as long as you have the setup as described in this guide, it will automatically update the proxy for you. If a yarn update fails, try running it again. It could be that I updated the script.

ControlCoreAngel · October 4, 2021, 4:41pm

Confirming Discord banned Roblox, but probably forgot to ban it from their deprecated discordapp.com, and that’s why it’s still working there.

When discordapp falls through, this proxy will be a good fallback. Thanks again.

Baysights · October 5, 2021, 3:58am

This is really helpful and all, but I think you could make this a little more efficient by using ProxyService.

This could be used as a means to provide a global proxy, instead of selectively using it for webhooks. This can help if you want to access APIs or such outside of Roblox’s useragent. I personally set it up and it was the same amount of work as this. They’re both doing essentially the same thing but yea. Thought I’d contribute that, however, good post. This is very helpful.

lewisakura · October 5, 2021, 11:16am

The point is to avoid running a ProxyService instance. I don’t want people using my server to make any web request they want. I wanted to keep the focus restricted to webhooks so that people can’t use my server to perform malicious web requests.

It would also mean I have to give out access keys, whilst this is publicly accessible.

Matrice · October 8, 2021, 10:25am

Your proxy no longer works now. It’s got some server-sided 502 gateway issue going on. discordapp.com is now a 403 too, so it’s getting challenging.

5022

lewisakura · October 8, 2021, 12:34pm

Oh, I’ll look into it when I get home.

CoolJohnnyboy · October 8, 2021, 9:17pm

Tried using your proxy, and this is what resulted:
https://gyazo.com/83364b86967f46a1cc0238f2c2579497

Probably because this is what is being fed into the webhook, I’d assume:

local mkt = game:GetService("MarketplaceService")
	local assetInfo = mkt:GetProductInfo(game.PlaceId,0)
	local Data = 
		{
			["username"] = "Frigus Cop";
			["content"] = "",
			["embeds"] = {{
				["title"] = "**Suspected Exploiter Kicked**",
				["description"] = p.Name.."has been banned from "..assetInfo.Name,
				["type"] = "rich",
				["color"] = tonumber(0xFF8C00),
				["fields"] = {
					{
						["name"] = "plr Name",
						["value"] = p.Name,
						["inline"] = true
					},
					{
						["name"] = "plr UserId",
						["value"] = p.UserId,
						["inline"] = true
					},
					{
						["name"] = "Exploit Probability",
						["value"] = severity,
						["inline"] = true
					},
					{
						["name"] = "Account Age",
						["value"] = p.AccountAge.." Days",
						["inline"] = true
					},
					{
						["name"] = "**Link to Profile**",
						["value"] = "https://www.roblox.com/users/"..p.UserId.."/profile",
						["inline"] = true
					},
					{
						["name"] = "Banned?",
						["value"] = "yes",
						["inline"] = true
					},

				},

			}}
		}
	Data = HTTPService:JSONEncode(Data)
	HTTPService:PostAsync(link, Data)

If I have to simplify that into a short message instead (which is probably it), I think I will just use Guilded to log this stuff. All I’m doing with the webhooks is logging potential exploiters, admin command usage, and if an exploiter who was autobanned tried rejoining the game afterwards.

lewisakura · October 9, 2021, 12:00am

I’m not sure what “Trust check failed” is. However, the proxy is currently down for unscheduled maintenance. I’ll bring it back up as soon as I can.

lewisakura · October 10, 2021, 5:06pm

The proxy is back up. Sorry for the downtime.

CoolJohnnyboy · October 10, 2021, 6:24pm

I continue to have the same message occur.

lewisakura · October 10, 2021, 7:57pm

Please DM me with the message you receive so we can resolve this, alongside a code sample.

lewisakura · October 10, 2021, 8:28pm

@CoolJohnnyboy If you can’t tell by the reply I gave you, I’m exhausted. Phew!

I did some research into your issue, you seem to be missing https:// off of the link. HTTPService doesn’t really like this much. Can you try adding it and seeing what happens?

ControlCoreAngel · October 11, 2021, 8:37pm

discordapp has fallen. ~~(Now I, Starscream, lead the Decepticons!)~~

Confirming for any future posters that this proxy is working. If you’re having issues, doublecheck your webhook URL. Remember to include https:// and that all that’s replaced with the proxy’s URL is discord.com or discordapp.com.

DeonysiaInvictus · October 14, 2021, 4:21pm

Thank you this has been a life saver for group

VisualPlugin · October 14, 2021, 6:28pm

~~I’ll be hosting your proxy on rblx-discord-webhook.us-3.evennode.com . That will save time on behalf of game developers who would have to host it themselves.~~

https://webhook.lewistehminerz.dev/ exists.

CoolJohnnyboy · October 15, 2021, 7:15pm

Works great now. Thanks for the effort dude! I really appreciate it

lewisakura · October 16, 2021, 12:48pm

Hey all,

I’m experimenting with a new feature. If you put /queue onto the end of your webhook URLs, your messages will be put into a RabbitMQ queue instead of being sent directly to Discord.

These queued messages are guaranteed to be sent, but are not guaranteed to be in order (though I try, promise!). If order matters, this isn’t for you.

Speaking of ratelimits, there is a much higher ratelimit for queueing messages rather than sending them directly: 10/1s. Compared to Discord’s 5/2s, this is very generous, and it doesn’t even stop your requests if you hit it, it just slows them down. Many benefits with few drawbacks!

Oh, right, if you use ?wait=true, that doesn’t work with this. You should treat this as fire-and-forget. If you want message data back, you can’t use the queue system.

Of course, if this system doesn’t end up panning out I’ll provide a one week notice that I’ll be removing it. However, I have high hopes, and hopefully you guys will get some use out of it, especially if you get ratelimited a lot.

Have fun! As usual, report any issues here or on GitHub.

tacovaIk · October 16, 2021, 5:03pm

Thank you for this, I use webhooks for game analytics and I was worried that I wouldn’t be able to do that anymore.

lewisakura · October 19, 2021, 3:37pm

I’m back~

I got bored, have this module:
https://gist.github.com/LewisTehMinerz/5308748d2d65a195ff580d4b9a54845f

It should make it a bit simpler to send messages.

local proxy = Proxy.new(id, token)

proxy:Send({ content = "abc" }, false)
proxy:Queue({ content = "abc" })

… will send abc to the webhook, both directly and via the queue system.

As usual, report any issues here ~~or on GitHub~~.