Guide to Web Development

New Chapter: Discourse Forum

Want to setup your own Discourse forum like Devforum?

New Chapters: Monolith and Microservices (Netflix)

Every been interested in how businesses like Netflix build their application system and handle it’s massive services?

Introduction to Web Development

A tutorial for Roblox users to enter web development!

This tutorial is expected to be expanded up with topics deemed important, if you believe something should be added please post a comment.

The chapter on REST API for scripting with Nodejs is incomplete but enough to get started.

Guides

How to Program a Website (Node.js Beginners Guide)
Guide to Scripting Bots | Javascript Tutorial | FunCaptcha and New Host Info

Contact

Discord: TechSpectrum#2620
Email: CourtSmith1101@gmail.com

Updates

2019-06-30T05:00:00Z

  • Discourse Forum chapter.
  • Monolith chapter.
  • Microservices chapter.
  • Netflix and Uber (Monolith to Microservices)

2019-03-16T05:00:00Z

  • Bootable USB chapter for Kali Linux and Fedora.

2019-03-04T06:00:00Z

  • Minecraft Server chapter.

2019-02-13T06:00:00Z

  • Career and Education chapter.
  • Double Authentication chapter.

2019-02-12T06:00:00Z

  • Added Docker chapter.
  • Added Kubernetes chapter.

2019-02-03T06:00:00Z

  • Added Cloudflare DNS setup instructions.

2019-02-02T06:00:00Z

  • Added instruction on how to setup a MySQL database on the server for a VPS.
  • Added pm2 auto-restart in the Digital Ocean tutorial.

2018-09-01T05:00:00Z

  • Updated Digital Ocean automated deployment section for batch script and sudo permissions.
  • Added the SSL configuration file path: sudo nano /etc/nginx/snippets/ssl-params.conf

2018-08-31T05:00:00Z

  • Update roblox-js library to the latest version: 4.0.2
  • Express Rate Limit no longer supports delay functionality in version 3, you can still get it in version 2.11.0 however this tutorial will not use it.
  • Planning OAuth2 tutorial for Discord and Google.

Table of Contents

  1. Introduction
    • About
    • Who is This For?
    • Disclaimer
    • Downloads
    • Helpful Links
  2. Tools
    • Cmder
    • Nodejs
    • Git
  3. REST API
    • What is an REST API?
    • Creating a Database (500 MB Free)
    • Creating an Express App
    • Scripting the Database
  4. SQL and NoSQL
    • About
    • SQL
    • NonSQL
  5. Web Frameworks and Libraries
    • About
    • Bootstrap
    • Angular
    • React
  6. Security and Attacks
    • DoS and DDoS
    • Cross Site Request Forgery
    • Spoofing
    • Phishing
    • Man in the Middle
    • Privilege Escalation
    • Click Jacking
    • Cross Site Scripting
    • Brute Force
    • Social Engineering
  7. SSH
    • What is an SSH Key and What Does it Do?
    • How to Make a SSH Key
  8. Digital Ocean
    • What’s Digital Ocean and Why use it?
    • Create a Droplet
    • Connect to a Droplet
    • Create a Admin User
    • Disable Root Login
    • Firewall Setup
    • Automated Deployment
    • DNS Setup
    • SSL Certificate (Free)
    • Nginx Setup
  9. Heroku
    • What is Heroku?
    • Setup
    • App Deployment
  10. Roblox Scripting
    • HttpService
    • Make a Request
  11. Discord
    • Create a Bot
    • Connect a Bot
    • Writing Commands
  12. MySQL
    • Setup
    • Tools
  13. Cloudflare
    • DNS Records
    • SSL Allowance
  14. Docker
  15. Kubernetes
  16. Career & Education
    • Job Types
    • Job Searching
    • Getting an Education
  17. Double Authentication
    • SMS
    • TOTP
    • Biometric
    • Geographical
    • Timezone
    • Security Device
  18. Minecraft Server
    • About
    • Requirements
    • Setup Spigot
    • Server Setup
    • DNS Setup
    • Startup and Connect
  19. OS Bootable USB
    • What is a bootable USB?
    • Fedora
    • Kali Linux
    • Boot from USB
  20. Discourse Forum
    • SMTP Server (Mailgun)
    • Setup SSL
    • Setup Nginx
    • Setup Discourse
  21. Monolith Applications
    • Pros
    • Cons
  22. Monolith Applications
    • Pros
    • Cons
  23. Netflix and Uber (Monolith to Microservices)

Other Web Topics:

  • Proxies: NGINX and Apache
  • OAuth2 (Google, Discord, Facebook)
  • Web Scrapping
  • Web Sockets
  • Framworks: Vue, Django, Metor, Go
  • Languages: Python, PHP
  • Load Balancing

Introduction

Table of Contents

Read Chapter

About

Table of Contents

This tutorial will cover several topic for creating a full project for Roblox in Nodejs using MongoDB as it’s database and Express as it’s web router for HTTP request. You will learn how to build an application that can use the Roblox API, use the Discord API, and most important build your own web API that will run on your host of choice.

We’ll talk about security such the attacks currently being used today by people who want to do you harm. It is very important you develop with security in mind and staying up to date with today’s methods of defending against them.

I am TechSpectrum, I develop clan databases and currently host for over 40 clans of varying size from 100 to 300,000 users. I been a Roblox since 2009 learning lua and eventually came across technology by RAT currently owned by @Polymorphic that utilizes a web based database. It was an inspiration that eventually drove me to learn how such a feat was possible on Roblox.

Today I wish share that knowledge with those who want to help themselves and learn how to down towards the path of development on a platform they love. I hope those of you who follow this tutorial enjoy, and come to understand a little bit about this field and have fun with it as I have and continue to learn with me.

With that said, lets get started!

Who is This For

Table of Contents

This tutorial is meant to teach people who want become web developers where to start and how to make a full fledged project from start to finish using Nodejs. It’s not recommend you start learning from this tutorial without basic knowledge of Javascript or Lua. This guide will cover how to cover the tools and the setup but not the language.

It’s difficult to find all the resources you need so this is a good quickstart to learning and understanding how most web projects work and everything involved with running websites.

Disclaimer

Table of Contents

I do not condone any malicious development or actions that may result from this tutorial. Please be ethical and understand that what you do with this knowledge I am not responsible for.

Downloads

Table of Contents

  1. Nodejs
    • The Javascript Runtime Engine
    • Built using C++
    • Uses the Chrome Browsers V8 Engine
    • Includes npm (Node Package Manager)
  2. Git
    • Version Control Software
    • Uses webhooks to receive updates.
    • Stores our application with version history.
    • Repo Sites: GitHub, GitLab, GitBucket
    • Includes: GitBash Console (Linux Based)
  3. Cmder
    • A robust and customizable command prompt.
    • Will be used to run multiple consoles under one window.
  4. Any script editor.
  5. MongoDB Compass
    • Editor for MongoDB Databases.
    • Download Community Edition Stable
  6. Postman
    • Used to test our API

Table of Contents

  1. Discord.js Documentation
    • The library for using the Discord API.
  2. Roblox.js Documentation
    • The library for using the Roblox API
    • Note: Some API is out dated (i.e. Wall Post)
  3. Discord Developers: My Apps
    • Your discord page for managing your applications.
  4. Discord Bot Permissions Calculator
    • Used to assign your bot privileges for invitations.

Last Updated: August 26, 2018


Tools

Table of Contents

Read Chapter

Cmder

How to Use

  1. cd Projects
    • Access a folder in your current directory.
    • Projects may be a folder in your Desktop directory.
    • If the folder has spaces use quotes: cd "My Projects"
  2. cd ..
    • Go back to the parent directory.
    • Program Files would be the parent directory of Adobe
  3. ls
    • List current directory files.
  4. C:
    • If you want to changes drives you can type change it like this.
    • If I used a external drive I may say E:

Here is a Windows Command List and a Linux Command List

We’ll be using a few Linux commands later for Digital Ocean which uses a Linux Machine such as Ubuntu.


Installation

  1. Make a new folder, name it Cmder.
  2. Extract or Click and Drag your cmder.zip file contents into your folder.
  3. Place your folder in your “Programs Files” directory in your C: Drive.
  4. Click and start Cmder using the launcher.
  5. On the bottom bar press and click the green plus button (New Console).
  6. Click on “Run as Administrator” and Start .
  7. Make sure you are currently in the directory of the Cmder Folder. (C:\Program Files\Cmder)
  8. Paste and run: .\cmder.exe /REGISTER ALL

Now you can right click anywhere and see Cmder Here


Tips

  1. You can right click a tab to rename the tab, restart the prompt, or close it.
  2. On the far right is blue button where you can access settings and go to background to add your own image or color scheme.
  3. Use 3 tabs.
    • One for the Application
    • One for installations, updates, etc.
    • One for your web host console (i.e. Digital Ocean).
  4. In Windows 10 you can click the top box where your folder name is and get the exact path of the folder which you can copy and paste into the command prompt to instantly access it.

Nodejs

Nodemon - Automatic Restarts

Before we start using Nodejs, first we want to install nodemon.

Nodemon is a package that will handle automatically restarting your application when it detects changes in your scripts when you save a file. This is convenient for us so we don’t have to keep stop and restarting the application ourselves.

To install nodemon enter npm install -g nodemon in the command prompt.

Without nodemon installed into our computers PATH variables we would only be able to run our application by saying node app.js but now we can say nodemon app.js to run it with nodemon monitoring.


Create a Project

You should be in your folder directory you want to save your project in before running any commands.

We’re going to want use a few packages, to make a package.json file run the following command:

npm init

You can fill in the details and type yes at the end to complete the setup.

Git

NOTE: Preferably come back to this segment after the “REST API” tutorial.

This is a popular version control software used by most of your large platforms and developers. There’s a lot about Git to learn but for most people you only need to know a few basics to using it that you’ll use all the time. It’s important to understand that this is not a app you can open up but a program that embeds itself into your PATH variables just like Nodejs.

In order to start using it, first we have to go to our project files directory and run the following command.

git init

It’ll tell you Initialized empty Git repository unless you already have a git repo in the folder. What this does is make a folder that’s set to hidden which contains all the information and functionallity of git for repository work.

Now we need to “commit” these files to be a saved version in time, however we first need to stage our files so we’ll run:

git add .

This command will stage our files and what that means is now we’ll be able to store these into a commit because they are ready. If we want to only commit one file we could say: git add server.js

If you want to see the status of your repository files: git status

Once your files are finished being staged we can now commit them and add a message to our commit so we know what was changed. It’ll be our authors note that will be something we can go back to later in time.

git commit -m "Starting point of the Project"

After the commit is finished, we now have something we can use and especially rollback to in a emergency.

NOTE: This next part is only for when you have a repository you want to use such as Heroku or Digital Ocean.

As of right now in this tutorial we don’t have a Repository we can send to; but we’ll go ahead and talk about push, pull, and remotes.

A remote is link that is public, for example: ```https://www.mywebsite.com/techspectrum/repo` is a url where a repo might be. We can create a remote like this:

git remote add MyRepo https://www.mywebsite.com/techspectrum/repo

We can then “push” to that repo as such:

git push MyRepo master

MyRepo is the name of my remote, and it’s pushing (uploading) our files to the websites repo through a webhook (we’ll make one later in Digital Ocean). Now the way git works is each commit is that not every commit is the entire project but the changes made and saved at that time. If we had version 7 of our project, and the server had 4, then when we push it’s going to send over commits 5, 6, and 7.

If were on a new computer with a empty git repo folder we’ll have to use:

git fetch MyRepo master

This will download all our commits, remotes, and so on. If a friend were working on a project with us, and we already had version 4 of the project, we’ll use:

git pull MyRepo master

Now lets pretend we had this situation:

Developer A: 10 → Pushes to Server
Server: 6 → 10 Updated by Developer A.
Developer B: 6

If Developer B pulls from the Server which is now version 10, he would get commits 7, 8, 9, and 10.

Now what is this “master” ?

Git has what we call branches, and that’s where other developer can branch off from others developers work to expand on a feature without disrupting the other projects. There may be a function “production” branch that gets update when the director performs a git merge on different branches to bring them together. Unless you are working with a team you may not often use this, so for this tutorial we’ll assume you’re only going to use the main “master” branch.

There are git GUI’s out there such as Git Kraken which can make the use of Git much easier on you. You don’t want to break your project so I would say you should get comfortable with it as much as possible through practice and learning the common commands and what exactly it is they do.

Here is what I suggest for Git GUI:
GitKraken
SourceTree

REST API

Table of Contents

Read Chapter

What is a REST API?

Representational State Transfer (REST)
Application program interface (API)

Communication and Endpoints

It’s a interface to application communication network often from client to server in web. In web programming our interface would be the endpoints. Endpoints are paths in our server that can be used to interact with the server to obtain information or perform an action.

Here is an example:
https://www.mywebsite.com/api/users/search
https://www.mywebsite.com/profiles/techspectrum

This would be a web endpoint that could be used to obtain search information from a database in our first example. It may send us back some JSON which we can convert to usable code for what language we are using (javascript).

In our second example we see I’m going to a profile, this may be a web page that sends us a display of my avatar and information so instead of JSON we’re being sent html, css, and other script files the page calls.


Methods

Methods are like action types we want to make on a endpoint when we send it a request. They act as a way to identify what we want to do.

Here’s a few:

  1. GET - Used to retrieve data from the server.
  2. POST - used to send data to the server.
  3. PUT - Used to update data.
  4. DELETE - Used to delete data.

In Roblox you used to only be able to use GET and POST but as of recently we can now modify headers to get the type of action we want. A endpoint can support multiple methods if you want to use the same one for different purposes.

Example:
https://mywebsite.com/api/profile

On this endpoint we may want to use GET to retrieve the user info. However we could also use DELETE if we want to destroy the users profile.

So where are these methods, how do we apply them?

Well they are in your http request header, that’s not something you can see in a search bar. if you want to see some of the request being made right now to your browser you can go to browsers web development settings. Then view Network and you’ll see a few 200’s and GET request if you reload the page.

Right now as I am writing this the website is actively sending request to it’s REST API to save my changes. Here is what it looks like:

As you can see it’s POST, which means my browser has script that are sending request to the server. When we browse on our computers to website we’re performing GET request. Our browser then uses a run time engine to display the website for us and interprets the scripts.


Status Codes

So what is this 200 and what does it mean exactly?

Status Codes are categorized in a series of hundreds, you can find the wiki here!

  1. Information (1XX)
    • You won’t see these often.
    • It’s used in the browser to communicate the state of the request initialized such as “please wait.”
  2. Success (2XX)
    • Tells the browser we made a successful request that sent back just fine.
    • The standard “OK” code when a request went through is 200.
    • Others: 201 (Created), 202 (Accepted)
  3. Redirection (3XX)
    • Commonly used in cases where you will be redirected such as by an OAuth sign in or moved to a new proxy.
  4. Client Error (4XX)
    • This means you did something wrong on your end.
    • The most commonly known 400 series code is 404 which means Not Found.
  5. Server Error (5XX)
    • This means the server did something wrong.
    • You might often see websites go down and Cloudflare will give you a error using their own codes (520, 521, etc.)

These are more or less guidelines, you do not actually have to follow or use this although it is preferred because most browsers and other clients will us the status code to perform the appropriate action.


Creating a Database (500 MB Free)

We’re going to be using a website called mLab which is a free database as a service provider for MongoDB databases. You will be able to make unlimited databases each with 500 MB free!

Sign up and login, and you be created with a web page that looks a bit empty, so we’re going to find and click on this button at the top of our page.

image

Choose the first option for Sandbox from any provider you wish.




Submit the order and locate your database back on your home page.

Click it and you’ll come across a similar layout.

  1. Collections - The database storage group (ex: users, logs, etc)
  2. Users - These are credentials used to access and connect to the database, we’re going to have to be make one in order to use our database.
  3. Stats - Explains variables in your database you’ll be able to see when connected.
  4. Backups - mLab’s backup system.
  5. Tools - Importing, Export, etc. It won’t be used unless you upgrade from Sandbox.
  6. MongoDB URI - We’ll be using this to connect to our database using Mongoose.

Lets head over to Users and add a user to our database.
image

Make sure you don’t make it read only or we can’t edit the database.
image


Creating an Express App

Express is a library for Nodejs which will handle our routing and middleware applications we want to use. We’ll be talking about some security measures you can add to take precautions against attackers.

In your command prompt (Cmder) make sure you are in your project folders directory and you have your package.json file.

To install express: npm install express --save

You should see a folder now called node_modules which contains all your packages used for express. Your package file should also now contain dependencies.

So now we’re going to make a new javascript file and call it server.js and another called config.js

In our config script we’re going to write the following:

module.exports = {
	database: process.env.DATABASE || "mongodb://RobloxDB:password1@ds133642.mlab.com:33642/roblox-database",
	port: process.env.PORT || 5000,
	secret: process.env.SECRET || "XXjwjw390Kp2k3203kwlwOOnsd33"
}

Make sure you use your databases URI from mLab and replace username and password with your user information you created. We also added a secret which will be what we include to authorize the access of our important API we don’t want others using without permission.

What’s process.env?
We have environment variables we can create and put on our computer.

Back on our server script, we’re going to type he following.

const express = require('express');
const config = require ('./config')

// Creates the Express App
const app = express();

// Route
app.get('/', function(req, res){
	console.log("GET Request.")
})

// Listen to Request
app.listen(config.port, function(){
	console.log("Now listening for request.")
});

You can now save these files, we’re now going to run our application using nodemon in the console.

image

If we were to go to http://localhost:5000/ right now we would have an infinite wait because the server is still waiting for a response. Though you will see a message pop up in your console.

image

What we want to do right now is just send a short message back to the client and to do that we’re going to be using the response variable. res

app.get('/', function(req, res){
	console.log("GET Request.")
	res.send("Hello World!")
})

image

Now we want to get into the meat of things and create our own API web routes. However we want to organize a little.

First off, lets delete our current app.get() function as we don’t need it anymore.

Instead we will replace it with:

// API Web Routes
app.use('/api/users', require('./api/users'));

Now make a folder and name it api and inside that folder we’re going to make another script called users.js

We’ll use this:

const express = require('express');
const router = express.Router();

router.get('/', function(req, res){

});

router.get('/:id', function(req, res){

});

router.post('/:id/updatecurrency/:value', function(req, res){

})

module.exports = router

Explanation: The router handles direction paths from other modules as if middleware. The :id we added is so we can use it as a variable for finding a player later on.

http://localhost:5000/api/users/2939382

Right now your project should look like this:
image

Now we’re going to be making some data to start working with.


Scripting the Database

Before we start first we need some middleware.

Mongoose: Object Data Modeling (ODM) library mongoose for using the database.
Body Parser: To convert request data from JSON into a JS object.
Express Sanitize: Converts data into a string so we don’t end up using malicious data.
Express Rate Limit: Will help prevent our API from being abused.
Helmet: Adds security headers for us that by default prevent a few basic attacks.

npm install body-parser --save` npm install mongoose --save ```npm install express-sanitizer --save
npm install express-rate-limit --save` npm install helmet --save`

I highly reccomend you read into each one and what they can do.

Here’s some documentation for express rate limit!

Now we have to edit our server.js script to look like this:

const express = require('express');
const mongoose = require('mongoose')
const config = require ('./config')
const bodyParser = require('body-parser');
const rateLimit = require('express-rate-limit');
const helmet = require('helmet');
const expressSanitizer = require('express-sanitizer')
const cors = require('cors');

// Creates the Express App
const app = express();

// Connect to Database
mongoose.connect(config.database)
mongoose.Promise = global.Promise;

// Rate Limit Rule
var apiRateLimit = new rateLimit({
  windowMs: 15*10000, // 15 minute window.
  max: 500, // start blocking after 100 requests
  message: "Too many request created from this IP."
});

// Middleware
app.use(cors());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(expressSanitizer()); 
app.use(helmet())


// API Web Routes
app.use('/api/users', apiRateLimit, require('./api/users'));

// Error Handling
app.use(function(err, request, res, next){
	console.log(err)
	res.status(422).send({error: err.message});
})

// Listen to Request
app.listen(config.port, function(){
	console.log("Now listening for request.")
});

So what did we do? First we added all of our packages we just installed, the we connected our database on mLab using Mongoose. Mongoose has a Promise that’s no longer supported so we had to replace it with Node’s Promise library.

A promise is asynchronous which means it can run along side other code at the same time without blocking other parts of a script. We’re using promises in place of callbacks.

To learn more about Promises check out the MDN Guide on Promises.

Next we added a rate limit, this is going to prevent the abuse of our API in the event someone learns it or your code ends up firing it off multiple times. It’ll help hold back traffic and you can change it’s settings depending on how much you feel need it to be tailored.

IMPORTANT
Without the API Rate Limit it’s possible to brute force your API to figure out your authentication token. a rate limit makes it much harder to forcefully get into.

Next we added some error handling which we can use if something goes wrong, it’ll send back a status code of 422 with an error message.

Moving on we’re going to make a folder called model and inside it a javascript file named user.js.

Inside of this script we will have:

const mongoose = require('mongoose')
const Schema = mongoose.Schema;

const UserSchema = new Schema({
	userId: {
		type: Number,
		required: [true, "Requires a userId."]
	},
	cash: {
		type: Number,
		default: 100
	}
})

let User = mongoose.model('user', UserSchema);

module.exports = User;

We’re using mongoose schema to create a structure for how we want to read our data. This Schema includes the properties and defaults we want. When a new user profile is made they will have 100 cash by default. If we don’t create a new user with the UserId it will fail. Lastly we took our Schema and create a new ‘user’ model.

Our collection will be automatically created and called ‘users’ because models are always singular and collections are always plural.

Last Updated: August 26, 2018

SQL and NoSQL

Table of Contents

Read Chapter

About

When working on the backend you’re going to want to figure out where all your data is being saved and how. Earlier in the REST API tutorial we used mLab who host a MongoDB database service we can use which is NonSQL. You should choose which database you want to use based on the community resources, your teams skills, and the demand you’ll meet in terms of scale, traffic, and how you might be able to scale up if you need to upgrade your resources in a short amount of time without loss of data.

SQL

Structured Query Language (SQL) was made back in 1982 by Microsoft. This is still to this day a common standard to use. These databases use relational data, this means user profile data may have a “user id” variable that exist in one database that can be used to collect the same “user id” value in other database for product orders, action logs, etc. These data is primarily stored into tables and follow a schema which structures the information we want to store. To improve performance SQL databases are scaled vertically such as increasing the server RAM and CPU.

SQL Databases: MySql, Oracle, Sqlite, Postgres and MS-SQL

NoSQL

NoSQL databases existed back in the 1960’s but were not given the same labeled before SQL came about. Just recently within the past decade this back become popular once again to use with the rise of new databases such as MongoDB, RethinkDB, and more. Unlike SQL data is stored as a document usually with key-pair values. This data is considered to be considered schema-less however that only means the schema is dynamic for unstructured data. You can still even use a schema for NoSQL data such as with Mongoose. Unlike SQL, NoSQL databases are scaled horizontally which means increasing performance through adding more servers to reduce the load.

NoSQL Databases: MongoDB, BigTable, Redis, RavenDb, Cassandra, Hbase, Neo4j and CouchDb

146 Likes

Web Frameworks and Libraries

Table of Contents

Read Chapter

About

There are a couple of popular web frameworks and libraries out there that are very common and we’re going to talk about a few of them. Currently is this tutorial you’ve seen we have used React. In this chapter we’ll go into detail about the role React would survive in a development and about who created it.

Learning this technologies is important but you should preferably focus on mastering one of them based on whatever is popular in your area in relation to job openings.

Bootstrap

This is a popular library that makes cross-platform compatibility much easier to accomplish. In today’s world we’re often using our phones to browse more than with our desktops. Back then it was uncommon and technology and design philosophy wasn’t really well developed yet as we tried to make the website be responsive to the size of the screen.

This is regularly used next to web frameworks that play on some of Bootstraps pre-built components.

Angular

This is a framework created by a small Google Team and is actively maintained by them. Now Angular has gone through a scizm during it’s development. AngularJS as it’s now known is the original version of Angular released in 2010 that relied on Javascript and jQuery. In 2016 Angular 2 was released, it was a version of the framework so different and rewritten it was incompatible with the old AngularJS structure which caused some outrage. Angular 2 (Now called just ‘Angular’) uses TypeScript which is a modified version of Javascript that uses Object Orientated Programming (OOP) syntax and logic.

ironically, Google does not actually use their own framework for most if any of the websites however it is a full fledged tool that is still strong and being used. It’s most popular for Single Page Application (SPAs) websites that don’t involve loading new pages but updating a single page.

Some website using AngularJS and Angular (2+).

React

Unlike Angular, React is actually used by it’s own creators – facebook. On it’s own React is just a library with routing capabilities for pages. React also has a very active community that has developed extensions that can turn React into a complete framework when combined with add ons and tools required to develop a website.

They also take a component based approach, where each section of a website such as a footer, sidebar, and header are all separate objects that can be reused on a web page where as Angular relies on update a single page without the need to re-load assets. It’s very popular for multi-page websites.

Websites using React

Security and Attacks

Table of Contents

Read Chapter

DoS and DDoS

Table of Contents

Denial of Service (DoS) is an attack aimed to make the service unavailable to other users who need it.

Methods of Denial

  1. Redirect - This can be done by redirecting traffic to another site, usually by cross site scripting which we’ll talk about soon.
  2. Close Connections - Making the host ports unaccessible that the service relies on to be publicly accessible.
  3. Destruction of Data - Deleting data that is unintended to be missing can cause certain pages and functions of a website to fail to load properly. This may occur due to a SQL Injection.
  4. Resource Exhaustion - The most common form of DoS where you can overload the website with request cause a blockage. In other words, the server will attempt to serve the DoS client first due the queue before a legitimate user who is now caught up in high traffic.

A single client used in the attack is only a DoS attack, however if a network of other computers is used it’s called a Distrbuted Denial of Service (DDoS). Attackers want to be as cost effective as possible in most cases so this is sometimes performed through hijacked and exploited computers. It’s still possible to program a structure that supports automated the process of making new virtual machines such as Linux that can be used by a commanding program.

You can learn more about a few famous DDoS attacks from Cloudflare’s blog.
https://www.cloudflare.com/learning/ddos/famous-ddos-attacks/

Cross Site Request Forgery

Table of Contents

Cross Site Request Forger (CSRF or XSRF), this is a type of attack that can allow the attacker to pretend to be you by hiding behind a form of legitimate access key such as a cookie for a logged in account. These attacks are usually performed masked behind what looks like a real email or link by a trusted source. Not everyone knows their trusted websites complete url, domain, etc and there’s ways to easily make it look like their own.

Example Situation:

  • John logins in and looks into his bank account.
  • The bank gives him a session token.
  • A hacker sends a fake email to John that looks like it belongs to his bank.
  • The link redirects to John’s bank which uses the stored cookie.
  • The hacker now has access and saved his cookie to be used as if he were John.

So how do we prevent this?

Well we can use what’s called a XCSRF Token, which is a representation of the client who made the original request. In other words you may have a token A100 when you click on the link, but the attacker client will get a token of his own B200 for his request. A cookie is used to store a session, but a XCSRF token is used to identify the client themselves during a request. If for example the original requesters token doesn’t match a request trying to use John’s cookie then it will be denied.


Spoofing

Table of Contents

An attacker will make their IP address appear to be a trusted source by modifying it. The IP, ARP, and DNS can all be spoofed to mask the real identify of the client. Similar to the CSRF but at a level that can be used more broadly for other attacks on a network including in a public area. This may often be wifi that seems to be yours or a trusted source (hospital, store, etc.) that you connect to when you are actually connecting to the hackers network.

Phishing

Table of Contents

Phishing is where an attacker pretends to be a trust person, company, or website. It often comes as an email which can start the process of other attacks if clicked such as a CSRF attack. They usually they to cause a sense of urgency or fear to make it quickly look into the links provided.


Man in the Middle

Table of Contents

This is where two parties believe they are communicating with each other such as a network between clients. However one of those clients is being intercepted and receiving the data instead. The attacker can then use this information they receieved to direct towards the original receiever it was meant for but with modified data. There are a number things a Man in the Middle can do as it can go left unnoticed.

An example is how Cloudflare has a man in the middle problem on location of their facilities that traffic data. One can sit at the facility and receieve any unencrypted data going out.

What you want:
A → B
A ← B

What’s actually happening:

A → MitM → B
A ← MitM ← B

This is even easier if the website is not using HTTPS but HTTP, this is very important. HTTPS will encrypt out going data making it harder for the middle man to figure out. HTTP will send the data in plain text which is a problem for us and anyone using the website.


Privilege Escalation

Table of Contents

This is a type of attack that can be performed once a hacker has used a scanner to check all your ports for vulnerabilities such as a open FTP port with no password required. Sometimes these programs used are created with backdoors often for the developers of the software (FTP) uses which wasn’t meant to be used by you or the attacker. Lack of configuration and oversight into what you use can open easily accessible paths for an attacker to take and add their own backdoors.

A fun story about a drone and how the developers of the drone themselves did not properly secure their ports and left default settings, let it be a lesson learned!

DEF CON 23 - Robinson and Mitchell - Knocking my neighbors kids cruddy drone offline
(47:33)


Click Jacking

Table of Contents

Click jacking is where an attack can start tracking what you type, what you click and more. The way they do this is often by using the X-FRAME-OPTIONS header. Some sites allow you to request the web page can show it within a HTML file with a frame tag. This allows them to put the real website on their own and use a script to log every action. This can be prevented by simply denying the X-FRAME header to be used.


Cross Site Scripting

Table of Contents

In short Cross Site Scripting (XSS) is where a scripter will insert a script into the website. The way the do this is when the website does not sanitize (verify) the data coming in. If we were to write some HTML with a script tag and post a comment, the website without sanitization would use that HTML and insert it into the website where it will appear on every client who now visits that page and views the comment. Unless the comment was properly turned into a string we would have running malicious code in our website.

A form of XSS is SQL Injection where you are using the same vulnerability to run SQL code.

This is preventable with a Content Security Policy (CSP) header, where we can add rules the browser will follow. Using a CSP header we can tell the browser to not load frame tags, script tags, etc. We can also whitelist what external websites we want to use.

For example, if Im using Font Awesome, I can add their web url to my CSP whitelist which will allow it to run.


Brute Force

Table of Contents

Brute force is a very popular method of attacking however it’s also one of the most resource intensive and unconventional method.

To clarify a Brute Force attack often will at least one of two things:

  • Use a long list of common passwords from several databases of also known leaked passwords.
  • Attempt every possible combination of characters to match a password.

This may take hours to days, and weeks depending on the resources involved. You won’t often find a single attacker who wants to spend somewhere from $100-300 to be able to use the more troublesome method #2 (every possible combination) of the attack to get in. The best way to counter attack is to simply waste their time for as long as possible.

The way you can stop this attack is by using a rate limit which will end up forcing them to use VPN’s and proxies to add more complication to setting up an attack because that rate limit will stop your average brute force attack if setup properly.


Social Engineering

Table of Contents

Out of every kind of attack that has up until now been about software related attacks and malicious code this is the most successful attack that can cause the most damage. Social Engineering is where you talk to the people over a chat or phone that hold important information about you. Such as a phone company or internet provider. The will usually only ask for basic information such as your name, ZIP code, and maybe a security question that can easily be figured out.

This is an attack not on you but by tricking the people responsible for your data using the most out of your public information just by researching you. Speaking from experience as a person has themselves had their internet provider by fooled by a Social Engineering attack they were able to access everything and even order a SIM card were they could gain access to my two step authentication.

Always use different passwords, that is the best well to defend against this, and don’t make everything about you public, try having a business email, personal, etc.

SSH

Table of Contents

Read Chapter

1 ) What is an SSH Key and What Does it Do?

Table of Contents

An SSH is a Secure Socket Shell (SSH) that performs a cryptographic connection between a server and a client.

There are two kinds of keys:

  1. Public
    • This goes on our web host such as GitLab, Digital Ocean, or Heroku.
    • When used by sites using SSH it acts as a whitelist for who is allowed access.
    • You must always have the matching private key.
  2. Private
    • This copy stays on your computer for the public key to reflect against for authentication.
    • Never share your private key with anyone, treat it like a Social Security Number!
    • You can add a password requirement to these keys when generating them.

I a team based situation, every member would have their own Public and Private key, you should never share these keys within a team as it could become compromised and abused.


2 ) How to Make a SSH Key

Table of Contents

If you are using windows, use gitbash which came with git!

GitBash Controls

  1. Copy: Ctrl + Insert
  2. Paste: Shift + Insert

Key Creation Steps

  1. Open Git Bash.

  2. Paste the text below, substituting in your email address which will generate a key pair.
    ssh-keygen -t rsa -b 4096 -C " *your_email@example.com* "

  3. When you’re prompted to “Enter a file in which to save the key,” press Enter. This accepts the default file location.
    Enter a file in which to save the key (/c/Users/you/.ssh/id_rsa):[Press enter]

  4. At the prompt, type a secure passphrase.
    Enter passphrase (empty for no passphrase): [Type a passphrase]
    Enter same passphrase again: [Type passphrase again]

How Copy the SSH to clipboard:

  • Mac: pbcopy < ~/.ssh/id_rsa.pub
  • Linux: cat ~/.ssh/id_rsa.pub
  • Windows: clip < ~/.ssh/id_rsa.pub

Now you can use this on whatever platform you use that supports SSH such as Heroku and Digital Ocean.


Digital Ocean

Get $10 Free Credit (2 Months) by using my referral link!
Table of Contents

Read Chapter

What’s Digital Ocean and Why use it?

Table of Contents

Digital Ocean uses virtual machines and host them for you so you can access them from your computer from a terminal/command prompt. You don’t often want to use your own home network and expose it to public traffic when hosting an API. You may also want to configure the machine to securely host the bot behind firewalls or go back to a restore point.

Using a virtual machine you’re able to run a functional OS capable of high performance and full customization without damaging your own computer or network.

Create a Droplet

Table of Contents

Digital Ocean of 2018 now has projects!

First we have to make a new project:

Now click create a new droplet!

Make sure that you choose Ubuntu as your distribution, and the first option for Standard Droplets as your size.The Digital Ocean droplet we’ll be using.

MEMORY CPU SSD DISK TRANSFER PRICE
1 GB 1 25 GB 1 TB $5/mo

Choose your region and continue on until you come across “Add SSH Key”

Now we can name our machine, and choose the project we want it to be under. Once you’re done click create.

When it’s finished being built we should now see this:

Connect to a Droplet

Table of Contents

These droplets are made with a root user that we can access with our SSH keys, go to the command prompt anywhere and attempt to connect to it using the IP address as seen on your droplet bar.

# Make sure to replace the IP below with your servers IP address
ssh root@206.81.10.28

You’ll be greeted with a authenticity can’t be established message, just type yes when it asked you if you want to continue.

If you gave your SSH a password it will ask you to type it in, you have 3 chances before it closes the connection.

If you connected you should now see this:
image

Create a Admin User

Table of Contents

Create an SSH user.

adduser roblox

Mod the User.

usermod -aG sudo roblox

Become the user.

su - roblox

Create an SSH directory.

mkdir ~/.ssh

Set permission to only allow this user into it.

chmod 700 ~/.ssh

Create a file for SSH keys.

nano ~/.ssh/authorized_keys

Nano is a editor that will let you copy and paste your SSH key here, make sure it’s on one line. When you are finished press Y to save and Ctrl + X to exit.

Allow the user access to the keys.

chmod 600 ~/.ssh/authorized_keys

Now exit to become the root user again.

exit

Disable Root Login

Table of Contents

Now we’re going to disable the password login, do exist again to close out of the connection then attempt to login as the new user who now has sudo admin.

ssh roblox@206.81.10.28

Now we need to update the SSH configurations file.

sudo nano /etc/ssh/sshd_config
  1. Find PermitRootLogin yes and change it to PermitRootLogin no
  2. Find #PasswordAuthentication yes and change it to PasswordAuthentication no

If you used a SSH key, PasswordAuthentication should be no already.

Now save and close it: Ctrl + X then Y then enter.

Now restart the SSH service:

sudo systemctl reload sshd

Now it will deny us access as root if we tried to login.

Firewall Setup

Table of Contents

We’re going to setup a basic firewall that we’ll configure to deny all traffic except through the standard HTTP port 80 and HTTPS 433 while allowing SSH logins.

This is not a full proof method but it will help prevent many attacks against your server if you have potentially vulnerable ports.

The firewall we’re configuring is ufw

Enable OpenSSH connections.

sudo ufw allow OpenSSH

Enable HTTP traffic.

sudo ufw allow http

Enable HTTPS traffic

sudo ufw allow https

Turn the firewall on.

sudo ufw enable

You can check the firewall status.

sudo ufw status

Automated Deployment

Table of Contents

PM2 is a Process Manager that will run and handles or applications giving us information such as up time and the ability to start, stop, and delete the process. It will also automatically restart our application if it fails. Heroku will only do it a few times for free users before you have to manually turn it on again. That’s where using a virtual machine and using PM2 is in some cases a better option.

Before we start we’re going to want to get Nodejs on our server.
Each line is a different command

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install nodejs

Install PM2 and Git

sudo npm install -g pm2
sudo apt-get install git

sudo Initialize a git bare repository at /opt

sudo git init --bare /opt/roblox-app.git

A git bare repo is specifically for pushing your work to and so on.
Now we’re going to clone to where our code will be located.

sudo git clone /opt/roblox-app.git /opt/live/roblox-app

Add a post-receive hook at /opt/roblox-app.git/hooks

sudo nano /opt/roblox-app.git/hooks/post-receive

Now we’ll use this command setup that will build our app and restart it.

Notice server.js, you should replace this with what script starts your applications such as app.js or bot.js for example. if you followed this guide however you may be using server.js

#!/bin/bash

echo ‘post-receive: Triggered.’

cd /opt/live/roblox-app

echo ‘post-receive: git check out…’

git --git-dir=/opt/roblox-app.git --work-tree=/opt/live/roblox-app checkout master -f

(pm2 delete "roblox-app" || true) && pm2 start server.js --name "roblox-app"

echo ‘post-receive: app started successfully with pm2.

Now we need to give ourselves permissions of the files.

sudo chown -R roblox:sudo /opt

Now to make it executable.

chmod ug+x /opt/roblox-app.git/hooks/post-receive

Now we’ll go back to our local computer.

You can check what remotes you have with git remote -v

git remote add live ssh://roblox@000.00.00.00/opt/roblox-app.git

Make sure you add your own IP and user (assuming you didn’t store it on root).

Now we can push to our live server.

git push live master

If you want to see if your app is running on the server.
pm2 list

Now we need to add auto-startup in case the server restarts!
pm2 startup

It will ask you to run a command as root, but we can use nano so we don’t need to access root.
sudo su -c "env PATH=$PATH:/usr/bin pm2 startup systemd -u roblox --hp /home/roblox"

If it’s running you now have automated deployment.

DNS Setup

Table of Contents

Make sure you setup your DNS or else you can’t properly setup Let’s Encrypt.
Go here to learn how to setup your DNS.

SSL Certificate (Free)

Table of Contents
A Secure Socket Layer (SSL)

https://letsencrypt.org/

Install the tools required by Lets Encrypt.

sudo apt-get install bc
sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Before you continue make sure your DNS is setup.

To check if your DNS is pointing to your droplet run this command which should return your droplets IP address.

dig +short app.example.com

Now lets generate a SSL certificate
Yes you do include the standalone

cd /opt/letsencrypt
./certbot-auto certonly --standalone

Setup Auto-Renewal for the Certificate, since it expires every 90 days.

/opt/letsencrypt/certbot-auto renew
sudo crontab -e

Now we’ll add an auto renew script then save and exit.

59 3 1 * * sudo service nginx stop && sudo /opt/letsencrypt/letsencrypt-auto renew >> /home/dev/logs/letsencrypt-reneval.log && sudo service nginx start

Once you save it, all we have to do now is proxy to from our domain to our app.

NGINX Setup (Free)

Table of Contents
Install NGINX

sudo apt-get install nginx

No we’re going to secure all traffic coming in, and to do that we need to configure NGINX.

sudo nano /etc/nginx/sites-enabled/default

Just delete everything inside and add the following:

# HTTP — redirect all traffic to HTTPS
server {
    listen 80;
    listen [::]:80 default_server ipv6only=on;
    return 301 https://$host$request_uri;
}

Now we’ll going to add some extra security to our SSL using the Diffie-Hellman group.

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

It’ll take a few minutes to complete but once its done we can now create a configuration file for the SSL that will further secure our server. We’re going to use an actual security experts configuration settings.

Now lets make the config file for the SSL.

sudo nano /etc/nginx/snippets/ssl-params.conf

Inside the configuration paste the following then save and exit.

# See https://cipherli.st/ for details on this configuration
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1; 
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off; 
ssl_stapling on; 
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

# Add our strong Diffie-Hellman group
ssl_dhparam /etc/ssl/certs/dhparam.pem;

Now open up the SSL configuration again.

sudo nano /etc/nginx/sites-enabled/default

Beneath your block that we used earlier we’re going to use the following that will be our HTTPS proxy.

# HTTPS — proxy all requests to the Node app
server {
    # Enable HTTP/2
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name app.example.com;

    # Use the Let’s Encrypt certificates
    ssl_certificate /etc/letsencrypt/live/app.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.example.com/privkey.pem;

    # Include the SSL configuration from cipherli.st
    include snippets/ssl-params.conf;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://localhost:5000/;
        proxy_ssl_session_reuse off;
        proxy_set_header Host $http_host;
        proxy_cache_bypass $http_upgrade;
        proxy_redirect off;
    }
}

Save and exit, make sure you changed app.example.com to your websites domain.
Then you can test if it works with the following:
sudo nginx -t
Now we exanble NGINX.

sudo systemctl start nginx

You can test the grade of your SSL here: SSL Server Test (Powered by Qualys SSL Labs)


Heroku

Table of Contents

Read Chapter

What is Heroku?

Heroku is a cloud platform that enables you can build and deploy your web apps to, which Heroku calls ‘Dynos’. You can use this platform to host your project server on. It is more simpler than Digital Ocean but with a few drawbacks.

It lets you monitor your app with logging so you can see what your app is outputting to the console, however this is limited, and you can only see very recent logs. Another drawback is that your app will not run 24/7 unless you verify a Credit/Debit card, this is because Heroku your it’s dyno’s (or apps) on an hourly basis. Every Heroku account is allocated a pool of hours which you can use with free dynos. They also go to sleep after 30 minutes of inactivity. (However there is a workaround to this which we will go over later).

Accounts are given a base of 550 hours each month in which your free dynos can run. In addition to these base hours, accounts which verify with a credit card will receive an additional 450 hours to the monthly quota.

Setup

Before you start deploying your app to Heroku, you need to create an account. After you have created an account, download the Heroku Command Line Interface, which allows you to execute commands from within the Command Line.

Make sure you have git and npm (Node Package Manger) installed and you’re ready to go!

App Deployment

Firstly, we are going to make a folder (this can be anywhere on your PC that’s easily accessible). This will serve as the directory for your server, where you can push to Heroku, or pull from Heroku. Make sure the folder name doesn’t have any spaces.

I’m going to call mine HerokuApp for example purposes, once made, transfer all of your app files to the folder.

Once you have done that, open up Command Prompt, or your command line of choice and type:

cd PATH_TO_DIRECTORY

In my case this will be:

cd C:\Users\Your_User_Account\Desktop\HerokuApp

This will change the working directory in your command line to the folder of our app, so when we execute more commands, they will affect our folder.

In your older have a file named “Procfile” - Do not give it a file type.

A procfile as a the execution command, and Heroku has two kinds of dynos we can use. A web process and a worker process. If we’re running a bot, we use a worker which will stay awake. If we’re using a website like a API server we use web.

In your procfile type:
worker: node myapp.js
or
web: node myapp.js

On a free account we can only run one of these processes so if you are using a web API you can’t run a Discord bot without paying $14 a month for two processes.

Press Enter, and it should ask you to fill in some fields, keep pressing enter through them all, you can edit these details later in the package.json file that will be created. This file will manage the information for your app, such as which node modules/libraries you are using, the name of the app, the main script, and other information.

First we will log in to Heroku:

heroku login

It will ask you to type in your Heroku account credentials (Email and Password).

Next type:

git init

This is going to initialize an empty git repository in our folder, if you want to read more about repositories, you can do so here.

Once it’s finished we now need to push all our files since it’s empty.

git add --all

Give it time to add all your content it will let you know when it’s finished, depending on the speed of your computer this could take some time.

When it’s finished enter the following:

git commit -am “Initial”

By committing this allows us to prepare to push the code to our web ap on heroku, but before we can do that we have to make one first, give the commit time to finish then move on with the following:

heroku create roblox-app

You can name this whatever you like, this new web app will show up on your Heroku account now such as this:

https://dashboard.heroku.com/apps

Finally, we can send our code to the web app like this:

git push heroku master

Once everything is finished being pushed it should tell you that it ran successfully to verify there was no errors.

31 Likes

Roblox Scripting

Table of Contents

Read Chapter

HttpService

Table of Contents

API Reference

What this service allows is for us to make http request to sites outside of Roblox, but you are not able to send request to Roblox. This would mean you would need a proxy server which is basically a server that you send a request to so it can do the intended request for you and send back the same information for you to use.

It’s important to know that Roblox has their own rate limit on it’s usage where you can only send 500 request per minute before it stalls for 30 seconds.

Before you can use the service remember to turn it on in studio under game settings

The service has three kinds of functions we can use, GetAsync, PostAsync, and RequestAsync which is new and has no restrictions on the request type such as allowing for multiple methods (i.e. Put, Delete, etc.). We’ll be using the new RequestAsync function for flexibility.

Making a Request

Let’s look over some example code provided by the wiki first.

    local HttpService = game:GetService("HttpService")
    
    local response = HttpService:RequestAsync{
    	Url = "http://httpbin.org/post";
    	Method = "POST";
    	Headers = {
    		["Content-Type"] = "application/json"; 
    	};
    	Body = HttpService:JSONEncode{hello = "world"};
    }
     
    if response.Success then
    	print("Status code: " .. response.StatusCode .. " " .. response.StatusMessage)
    	print("Response body:\n" .. response.Body)
    else
    	print("The request failed: " .. response.StatusCode .. " " .. response.StatusMessage)
    end

So obviously, we’re going to get the service first before we can use it. Now we just have to break down the function and what it is we’re doing with it.

Url - Your website link, now assuming you setup your own server using Heroku or Digital Ocean you could use your own web server based on the domain you purchased the records you setup. If setup properly you can provide it.

Method - The type of request that’s being made, and you can have multiple on the same web route. The most regular methods you’ll find are GET, POST and less commonly PUT and DELETE.

Header - With roblox you typically want to send JSON as it’s a standard in most cases. However you could also provide a few more details such as if the website the request is being made for may want security details in the header.

Body - This is variables you’ll find, in the example case when we basically saying: http://httpbin.org/post?hello="world" in JSON form which our Express App we convert into a Javascript Object to be used.

If we were to use our own server and based on our REST API chapter it would be a request that looks like this:

local userId = 12030293
local response = HttpService:RequestAsync{
	Url = "https://mywebsite.com/api/users/"..userId; 
	Method = "GET";
	Headers = {
		["Content-Type"] = "application/json"; 
	};
}

Keep in mind how you handle request and don’t hit the limit.


Discord

Table of Contents

Read Chapter

This chapter is a rehash of the original thread!

Creating a Bot

First we visit the developers page!

Create a new Application!
image

Give it an image.
image

Name it.

Make a bot.


image
Copy your Client Id.
image

Go here to the calculator and enter your Client Id, then click the invite link below.

Add it to a server.


image

Connecting a Bot

Make sure you get the npm package: npm install discord.js --save

var discord = require('discord.js');
var client = new discord.Client();
var token = "TOKEN_HERE_FROM_THE_BOT_PAGE"
client.login(token)

If we were to start the bot now we would see it online assuming you put your token in from the Bot page.

Example Script

var discord = require('discord.js');
var client = new discord.Client();
var token = "TOKEN_HERE"
client.login(token)

client.on("ready", () => {
  client.user.setGame(`Making HL3`);
  console.log(`Ready to serve on ${client.guilds.size} servers, for ${client.users.size} users.`);
});

client.on('guildMemberAdd', member => {
  let guild = member.guild;
  let user = member.user
  console.log(`${user.tag} joined ${guild}`)
});

client.on('guildMemberRemove', member => {
  let guild = member.guild;
  let user = member.user
  console.log(`${user.tag} left ${guild}`)
});

client.on('message', (message) => {
	console.log(`${message.author} said ${message.content}`)
});

So here we added a few events, ready, guildMemberAdd, guildMemberRemove and message. You should know that guilds are your discord servers. What this will do is trigger a message when someone joins the server, leaves, or speaks. By doing set game we’re changing the bots current playing game status.

Writing Commands

var prefix = env.process.PREFIX || '!';

function isCommand(command, message){
	var command = command.toLowerCase();
	var content = message.content.toLowerCase();
	return content.startsWith(prefix + command);
}

client.on('message', (message) => {
	if (message.author.bot) return; // Dont answer yourself.
		var args = message.content.split(/[ ]+/)
    
	if(isCommand('Ping', message)){
    		message.reply('Pong');
    	}
   	if(isCommand('Promote', message)){
    		var username = args[1];
    		if (username){
    			message.channel.send(`Checking ROBLOX for ${username}`)
    		} else {
    			message.channel.send("Please enter a username.")
    		}
    		return;
	}
});

In this example we’ve added a function meant to handle checking our commands. Note that I used a environment variable check for PREFIX, you should use this if you host a web server bot and a local bot. If you make a environment variable on them it will use it so you don’t run the same command twice.

The first command is for saying ping such as “!ping” and the bot will respond pong.

The second command is a check to make sure you gave a username.

If the command was “n!promote TechSpectrum” then args[0] is “n!promote” and args[1] is “TechSpectrum”.

Connect to Roblox

We’ll need to install npm install roblox-js and require it with var roblox = require(‘roblox-js’);

(Documentation here!](Home · sentanos/roblox-js Wiki · GitHub) by @Froast

Now we can login.

roblox.login({username: "Username_Here", password: "Password_Here"}).then((success) => {

}).catch(() => {console.log("Failed to login.");});

Roblox-js is a library that uses promises, along with a database ODM like Mongoose that also uses promises it means less blocking for traffic.

Now lets add a promotion command, below is the full script we should have.

var discord = require('discord.js');
var roblox = require('roblox-js');
var client = new discord.Client();
var token = "TOKEN_HERE"
client.login(token)

roblox.login({username: "Username_Here", password: "Password_Here"}).then((success) => {

}).catch(() => {console.log("Sorry, it failed.");});

>**NOTE** - The cookie session will expire within 24 hours, you can write a script to re-use and >refresh a cookie. You can store the cookie in a database or JSON file and save it with the date it >was created to time a refresh. If you login at least 25-30 times (est.) within 20 minutes you will get >a captcha!


client.on("ready", () => {
  client.user.setGame(`Making HL3`);
  console.log(`Ready to serve on ${client.guilds.size} servers, for ${client.users.size} users.`);
});

client.on('guildMemberAdd', member => {
  let guild = member.guild;
  let user = member.user
  console.log(`${user.tag} joined ${guild}`)
});

client.on('guildMemberRemove', member => {
  let guild = member.guild;
  let user = member.user
  console.log(`${user.tag} left ${guild}`)
});

var prefix = '!';
var groupId = 2750654;
var maximumRank = 20;

function isCommand(command, message){
	var command = command.toLowerCase();
	var content = message.content.toLowerCase();
	return content.startsWith(prefix + command);
}

client.on('message', (message) => {
	if (message.author.bot) return; // Dont answer yourself.
    var args = message.content.split(/[ ]+/)
    
    if(isCommand('Promote', message)){
    	var username = args[1]
    	if (username){
    		message.channel.send(`Checking ROBLOX for ${username}`)
    		roblox.getIdFromUsername(username)
			.then(function(id){
				roblox.getRankInGroup(groupId, id)
				.then(function(rank){
					if(maximumRank <= rank){
						message.channel.send(`${id} is rank ${rank} and not promotable.`)
					} else {
						message.channel.send(`${id} is rank ${rank} and promotable.`)
						roblox.promote(groupId, id)
						.then(function(roles){
							message.channel.send(`Promoted from ${roles.oldRole.Name} to ${roles.newRole.Name}`)
						}).catch(function(err){
							message.channel.send("Failed to promote.")
						});
					}
				}).catch(function(err){
					message.channel.send("Couldn't get him in the group.")
				});
			}).catch(function(err){ 
				message.channel.send(`Sorry, but ${username} doesn't exist on ROBLOX.`)
			});
    	} else {
    		message.channel.send("Please enter a username.")
    	}
    	return;
    }
});

If you’ve scripted a database with mongoose you can use the mongoose model and get the user data to display it in discord.

MySQL Database

Table of Contents

Read Chapter

SQL DATABASE SETUP

Install MySQL

sudo apt-get update
sudo apt-get install mysql-server

Test if it’s running.

systemctl status mysql

Secure the installation: You will want to type yes (y) because it’ll reduce the risk of attackers getting in.

mysql_secure_installation

Configure the MySQL Server

Access the MySQL configuration file.

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Look for “bind-address” and set it to be your servers IP address.

Now we need to secure the connection since it will be over the internet, so type the following:

require_secure_transport = on

And to allow for SSH connections to work we run this command:

sudo mysql_ssl_rsa_setup --uid=mysql

Now restart the MySQL server.

sudo systemctl restart mysql

Now to confirm that it’s listening on our servers port.

sudo netstat -plunt | grep mysqld

You should see it listening to port 3306

Now we let our firewall know we want to let it be accessible.

sudo ufw allow mysql

Setup a Remote User

Connect to the root user in MySQL.

mysql -u root -p

Create a database.

CREATE DATABASE database;

Create a new user and give it a password.

CREATE USER 'roblox'@'%' IDENTIFIED BY 'password';

Grant the new user all privileges.

GRANT ALL PRIVILEGES ON database.* TO 'roblox'@'%';

Update privileges.

FLUSH PRIVILEGES;

Now exit the MySQL prompt.

exit

Programs to Connect to the Database:

  • HeidiSQL
  • MySQL Workbench

Cloudflare

Table of Contents

Read Chapter

Cloudflare has some DDoS protection services which helps prevent massive attacks from types like bot networks that could potentially bring down your website.

First you need to make an account with Cloudflare, and add your website.

It’ll have you go through some steps to make sure that your DNS is configured and see what’s already setup.

Note:

  • You do not need to setup networking on Digital Ocean
  • You will use the Nameservers provided by Cloudflare and put it on your DNS redirects similar to how you would of set it up if you let Digital Ocean handle.

When Cloudflare is finished confirming your DNS once you added the Nameservers you can create the following DNS records. (Page is on the DNS tab!)

Make sure that you put your discord droplet’s IP, or whatever other VPS that gives you an IP. What the image above shows is one of my own setups.

Now we need to allow universal SSL records, so go to Crypto and change the settings from Flexible to Full.

You should now have a working Cloudflare setup for your server.

37 Likes

Docker

Table of Contents

Read Chapter

Docker is a system that allows for you to take an application and store it in a container. It acts as a virtual OS environment on the application level inside an existing machine while not having the same overhead and abundance of needless resources if you were to run the application on multiple servers.

Its a great way to contain your project, if you spread your application into microservices.

A microservice is a subset of parts such as: a web page, authentication server, database, and API. Instead of running each as a single application you are dockerizing the parts so if one fails the rest can still run just fine.

When you combine technology with Kubernetes you can scale your microservices to fit the demand of the traffic or replace a dead docker that crashed and stopped responding.

Kubernetes

Table of Contents

Read Chapter

Kubernetes was developed by Google and open sourced for development such as with the company Red Hat. This technology was created to vastly improve web scaling where we use to only rely on techniques such as spreading traffic to other servers running duplicates of usually the exact project.

What kubernetes does is allow you to run your dockerized services in clusters, what this means is you can take the web page service and if it gets too much traffic the kubernetes will orchestrate your environment so that it creates another docker to send traffic to reducing the stress on one application.

This is extremely useful because it scales what needs to be increased and reduces where work isn’t needing essentially optimizing your servers full potential.

If you ever want to work on a web service that you expect to have a large amount of traffic or even just want to optimize and make your service perform to the best of its ability for the future by having the work done early, I suggest you look into using kubernetes and its widely growing and now supported on Digital Ocean.

Career & Education

Table of Contents

Read Chapter

Job Types

  1. Frontend
  • HTML, CSS, Javascript
  • Web Designer
  • User Experience (UX) Designer
  1. Backend
  • Database Developer
  • Javascript, PHP, SQL
  1. Data Analyst
  • Statistics
  • Database Languages (SQL)

Job Searching

For those of you who want to pursue a career in the field of web development you have to understand a few things such as the resources available to you, and how to find what you want to do.

To start lets think about what your area you live in is, you want to understand what’s popular around you because some business may be in demand for database analyst, angular, or react web designers. It’s important to look into this by using a site to find local jobs:

Job Sites:

Be sure to read what jobs are demanding carefully, you should also go to their website and see if you can apply directly through it. Most recruiters are looking for people with prior experience and at least an education with a degree.

There are some but not many jobs where you can enter the field with no education but an impressive portfolio.

You should put all your effort into your portfolio website to be professional as possible.

Getting an Education

There’s multiple ways to get some form of education in the field, such as a university or bootcamp which is what I’ll be discussing.

There are different schools popping up for an online only education such as https://lambdaschool.com/ where you do not need to pay a think until you get a job after you graduated.

Now this is usually 9-12 months depending on if you are Part-Time or Full-Time in a university, however that’s where bootcamps come in if you want a shorter term education.

Bootcamps will typically last about 1-3 months and you may be required to move locations in order to participate. The important thing in this is that you are able to work under time constraints and with teams in person versus an online education that may involve less hands on help and teaching.

Here’s a few sites to find bootcamps:

  1. Best Coding Bootcamps of 2021-2022 | Course Report
  2. Best Coding Bootcamps of 2021-2022 | SwitchUp
  3. Web Development Immersive Course | Devmountain

Universities:

  1. Web Development Degree Online | Bellevue University
  2. https://www.fullsail.edu/degrees/web-design-and-development-bachelor
  3. Top 15 Most Affordable Online Bachelor’s in Web Development and Design – Graphic Design Degree Hub

Double Authentication

Table of Contents

Read Chapter

Double authentication has been around for quite some time and is one of the next steps in increasing our security to reduce attacks. Any decent professional business needs to have include some form of double authentication service in order to protect its customers. This is very important because security practices in the modern world should not be taken lightly.

SMS

This method involves sending the user a text message through their phone service, which will typically cost to use unless you use a free service if you have a low customer base.

The issue with this method is not everyone is willing to give their phone numbers online, and if you do store this is in a database it’s best to encrypt it like regular password data.

Another issue is that if you lose your phone it is possible for an attacker to not even need your password but just your phone so long as they can figure out your email and the sites you use.

Authy / Google Authentication / TOTP

Time-Based One Time Passwords (TOTP) is one of the more popular double authentication methods used by websites like Discord uses.

This method involves a secret key on your account which revolves and changes the code required based on the current time. In order to make sure the user and the service provider are on the same time they may use the UTC or Unix time and round to the nearest number so that the code will remain the same long enough for you to type it in.

The advantage of this over SMS is if your phone company gives an attacker your number like a SIM card they can’t get access because the password is only stored on the device itself. However if you lose your device it’s over, unless you kept a copy of the one time backup codes.

This is why some companies offer both SMS and TOTP.

Biological (Fingerprint)

Some companies like PayPal, Chase Bank, and more offer fingerprint logins. This is useful but only for your phone, you still do not need your fingerprint if you were signing in only on your computer unless you required the user to have the phone for signing in.

Geographical (Location)

One of the less common methods of authentication is location based access, normally you might see this for private companies or with services by google where if someone tries to access your account from a different location it would be blocked because you have never authenticated from that area before.

This is useful but otherwise possible to get around with a VPN unless you were to track the devices machine Id.

Time

Another less common though useful for work, is a login that only works within a given time such as if you were working a job during the evening. If someone were to try to login at midnight or early in the morning when you were not working this login would be rejected.

Security Device

Sometimes you can even encode a device like a USB or card to have a code which allows for you to access and login to another device or website. One example of this is hotels which have their doors require an key card.

In the military you may also not be able to access a military computer until you put in a secure USB and a administrator allows for you to access the login then proceeds to monitor your actions.

Minecraft Server

Table of Contents

Read Chapter

About

This is for those of you who want host your own game server, I would probably like to cover Rust and Garrys Mod in the future. I will however be covering Hytale when it releases. I understand this isn’t related to roblox but it is pretty cool to learn how to do if you want to explore what more you can do with the web and practice with other languages such as Java using Spigot, which is a modified version of Minecraft for custom servers.

I’m going to teach you how to make a Spigot server that’s accessible through a subdomain such as “play.minecraftgames.com

Without further ado lets start!

Requirements

  1. Java - http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html
  2. Spigot - Spigot
  3. Digital Ocean Server (Based on the Chapter Here!)
  4. PM2
  5. Git
  6. Linux User (This tutorial will use “minecraft” as the user, replace it with your own)

Setup Spigot

Create a folder on your computer and name it something such as “Spigot Server” or whatever you want. When you’re finished drop the spigot jar into the folder and double click the jar.

Java should be running the build to create the files the jar needs in the folder, this sometimes can take a minute to finish.

First time around you will see it only build a few files such as the “eula.txt”

Go into the file and change the agreement from false to true.

Now double click and run the jar again so it can finish building the rest of what it needs.

Create a text file, you are going to make two different scripts.

Assuming your spigot jar name is “spigot-1.13.2”

JAVA -Xmx1024M -Xms1024M -jar spigot-1.13.2.jar

Save as “start-sever.bat” with Save As Type being “All Files”

Now make another one with:

!/bin/bash

cd /opt/live/spigot-server;

JAVA -Xmx1024M -Xms1024M -jar spigot-1.13.2.jar

Save as start-server.sh with Save As Type being “All Files”

The first file was for Windows to start the server (Your computer), the second file was for Linux (our server).

What we told it was a command to start the server with 1 GB of RAM (1024M)

Finally we make the folder a git repo.

git init
git remote add live ssh://minecraft@YOUR.IP.ADDRESS.HERE/opt/spigot-server.git
git add .
git commit -m "Spigot"

Setup Server

First we need to install java to our server.
sudo apt-get install git openjdk-8-jre-headless

You can test what version you are running with java -version and should see something like this.

image

Assuming you already followed the steps to setup a Digital Ocean server you should have a linux user already created.

Now we need to tell our firewall to allow port access to Minecraft.
ufw allow 25565/tcp

Next we need to create a git repository, this is similar to the previous Digital Ocean chapter. I will assume you already installed Git and PM2, if you have not please go to the chapter to see how.

Now lets create a empty bare repo.

sudo git init --bare /opt/spigot-server.git
sudo git clone /opt/spigot-server.git /opt/live/spigot-server
sudo nano /opt/spigot-server.git/hooks/post-receive

Now add the script.

#!/bin/bash

echo ‘post-receive: Triggered.’

cd /opt/live/spigot-server

echo ‘post-receive: git check out…’

git --git-dir=/opt/spigot-server.git --work-tree=/opt/live/spigot-server checkout master -f

(pm2 delete "spigot-server" || true) && pm2 start start-server.sh --name "spigot-server"

echo ‘post-receive: app started successfully with pm2.

Now to give ourselves permissions over the file.
sudo chown -R minecraft:sudo /opt

Now make it executable.
chmod ug+x /opt/spigot-server.git/hooks/post-receive

Now on your computer push your files to the server with the remote made from earlier.

git push live master

Make the start-server script executable, when the project is uploaded.

chmod +x /opt/live/spigot/start-server.sh

DNS Setup

Assuming you are using cloudflare and/or your minecraft server is being ran on the default port make a new A record with the subdomain you want.

For one of my servers I am using “mc” for “hytavern.com” so my server is accessible on “mc.hytavern.com

If you aren’t using cloudflare, I suggest you read this guide:

GoDaddy: Setting Up A Minecraft Server Sub-Domain in Godaddy - Apex Hosting
Namecheap: https://www.namecheap.com/support/knowledgebase/article.aspx/9765/2208/how-can-i-link-my-domain-name-to-a-minecraft-server

Startup

Now PM2 should of ran your server command script start-server.sh but if not you can simply refresh it with pm2 restart spigot-server and to read your logs go to pm2 logs spigot-server and see it build.

When you make changes to your local project and upload them to the server now it will restart the server with your changes.

Now go to minecraft multiplayer and add your server.

image

You should be all set to go, if you want to learn more about what you can do research “BungeeCord” for multi-networking several spigot servers to work together. This is similar to load-balancing to reduce stress on one server. Of course this will cost a pretty penny but it’s required to run large-scale servers.

OS Bootable USB

Table of Contents

Read Chapter

What is a bootable USB?

A bootable USB is where you have an Operating System (OS) installed on a USB device that can be used by your computer to start up with. This involves turning off your computer to restart and enter the options menu, where you can modify the BIOS to look for what device to look for to boot from first. We’ll go over how to install Fedora which is a version of Linux.

Fedora Linux

First we need to download Fedora, you can find the Media Writer on their website here:

Download Fedora Workstation

When its finished installing, click Workstation, this is a desktop GUI version.

image

Kali Linux

Kali Linux is a version of Linux built around the existing OS. It’s primarily used for security penetration test by security companies. It’s tools are usable however in legitimate attacks, and Offensive Security (Creators), as well as I, do not endorse unethical hacking.

We’ll need to get a version of Kali from the website, be sure to get the bit version that’s correct to your computer. If you have a 64 bit Windows computer, get 64 bit preferably.

Get Kali | Kali Linux

Full Version (64 Bit): https://cdimage.kali.org/kali-2019.1a/kali-linux-2019.1a-amd64.iso
Light Version (64 Bit): https://cdimage.kali.org/kali-2019.1a/kali-linux-light-2019.1a-i386.iso

Note: Light has less tools to make the download smaller, most tutorials use the full version, you can still download what tools you need from the Kali Linux website.

Once that’s finished get Rufus.

https://rufus.ie/

This will let you burn the ISO onto the flash drive, you will need to format your USB, make sure you take everything you want off of it first.

When your image is finished burning to the drive you can eject the USB.

Right now if we started up the USB OS, we wouldn’t have anything save since only our hard drive is being used to save and write to by our hardware’s OS.

Here’s a video on how to make a partition for save persistence:

How To Make Fully Loaded USB PERSISTENCE | Kali Linux 2018.2 - YouTube

Full Video Setup (RECOMMENDED):
How to make Kali Linux persistent USB - YouTube

Boot from USB

This is different for every computer, but what you usually want to do is turn off and restart your computer. Wait until it ask you to enter your BIOS such as with F10. It varies based on what you are using so look up what motherboard version you have and find a guide or video on how to enter these settings.

You want to change your boot menu to use USB or Removable Drive as the first choice, if it doesn’t find your drive, it goes to the next option.

On my computer when its starting its F10.



FEDORA




KALI LINUX
Note: Only use persistence if you have a split storage available on the flash drive.
image

Please check back to the Kali section for the remainder of what to do.

Discourse Forum

Table of Contents

Read Chapter

SMTP Server

A Simple Main Transfer Protocol (SMTP) server is required in order to use Discourse, and for this guide we’ll be using Mailgun.

It’s a service where you can pay as you go, where you start with, so once you make an account we’ll start adding our domain.

Before you go any further, please make sure you purchased a domain
Please see the DNS setup chapter, and preferably also use the Cloudflare chapter.

Make sure you do mg before you domain name, we’re going to use this for your DNS to speak to our SMTP server on mailgun.



  1. Make 2 TXT record type, and paste the valuest.
  2. Make 2 MX record types, set the priority to 10, then paste the value given.
  3. make a CNAME record with the given hostname and paste the value given.

Lastly, we’re going to assume you want to host your forum on a subdomain; forums.website.com.

While we’re setting up the DNS now for the mailing server, lets go ahead and add a CNAME “forum” that is an alias of your websites name.

This may take time for your DNS to properly register, give it around 15-30 minutes (sometimes it may be hours)

Your DNS setup should look a little something like this (ignore the mc, that was for minecraft).
Just make sure if you’re using cloudflare, that the email.mg has a grey cloud.

Once you check your records and it comes out right you should get the following checkmarks:
image

The other records like MX may still be a “?”

When this is finished you’ll be able to access your dashboard.

Now we need to login to our server and start installing Discourse.

Warning
You can use 1 GB of RAM but it’s recommended you use 2 GB. Be careful, if you are using Digital Ocean build your server at what you want to keep it at. If you do it at $15 for one tier, you won’t be able to downgrade due to the memory storage being undowngradable.

Setup SSL

We want our site to be running on a secure server (HTTPS) so we’re going to be creating an Secure Socket Layer (SSL) certificate to encrypt our traffic which makes our site safer for our users.

sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Go to the directory:

cd /opt/letsencrypt/

Install missing packages:

./letsencrypt-auto --help

Generate a certificate, replace the text below with your email and domain name; again you must make sure you use the forum subdomain for this.

./letsencrypt-auto certonly --standalone --email username@example.com --agree-tos -d forum.example.com

Setup Nginx

If you gone through my Digital Ocean tutorial you should have this part mostly done but for those of you skipping to this chapter we’ll go over it again.

Install Nginx:
sudo apt-get install nginx

Make a diffie-hellman group.
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Make a configuration for the SSL.
sudo nano /etc/nginx/snippets/ssl-params.conf

Paste the following into your file.

# See https://cipherli.st/ for details on this configuration
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1; 
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off; 
ssl_stapling on; 
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

# Add our strong Diffie-Hellman group
ssl_dhparam /etc/ssl/certs/dhparam.pem;

Create a new config file for Discourse
sudo nano /etc/nginx/sites-enabled/discourse

Here is what your file should be like: keep in mind you should replace all example domain names with your own name.

server {
    listen 80; listen [::]:80;
    server_name forum.example.com;  # <-- change this
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name forum.example.com; # <-- change this
    ssl_certificate /etc/letsencrypt/live/forum.example.com/fullchain.pem; # <-- change this
    ssl_certificate_key /etc/letsencrypt/live/forum.example.com/privkey.pem; # <-- change this
    include snippets/ssl-params.conf;
    http2_idle_timeout 5m; # up from 3m default

    location / {
        proxy_pass http://unix:/var/discourse/shared/standalone/nginx.http.sock:;
        proxy_set_header Host $http_host;
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Test your configuration to see if there are no issues.
sudo nginx -t

Now restart nginx.
sudo service nginx restart

Setup Discourse

Install Docker / Git
wget -qO- https://get.docker.com/ | sh

If you read my earlier chapter on Docker you should have a bit of understand of what it does; it simply “containerizes” our Discourse forum on a virtual VPS within our Linux server. You can read that chapter for more details.

Install Discourse

sudo -s
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse

You will need to be root for the setup, when you are we can run the setup process.

./discourse-setup

You’ll have to answer the following questions:

Hostname for your Discourse? [discourse.example.com]: 
Email address for admin account(s)? [me@example.com,you@example.com]: 
SMTP server address? [smtp.example.com]: 
SMTP port? [587]: 
SMTP user name? [user@example.com]: 
SMTP password? [pa$$word]: 
Let's Encrypt account email? (ENTER to skip) [me@example.com]: 
  1. Hostname: Assuming you want to host this on subdomain “forum”, we’re using forum.hytavern.com (use your domain name)
  2. Make the email an account you regularly use.
  3. Your SMTP address is: smtp.mailgun.org
  4. Leave the port what it is, just press enter.
  5. Get your credentials on the Mailgun website, you’re typical username will be postmaster@mg.website.com
  6. You may need to do a password reset on the website, DO NOT mess this up, you can copy and paste your pass into this field.
  7. We’re going to do ENTER and skip their Lets Encrypt creation because we made our own.

Now we need to configure it to work for nginx so we’re going to modify our yml file for deployment.

Make sure the discourse container is not running:
cd /var/discourse
./launcher stop app

sudo nano /var/discourse/containers/app.yml

  1. Comment out "templates/web.ssl.template.yml" with #
  2. Add "templates/web.socketed.template.yml"
  3. Comment out expose: and "80:80" and "433:433"

Your setup should look something like this:

image

Before you rebuild the app…
If you want to integrate discord posting for new threads into your discord server get this plugin.
https://github.com/discourse/discourse-chat-integration.git
Set up Discord notifications using the discourse-chat-integration plugin - admins - Discourse Meta

Now we rebuild the app.
/var/discourse/launcher rebuild app

Monolith Applications

Table of Contents

Read Chapter

So this Part 1 of the discussion Monolith vs. Microservice applications, and this chapter focuses more on the definition of what a Monolithic application is. Monolith applications in short; is a combined application of many parts: API, Authentication, Databases, Webpage, etc.

More complexity such as Microservices isn’t always the solution. You should be scaling your solution to your problem.

Pros

Sometimes depending on the size of your business this is all you need, such as running a local business or your have a small team. Typically projects of this type don’t always have to take to long to rebuild, and can be more maintainable depending on what the application is for, and you have the server balancing to handle the traffic. Development can be a lot faster when your team understand the entire project inside and out and test quicker.

Cons

This means when a server goes down usually the entire application may end up going down if there is no backup. Another issue is if one part of the application fails, the entire structure can domino and start breaking. If one part of your application relies on a portion to function and it fails, well it’s no longer available and can try to fallback on cache but with so much traffic this is also prone to fail. Monolith applications very difficult to scale, you almost certainly won’t be able to scale for international level business. Now, going against the pro is having an enterprise size team means developers will step on each others toes more easily and break each others work. Trying to debug for this because a monolith task in itself at times without the proper tools.

Microservices

Table of Contents

Read Chapter

This is Part 2 of the Monolith vs. Microservices discussion.

More complexity such as Microservices isn’t always the solution. You should be scaling your solution to your problem.

Lets talk about something a little complicated but break it down and simplify it; it’s something that’s been around for years but big companies are just now giving a name.

Microservices are essentially broken up parts of a larger application; remember our Monolith explanation? You would have your website on it’s own server, the API on it’s own server, and so on. If one portion such as perhaps the API server goes down for the Xbox or Playstation users the service is still available for PC, TV, etc. These microservices are typically handled by Docker images.

A docker image contains each individual application (component); and Kubernetes organizes the amount of docker images needed in order to handle the stress and demand required of those components.

So lets think about an API service for coding, maybe too many people are hitting that server, so Kubernetes will “spin” up another instance of that docker image.

Pros

Microservices create availability, and is able to scale to international level when dealing with the massive amount of traffic that needs to be organized. You are also less prone to massive application crashes across the board as long as you avoid accidentally looping the communication of your servers and then it can’t communicate back to the client (you).

In a large team, especially enterprise, your teams can now work properly on their own section of the project. If something goes wrong on one server, it’s easier to pinpoint the source of the cause.

Cons

It takes a massive amount of time, and organization to get right. It can be slow to develop for Microservices for small teams, and often becomes too much work. You need a sizable team capable or handling the network of services and communicate properly because not everyone may know each part of the application more than ever but their own portion they are assigned to.

Netflix and Uber (Monolith to Microservices)

Table of Contents

Read Chapter

Netflix is great example of transitioning from Monolith to Microservices architecture due to demand and changing how their service operations from DVD delivery to streaming.

Here’s some very informative talks in the process that you should watch if you want to understand the process and mindset that goes into this business decision.

Mastering Chaos - A Netflix Guide to Microservices - YouTube

Microservices at Netflix Scale: Principles, Tradeoffs & Lessons Learned • R. Meshenberg • GOTO 2016 - YouTube

"Caching at Netflix: The Hidden Microservice" by Scott Mansfield - YouTube

And this last one is Uber’s advice:

What I Wish I Had Known Before Scaling Uber to 1000 Services • Matt Ranney • GOTO 2016 - YouTube

33 Likes

Wow this is amazing, great job :+1: I’ve got my reading work cut out for me :wink:

8 Likes

This is a really great tutorial. Seriously wish I had this when starting out, it would’ve made my life so much easier. Keep up the good work!

2 Likes

I only been doing this for a little over a year, Im currently trying to learn more SQL due to most clients asking for it due to the analyzing tools supporting them way more than NoSQL which uses business intelligence with a ridiculous price tag. On top of that I’m learn ethical hacking so I can better understand attacks and secure my applications againdt them.

I’d love to see more web development project come to roblox. If I had tutorials like this around that could lay put the groundwork for me I would of had a easier time learning what seemed like a daunting and massive undertaking which is why I never dared to touch web debelopment until… Discord bots.

Anyways, Ill keep learning and end up sharing tutorials and my services.

4 Likes

I’m also gonna toss in a suggestion for serverless here as it’s a really great tool for building REST APIs these days, especially with services like Google Cloud Functions offering quite high free tiers. Not usable for Discord bots though.

2 Likes

Yeah and thats another thing, Google and AWS, I need to cover them eventually, its too bad web development aint cheap, heh.

Note to self: Apache and Nginx comparison

2 Likes

Yeah, it really hurts beginners getting into it. This is improving though with a lot of larger tech companies such as Google and Amazon providing better and more interactive tutorials and such. It would’ve been nice if I could’ve learned technologies like Kubernetes earlier, but cost really prevented me from doing so. Thankfully GCP has the $300 free tier thing otherwise I still probably wouldn’t have learned some of the things I know now.

Note to you: Nginx is superior to Apache

4 Likes

Perhaps Ill consider hosting for kickstarting projects free that interest me. My clan database service does pretty well with its income so maybe I’ll save up enough if anyone wants to take me up on that offer.

PS: Agreed the load balancing especially seems to be of more preference.

2 Likes

Very well made tutorial, great read and serves as a very good resource for those looking to get into Web Development.

2 Likes

I think @grilme99 would want this post

To grillme99:
You should read it all. It’s useful very useful. I’m using this tutorial to create a promotion system.

2 Likes

Great tutorial, I wish this had been out when I started on a database for one of my groups because I ended up using PHP instead.

1 Like

PHP of all languages?!?

1 Like

I already was :stuck_out_tongue: thanks for the heads up though!

1 Like

Yep, I’m working on converting some of it to sentanos’ roblox-js package though.
Also, what is wrong with PHP?

1 Like

One recommendation from me, is not to use rethinkdb, it’s not very well supported now, the company who worked on it closed. So it’s stuck in a open source limbo, use mongodb if you’re going to use a nosql database.

3 Likes

Can you continue this to tell you how to the User Schema in REST API to add to the MogoDB Database?

1 Like

Yes I will be updating however recently just yesterday roblox-js’s login no longer works so I have to update and fix it or simply use a different library.

There isnt much remaining left to the API tutorial besides applying the schema. Sorry about that.

1 Like