NodeJS Datastore

Node JS datastores

Hello today I have written a simple module to allow you to host your own datastore server.
It is small and simple to use and set up.

Prerequisites

On your server you need to be have node installed.
If you are running a Debian or Ubuntu linux based server install this:

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs

This will install the base NodeJS which is required to run the script on the server.
Then what you need to do is install NPM, which is NodeJS’s package manager.

sudo apt install npm

That will install NPM.
Now you need to install the MySQL package with NPM.

sudo npm install mysql

Now you need to install and setup MySQL.
If you want to set up the cli to create the database and the table, I am guessing you already have MySQL set up and have a basic knowledge of MySQL.

Follow this article on how to install a LAMP stack which will provide you with MySQL and PHPmyAdmin: Here

Now you have installed PHPMyAdmin go to the homepage and log in.
Go to the SQL tab (2nd from the left on the top) and copy and past the following into the text input.

CREATE  TABLE  `store` (
`dataKey`  varchar(50) NOT  NULL,
`value`  text  NOT  NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER  TABLE  `store`
ADD PRIMARY  KEY (`dataKey`);

This will create the Database and table for you.

Brilliant! You are ready to get the files set up on your server.

Setting up

Now you need to make a file called index.js and open it.

Paste the following into the file.

var http = require('http');
var url  = require('url');
var mysql  = require('mysql');
const hostname = 'gsck.co.uk'; // REPLACE WITH YOUR DOMAIN
const port = 1234;

var con = mysql.createConnection({
    host: "localhost", // YOUR HOST HERE
    user: "", // YOUR USERNAME HERE
    password: "", // YOUR PASSWORD HERE
    database: "datastore"
});

const server = http.createServer((req, res) => {
    res.statusCode = 200;
    res.setHeader('Content-Type', 'text/plain');
    var urlParts = url.parse(req.url, true)
    var key = urlParts.query.key
    var value = urlParts.query.value
    if (value == null) {
        con.query("SELECT * FROM `store` WHERE `dataKey` = '" + key + "';", function (err, result, fields) {
            if (err) {res.end("err")};
            if (JSON.stringify(result[0]) == null) {
                res.end("error");
            }else{
                res.end(JSON.stringify(result[0]));
            }
            
        });
    } else {
        con.query("SELECT * FROM `store` WHERE `dataKey` = '" + key + "';", function (err, result, fields) {
            if (err) {res.end("err1")};
            if (result[0]==null) { // MAKE ROW
                con.query("INSERT INTO `store` (`dataKey`, `value`) VALUES ('"+key+"', '"+value+"');", function (err2, result2, fields) {
                    res.end("successful");
                });
            } else { // UPDATE EXISTING
                con.query("UPDATE `store` SET `value` = '"+value+"' WHERE `store`.`dataKey` = '"+key+"';", function (err2, result2, fields) {
                    res.end("successful");
                });
            }
        });
    }
});

server.listen(port, hostname, () => {
    console.log("Loaded. Running on "+hostname+":"+port);
});

Now exit and save the file and run it.
To run it do

node index.js

Now that the server is running we can now move to the Roblox side.

Get this model, and put it into studio.

Now open up the modulescript and put in the host. This must include the http:// and port or it will not work.

Here is an example:

local nodeDS = require(game.ServerScriptService.nodeDatastore);
local http = game:GetService("HTTPService");
local tablee = {Test="lol",Points=10};
nodeDS:Set("testKey",http:JSONEncode(tablee));
print(nodeDS:Get("testKey"));

VoilĂ , you now have set up your own custom datastore server!

Feedback would be appreciated!

23 Likes

FYI the code isn’t super clean, but it works.

What node server do you use?

Well I am hosting it on my own VPS which I host all of my stuff.
You could possibly use heroku to host it instead, but I have never used it before so I wouldn’t know what to do.

1 Like

Okay thanks.

1 Like

Why use this instead of Roblox DataStores?

Possibly could be used as a backup if anything, you could have a check for getting data from DataStores if that fails attempt to fall-back to this if that fails then well you’re toast. This is probably the best use case in my opinion.

Roblox Datastores are limited by datasize and rate limiting, where as this you are limited to the 500 http requests a minute.

1 Like

And for stuff like Bereza’s saving method on mining heavan you wont be filling roblox with loads of saves etc

1 Like

Thats another idea. I never thought of that.

I actually learned how to code in NodeJS and make databases because at the time several years ago and even today DataStores are still not want I want.

Cross-Place Compatible, Not Universe, but different places like groups. Plus the level of freedom and control of data isn’t entirely quite there for me. The only reason I would still use Roblox’s Datastore over my own web server is because I can trust their servers to handle the load if my game is to have thousands of people - however with my servers I have no guarantee it can handle that much until it start breaking.

It’s one of those trial vs. already tested cases.

And personally, I still prefer MongoDB over SQL.

2 Likes

I’ve got a bad feeling about this…

9 Likes

SQL Injection waiting to happen lol

2 Likes

Not really, if your hosting it and firing it from the server why would you bother running an SQL injection attack

Sure, then it’ll become an issue but if you aren’t publicly displaying the address and port you are fine. It should only ever be written on the server side and never the client side

True, I do plan on updating the node script with better security tonight if all goes to plan

1 Like

Literally around a 30 second job to fix your SQL injection vulnerabilities.

I am aware

Welp. Pretty well, but one warning.

You could use REPLACE INTO instead of doing SELECT + INSERT / UPDATE.
https://dev.mysql.com/doc/refman/5.7/en/replace.html

In this case it would be just

} else {
        con.query("REPLACE INTO `store` (`dataKey`, `value`) VALUES ('" + key + "', '" + value + "');", function (err, result) {
            if (err) {res.end("err1")};
        });
    }

Few more tips:

  • Personally I prefer doing con.query("REPLACE INTO `store` (`dataKey`, `value`) VALUES (?, ?)", [key,value]) to prevent sql injection (it also is easier).
  • Unsure if it changes anything, but I use pool to make sure that connection isn’t closed.
  • I prefer using promises. It is more supported by Node.js by itself + better error catching. ( promise-mysql dependency ).

Oh, I didn’t know that that existed. Whenever I get a chance to update it I will