Amazon Storage Server
I made an Amazon server located in Ohio to store game data across universes. It uses the Dynamodb structure and caches get requests in RAM for some time and updates the caches from posts for a couple minutes.
What can it do?
Currently it can:
- handle 300000 cached requests per second,
- return cached data in sub 1ms,
- get data from Dynamodb in 37ms on average,
- do posix multithreaded processing,
- store complicated Lua tables and arrays without userdata.
Why should you use it?
Because with this, you;
- can easily save data across games and universes,
- don’t have to worry about error handling and server errors,
- can handle all of your data receiving and sending needs with 2 simple functions.
Amazon.rbxl (17.5 KB)
This is a place that comes with an Amazon wrapper module in ServerStorage and a TestScript in ServerScriptService. Run the game and the game will load data and then save data to the Amazon servers. You’re free to use the simple wrapper however you like.
Scripts
--Test script inside ServerScriptService
local Amazon = require(game.ServerStorage.Amazon)
local UserId = math.floor(tick()/120) --Get an user id valid for ~2 minutes
local Data = Amazon.LoadPlayerData(UserId) --Immediately load player data of the user id
local PlayerData = {}
if Data[1] and Data[1]==591 then --No data code
print"no data"
PlayerData.Points = 1
else
PlayerData = Data --There is data and we can add it
end
print("Points:",PlayerData.Points) --Current data value
PlayerData.Points = PlayerData.Points + 1 --Increase it by 1
Amazon.SavePlayerData(UserId,PlayerData,true) --Add it to save queue, the 3rd argument is importance
Amazon.Step() --Make a save step and actually send the data to server
--(since importance was set to true, step here does nothing)
--Wrapper module inside ServerStorage
local module = {}
local requrl = "http://18.217.51.64:9002/"
local get = "getdata/"
local post = "postdata/"
module.AuthKey = "Game1"
local http = game:GetService("HttpService")
module.AutoSave = true --True or False
module.Verbose = true
local fwarn,fprint=warn,print
local function print(...)
if not module.Verbose then return end
return fprint(...)
end
local function warn(...)
if not module.Verbose then return end
return fwarn(...)
end
local SaveRequests = {}
module.SavePlayerData = function(UserId,Data,Important) --Number, Table, Bool
assert(UserId,"UserId can't be nil",2)
assert(Data,"Data can't be nil",2)
assert(typeof(UserId)=="number","UserId needs to be a number",2)
assert(typeof(Data)=="table","Data needs to be a table",2)
Data.UserId = UserId
module.SaveRaw(Data)
if Important then
module.Step()
end
end
module.SaveRaw = function(Table)
Table._ltik=tick()
table.insert(SaveRequests, Table)
end
module.LoadPlayerData = function(UserId) --Number
assert(UserId,"UserId can't be nil",2)
assert(typeof(UserId)=="number","UserId needs to be a number",2)
local res,body = module.GetRaw{Body={saves={{key=UserId}}}}
return body.saves[1],res
end
module.GetRaw = function(Table) --Run for when player joins to get their data
assert(Table,"Request can't be nil",2)
local url = requrl..get..(Table.AuthKey and Table.AuthKey or module.AuthKey)
local method = "POST"
local headers = {}
headers["Content-Type"]="application/json"
local body = Table.Body and http:JSONEncode(Table.Body) or "{}"
local res = http:RequestAsync{
Url=url,
Method=method,
Headers=headers,
Body=body
}
if res.StatusCode<=299 then
body = http:JSONDecode(res.Body)
else
warn("Req failed:", res.StatusCode, res.StatusMessage)
end
return res,body
end
module.Step = function()
if #SaveRequests>0 then
local reqSave = {}
for i,tab in pairs(SaveRequests) do
reqSave[i]=tab
end
table.sort(reqSave, function(a,b) return a._ltik<b._ltik end)
SaveRequests={}
local res,suc
repeat
local t = tick()
suc,res = pcall(function() return http:RequestAsync{
Url=requrl..post..module.AuthKey,
Method="POST",
Headers={["Content-Type"]="application/json"},
Body=http:JSONEncode{
saves=reqSave
}
}
end)
print("time http taken to save: " .. tick()-t)
if not suc then warn(res) end
until res and (res.StatusCode<=299)
local immediateRedo
local body = http:JSONDecode(res.Body)
if body.saves then
for i = 1,#reqSave do
if body.saves[i] and typeof(body.saves[i])=="table" then
if not body.saves[i][1] or body.saves[i][1]>299 then
warn("error while saving code",body.saves[i][1])
warn("error while saving reason",body.saves[i][2])
table.insert(SaveRequests,reqSave[i])
immediateRedo=true
else
warn("save success")
end
else
warn("not saved",body.saves[i])
table.insert(SaveRequests,reqSave[i])
immediateRedo=true
end
end
else
immediateRedo=true
for i = 1,#reqSave do
table.insert(SaveRequests,reqSave[i])
end
if body[1] and typeof(body[1])=="number" then
warn("save global error:",body[1],body[2])
else
warn("unknown error:",res.StatusCode)
end
end
if immediateRedo then
module.Step()
end
end
end
game.Players.PlayerRemoving:Connect(function()
if module.AutoSave then
ypcall(module.Step)
end
end)
coroutine.wrap(function()
while wait(3) do
if module.AutoSave then
ypcall(module.Step)
end
end
end)()
return module
Amazon Module API
void module.SavePlayerData(number UserId, table Data, bool Importance)
Will add your Data to a queue. If Importance is set to true, it’ll call Step function right after and immediately save the data. The given Data table can have any complexity that’s valid for JSON in it.
table Data module.LoadPlayerData(number UserId)
Yields. This function takes an UserId number and returns Data that was saved. If there was an error, it’ll return {ErrorCode, ErrorReason} as a response.
void module.Step()
Yields. An important function which does the actual saving to the server. When you call SavePlayerData as not important, it adds the data to a queue and actually saves and sends the data to the Amazon Server when it’s called. This means your non-critical data should be set as unimportant and be waited for more data to be sent together.
Every request to the server, strains the server so it’s a better idea to send a whole bunch and let it solve it in parallel. Every request also takes away from the currency of the key, exhausting it.
module.AuthKey = "Game1"
This is the key field of the module. This is either the public key, or private key given to you by me. It represents the name of the table your data is getting saved to. The public keys are public and can be edited and read by anyone.
module.AutoSave = true
This field does a step call whenever someone leaves the game and every 3 seconds, if you don’t want to worry about step function calls yourself.
module.Verbose = true
Boolean field prints and warns everything for debugging.
18.217.51.64:9002 is the current IP:Port address of the server. It will be transferred to a domain later.
For now, I’m going to keep it in an open beta state to collect some errors while you (devforum) uses it freely after which I’ll offer the service for some price. I would like you to test its robustness with the public key “Game1”. I might wipe out the table if I think it’s being abused so don’t put any sensitive information in to the public key and if you would like to use it for some kind of production release, send me a message through devforums and I’ll give you a free upto 200 MB data private key to use for yourself.
In the future, I will add ordered data stores and leaderboard systems but before that, I need to see if the current system works as robust as possible.
The system is always subject to change and I update it pretty much daily. Your public key data can and will probably get lost while I close and open the server. This won’t be a problem as I will be doing load balancing type Amazon server array where updates will be pushed slowly and will be handled in parallel for validity checking the updates.