How can I use datastores as well as external databases for an in-game live marketplace?

Help and Feedback Scripting Support
1

Hey guys! I’ve been working on a future game of mine which will be an open-world exploration game, the game will be having a large map spreading over 50+ places, and I wanted to implement a live in-game marketplace where people could buy other’s stuff as well as sell their stuff(like ores and ships). This marketplace will only be accessible in 6 places with a large player capacity.

I thought a lot about this and I came up with a few ideas of how I could implement this, but they have their own limitations and hence I need help to solve this, here are the ideas and their limits-

1- to Have an external database in my own house(Raspberry Pi 4B 4 GB RAM with an SSD), all the information of what needs to be sold will be stored in it, and one server will get this information every 60 seconds and write it into a Roblox datastore so that other servers and places can access this datastore. Whenever a player starts to sell stuff, that data will directly be written to the external database
limitations - I’ve heard that Roblox takes some time to write data onto the datastores, this could lead to a slowdown of the refresh of the market in other servers. Moreover, there are limitations of Roblox’s datastores, hence I’m worried.

2- Almost similar to above, but all the servers will directly request data from the external database, Roblox’s datastores will be ruled out here
limitations - since I don’t have much hardware or capital to support 20+ servers(in the future) to request information, this could be too much for what I plan to use - a Raspberry Pi 4B 4 GB RAM variant. (The reason I cannot upgrade to better hardware is that I’m 16 and still living with my parents)

Can I kindly get help on this issue, I would love to get more ideas from wonderful developers like you! :slightly_smiling_face:
Thanks,
loldude0912

(I’m sorry if I’ve written this badly, it’s my first topic on Dev Forum!)

1 Like
2

I’m confused why you need an external database here. With proper implementation, Roblox’s datastores can work perfectly here.

If I understand correctly what you’re trying to do is create a user-run store (kind of) like a town square marketplace.

Since Roblox’s datastores have a 6-second save limit, you can retrieve the information from people buying and selling things and store it in a server-sided table, to be autosaved every 6 seconds. In the meantime, complete the action for the player so they don’t experience any delays.

Using an external database seems to just be complicating things here.

3

As far as I know, filtering data is harder by using Roblox’s datastores. It’s much simple by using a RDBMS like MySQL along with PHP or JS, hence it’s easier for me as I’m familiar with these languages. Moreover, Roblox has limits on how many datastore requests we can make which is 60+NumberOfPlayers*10 for both reading and writing, while an external Database wouldn’t have any such limit. That’s why I’m reluctant to rely only on Roblox’s datastores.

In fact, rather than using complicated system for filtering or just retrieving data will take more computational work than just retrieving it from an external database.

4

Isn’t Lua only able to read JSON data? I don’t know that much about external databases but I’m pretty sure this is one limitation.

5

Don’t worry about sending and receiving data, I can handle that along with encoding the data into JSON on the external server before sending. The main problem here is how will I get the data from backend to frontend.

1 Like
6

Forgive me for being a little slow at this :laughing: but maybe I can clear some things up for you (and vice versa):

Theoretically it can be more difficult, however, you can separate info with a certain key in tables, and save those tables to datastores. I may not know how your marketplace is/will be set up, but here’s some sample code:

local mainTable = {
    ["Player_123456789"] = "Item-Sword;Price-250"
    -- etc.
}
-- From here you can separate the string into readable data like so:
local str = mainTable["Player_123456789"]
local table1 = string.split(str, ";")
print(table1[1], table1[2])
-- outputs "Item-Sword" and "Price-250"
local table2 = string.split(table1[1], "-")
print(table2[1], table2[2])
-- outputs "Item" and "Sword".
-- The "Item" would be discarded, and now you have the name of your item.
-- Simply repeat to get the price

This info can then be sent to the client’s UI using RemoteEvents.

This is per minute, not per game session. Assuming you have 20 people playing, that’s 260 requests per minute. If you’re still worried, you can cache the results per-server every 10 seconds (with an optional “refresh” button):

local tempdata
local data
local function refresh()
    data = store:GetAsync("Data")    
end

game.ReplicatedStorage.Request.OnServerEvent:Connect(function(plr)
    game.ReplicatedStorage.Request:FireClient(plr, data)
end)

game.ReplicatedStorage.Refresh.OnServerEvent:Connect(function(plr)
    refresh()
    game.ReplicatedStorage.Refresh:FireClient(plr, data)
end)

while wait(10) do
    refresh()
end

This code probably won’t work off-the-bat but it gives you an idea.

Probably, but the difference is small.

Am I missing anything?

1 Like
7

Ooh, nice idea you’ve got there, although what I’ll be storing will be way complex, this idea is interesting. I’ll do some brainstorming for implementing this kind of system!
But there’s also another problem-Get/SetAsync will cache its return to save computation times. This could lead to a big slow down as the datastore in other servers won’t get updated frequently. And apart from the market, I also need to store inventories, position, and money of the players along with some more data.

yea I know that it’s per minute, but I will already use a lot of it for storing and retrieving other data like I told earlier.

It would be far easier for the server to just get it from an external server rather than decoding and recoding the data into what you suggested earlier. Since the game will be very heavy, adding this recoding and decoding of data from datastores will further burden the server and slow it down, resulting into chaos(probably).

1 Like
8

It only has a cache of 4 seconds. Again, I don’t know your setup, but is 4 seconds a huge slow-down, especially if you’re only retrieving every 6-10 seconds?

I just ran some tests to confirm my suspicions: If the data is saved/retrieved from different stores, it won’t have any delays or hit the limit. So save the inventory in a different DataStore then the position, position different from money, etc.

It’s always up to you, but I do recommend just using Roblox’s datastores.

Have you considered teleporting the players to a marketplace place while buying things, to reduce lag?

9

possibly yes, each server where the market will be available will hold 50 players max,hence I wouldn’t take the risk, But alright, I’m planning a system based entirely on Roblox’s datastores.

It will take time to teleport them, hence, the feature won’t be available instantly. This will be a downside if the game needs a ‘premium-feeling’…you know what I mean.

1 Like