Map/Server System Limited By Roblox's Datastores

Help and Feedback Scripting Support
, ,
1

Hello :waving_hand:

For the past few weeks, me and one of my friends have been building upon an update for my game “backrooms drift”.

This update would initially include a “Roblox Charts” Map Viewer, with public and private servers in each custom player map. Players can build their maps with parts and models provided, add thumbnails to their “game”, custom names etc, you get the gist of it.

Now at first, we did not originally think of the limitations of this, and built upon it through testing with only 1 - 3 people creating maps, which is where the problem grew worse.

Unfortunately, we had released it. And with this came the horrid cost of players loosing their data due to the maps filling up datastore queues and slowing it right down due to the large amounts of maps being created by many people.

So in short, here is what we are trying to achieve/have achieved:

We need a way to store player maps.
Each of these maps will just be the information about the map. Likes, Visits, Rating, ...

We would need to display and sort all of these maps by different categories like: Top Liked, Top Playing, ...

The problem is that we need a way to get and sort all of the maps that exist. (the main reason of the whole problem, sorting)

These maps will also display live player counts (updated each 5-10m)

As well as this, I personally do not want to use an external datastore (like firebase, which in fact we have tried to use). But, if that is to be the only solution, then we would like a heads up of what is best and if their are any open source wrappers for these external databases etc.

1 Like
2

Do you have analytics or stats on the average size of your data? Have you used any serialization or compression methods? How are you storing your keys and values into datastore?

I’ll assume you’ve already made sure you have only saved the necessary data. If your necessary data is large, I’d look into serializing player data, especially for the maps.

3

the problem isnt necesarily about storing the players maps but getting every one of the maps from all the players and having to sort the data.

2 Likes
4

Your game is limited by Roblox datastore limitations when attempting to fetch data. If the problem isn’t storing the player maps and the problem is getting the maps and sorting, then where does the root problem stem from?

Why is the title X but your problem is Y?

5

I apologise about the title

i did talk about this being the problem, however that aside.

The root problem is getting all the maps to then sort later since the maps are not stored in one spot that is easily accessible.

The maps are stored in their own player key. We need a way to store and get each map to use for sorting without external databases. (if possible ofcourse)

We’ve tried multiple solutions such as batching maps together like pages but it did not seem to be the best solution.

1 Like
6

If your game is fetching more than the datastore limitations, you have an issue with how you’re storing your data. Sure, you could extend to a DB with no limitation, why not? But that leads to an issue of scalability: If your game blows up to 1-100k CCU, then you’ll end up upgrading that storage size eventually.

So it’s more efficient to write a strong, efficient, and scalable datastructure, THEN store it. It would make more sense if you have a collection of maps in a key and post it to the datastore, that way you’re only requesting once instead of X amount.

You then, can migrate the data from player keys over to the map key.

7

Then you’d just migrate the maps to one easily accessible key in your datastore / DB and that should fix the limitation issue + sorting, since you all have to do is insert it into the blob and it should already be ready for sort: whether that’s in-game or in the DB.

8

how exactly can we store all of these maps and get them all for sorting later on?

were talking long term here, how exactly are we going to store all the maps under the 4mb limit, you cant.

What if the game blows up again and reaches 100k ccu and millions of maps are made.

1 Like
9

here, to help you a little more so you understand what exactly we want to achieve, the game Create A Cart Ride does perfectly what we want.

1 Like
10

You would have to start serializing your data to get it compressed enough to fit under the 4 MB limitation. There’s options to extend map keys (map_key1, map_key2), etc. if you prefer to abstract your keys for any reason.

If serialization isn’t an option for you, you’d have to make cuts on your data, or extend to an external DB. If you feel like the investment for an external isn’t good enough, then I would send a ticket to Roblox to request for limitation changes, which has been done for some of the studios that I developed for.

I know you said your final option would be external, but you can always look for other options of storage, such as a web server with a blob db / pointer db, rather than dedicated DB services like MongoDB and Firebase.