How can I have a lot of global leaderboards without exceeding datastore limit?

Help and Feedback Scripting Support
, , , ,
#1

1. What do you want to achieve?

I am developing a game with a lot of leaderboards (for each map there’s a leaderboard). The team decided to have a lot of maps (100+). But it doesn’t sound possible according to the datastore limit for GetSortedAsync()

2. What is the issue?

You can see the limit right there:

image
image892×191 19.8 KB

With GetSortedAsync() only being able to execute that many requests, I can’t really get that many leaderboards on the game.

3. What solutions have you tried so far?

I’ve tried to implement my own sorting methods. Or putting them into cache when they’re sorted and updates it accordingly based on the number of maps existing to meet the requirements. However, I ran into a few problems:

  • I have absolutely no idea how to make a custom GetSortedAsync() function
  • Even though the 2nd idea could work, there’s no way I could have everything initialized upon server creation.
  • With a lot of maps it can cause a lot of delays, and could just get even worse (10 minutes per update)

And yes, I did look on the developer hub but I can’t find any useful information regarding this problem.
Thank you for helping me, I greatly appreciate it. :smiley:

1 Like
#2

You can try having 1 script for getting the data and you can access the data from the leaderboards using CollectionService. Having 100+ scripts that get the data is very bad practice. ( I haven’t used datastores a lot so this solution might not work )

#3

What exactly do you mean by using CollectionService for getting data? And no I’m not getting 100+ scripts for retrieving data. My problem is that I will break the datastore limit for GetSortedAsync() if I have too many leaderboards. (I can’t combine all of the leaderboards into one as OrderedDataStore can’t sort dictionary values)

#4

You can just make bunch of OrderedDataStore’s unique DataStore name for each one.

#5

Using collectionservice to put data in the leaderboards in just 1 script is what I mean.

#6

You could try to send a request every minute with all the new data received in that minute. So instead getting and updating every single time something is changed, you do it all at once every minute. It could lead to minimal data losses, but that should be fixable too by updating just before the server shuts down

#7

I’m not really getting the idea here, you can put data together using CollectionService?

#8

No, you can put all of the leaderboards in a table ( which is what CollectionService is for ) and loop through them to avoid getting a script for each leaderboard.

#9

That wouldn’t be a problem if I only have one OrderedDataStore. In this case however, I have multiple OrderedDataStore and calling them all at once would most likely result in the requests being overloaded

#10

Do you have multiple leaderboards requesting for different data? Such as one for kill count, rounds won…

#11

May I know what the maps exactly are? Are they supposed to be completed by every player and how long does it take to complete the player? (I’m guessing the leaderboard you’re making is how many times a map is completed)

#12

No not like that. My game is like kind of a minigame style, it’s just I want to put a leaderboard for people times on each minigame map. And I don’t think it’s possible to handle all of these with only one OrderedDataStore (maps may have different times, beside, I can only store a numerical value, having a dictionary to indicate the minigame’s name/type won’t work)

#13

You could request for the data only when the map is loaded. Since the limit scales for the amount of players you have, you could add maps when you can afford it.

#14

Basically, for every map there’s a leaderboard to show the player’s completed time on the map.

#15

Well, in the game you’re supposed to be able to see all leaderboards without a restriction, and only getting data when a map is loaded could mean that the player can never check the times unless it’s loaded in-game. (The reason being I implemented a replay system like in Parkour). You can think of each map as a time trial in Parkour (Each time trial has their own unique leaderboard)

#16

And how does the lobby work? Do you see every single minigame in the lobby? If not, what you could do is load leaderboards only for maps that are being used and ignore unused minigames, that way you’ll save many, many requests
(You just answered it so ignore this)

#17

Even though this solution kinda sucks, my only solution would be to use up the 5 in 5 + players * 2 requests a minute and update each leaderboard every 20 minutes (if my math isn’t wrong). The reason you wanna only use up 5 requests is in case a a request fails and you need that extra 2/4/6/8/… requests a minute.
You want to have a delay of 12 seconds between updating each leaderboard (so update leaderboard 1, wait 12 seconds, update leaderboard2, wait 12 seconds, and so on). Yes, this does mean that each leaderboard will only get updated 3 times an hour, but it’s the only thing I can think of right now. You could also go a bit riskier and use the equation 5 + numPlayers * 2 to change wait times (60 / resultOfEquation), which will leader to quicker loading, but that would be riskier - if something somehow fails (because roblox is acting stupid on a day or something), that map wouldn’t be updated for a way longer time than usual. I know you didn’t really want this but honestly it might be the only way to go.

#18

I know that this way will definitely work, I am just trying to find better solutions. This is probably the last resort and I’m sure I can accomplish that. After all despite having a lot of time trials (in Parkour), I think they all get updated frequently enough, and I’m trying to figure that out right now.

My best lead right now is to try to cache the leaderboards and use a custom sorting algorithm for a custom GetSortedAsync(). For example, taking 100 players and sort them while only displaying 25 players, which would make it somewhat accurate assuming that people near to the top 25 is more likely to get to top 25. However, this still have a problem of the maps not being updated frequently enough, overtime the leaderboard will go out of sync. (I just realized if the leaderboard doesn’t get updated sorting 100 players is a waste of resource)

#19

Is it necessary for you to get the leaderboard data for all map at the same time? Is it not in your idea that you only get one leaderboard and only show one leaderboard at a time?

#20

You could make one leaderboard and when a player wants to see data for a map, they could request for it and every player would be able to do 1 request each minute. If the data was already asked for by the other servers that minute, ( which you can check with messaging service ) you can give that data and the 5 extra requests would be in case it failed. Not every player would actually ask for data or ask for the same data as others, so you would have a lot of extra requests.