I am trying to create a system that saves a lot of items in the datastore and they get displayed in a list. The items need to be cross-server accessible so every players sees the same list. Every player can save a limited amount of items to the datastore (150). The issue is that I cant get the items from the datastore to display for the player as I would need to do many getasync calls and it would exceed the getasync call limit roblox has for the datastore (which is 60 + 10 per player).
I want for example to get every item there is in the datastore and sort them out in a loop to order them in popularity and then display the most popular items in the list for the player.
I have thought about GetOrderedDataStore to sort them in popularity but then I would have to save items double as I also want to sort them in for example newest items. I have also come across to use an external datastore to host myself but Im not sure if that’s the best way to do this.
Why don’t you just save the items in a JSON under one key? One key has limit of 4MB. You can then just load the data, and sort it in your code.
If there will be too many items, you can kind of shard it into multiple keys.
I think this is the best approach, since I’m used it that way for player data as well.
It can add up to many items as every player can upload 150 which adds up quickly exceeding the 4MB limit. Saving them under multiple keys (like per userId) is also a problem because I want to get all the items there are which has the issue that I need to do a lot of getAsync calls which is also has a limit (certain amount per minute).
I don’t quite understand your planned structure.
What do you actually plan to do? How to events are viewed, how they can add items, and such.
There’s no need to save it under userId…
Well what Im am planning to do is making it possible for players to save their outfits to a list of outfits for others to be able to see/wear them. So it would make sense to save it under the userId (so it is easy to collect all the outfits a player added).
And then you can either move through the list of all outfits and find the outfits that have the same userId as the user. Or your user will have its own key outfits with json array [12569, 12570, 13888, 14785].
To make it more optimal, you can calculate how many outfits will fit into one 4MB key, for example 200. So you’ll structure your datastore with multiple keys for outfits with id 0 ~ 199, 200 ~ 399, 400 ~ 599, etc..
When your user outfit array contains for example id 12569, you will calculate 12569 // 200 = 62, you’re gonna load outfit shard 62 and then cache it, so you don’t have to load it from datastore once again different user has outfit from same shard.
From real point of view, one outfit JSON will have max of 200 characters, it’s 20k different outfits into one shard, so you can change the 200 to something like 10_000. As many outfits in one-shard, the less you have to load and cache the shards in the future.
Hm, so from what I understand what you are saying is to calculate the data one outfit takes and know approximately how many outfits can be saved into one datastore key? so you say 20k outfits in one key? this way would work until there are like 2 million outfits I guess.
