Creating a large-scale datastore that can be indexed quickly & randomly

Hey!

I’m creating a game management system that stores a bunch of data such as bans and permission levels. The catch is that I need to be able to index and/or iterate over this data randomly even when the key (UserId) isn’t present. I’ve come up with a couple ideas, like using a non-clustered database implementation with DataStores, or just storing all data in a table under a single key. These methods all seem to have limitations where the drawbacks outweigh the benefits. Somebody had also suggested OrderedDataStores to me, but these won’t work in my situation because I want to be able to store all different types of data, not just integers.

An example use-case of why I need this is for listing off banned users.

My question put simply: How can I create an expandable database [with datastores] that I can index/iterate over without too many drawbacks?

1 Like

make the number of character for both the key and value small, and save datastores to each player’s userid, not one key
Also recommend using DataStore2

If you are looking to compress your data in order to save space in the DataStore, I would recommend using BitBuffer, BitBuffer is an easy-to-use compression module that is open-source.

https://github.com/dekkonot/bitbuffer/blob/master/src/roblox/init.lua