Like I know that usually you have to do the :GetAsync(key) but is there a way to just read all of them, without the keys?
If not, is there another solution?
–
edit: I’m thinking I could maybe just use an ordered data store. My data isn’t numerical or something that needs to be ordered but OrderedDataStores have the pages thing
edit2: never mind the values need to be positive integers
edit3: what if I just make the datastore a single key and value, with the value as a giant array with all the values
edit4: edit: I’d also like to note that I forgot the pretty important info that I’m using this for making an application center
Use a combination of a datastore with a numerical key, and an ordered datastore of just that key.
Whenever you write to the database, you also pop your key in the ordered datastore.
When you want to read everything, you first read all the pages of the ordered datastore to get all your keys back.
Then you can just walk your original datastore because you know all the keys.
Bonus tips:
use key and time as your key/value pair in the ordered store. You have to use something and this way you get a free chronological sort
whenever doing any datastore operations you should wrapper them behind checks to the current remaining budget and a wait(). Roblox is kind enough to try and throttle you and give you a buffer but its really an error to get any warnings back from the datastore service at all.
If you need this at scale, I would recommend an external server where you send a message whenever you touch a key, and then that external server saves a key-value pair of (key, timestamp) in a database.
You could do the same with ordered datastores, but reading from ordered datastores is quite slow at scale because you are limited to a budget of 5 requests per minute for at most 100 entries at a time, so you can only fetch 500 entries a minute. And then, if you have a constant high volume of data being moved, your results would get mangled because data is being changed before you are finished reading it.
The ordered datastore approach would work well at a low scale though, i.e. for a clan/group place it would probably be fine, but not for a production game intended for the front pages.
Obviously defer to buildthomas on this one, he knows more about datastores than anyone
eg: I use this method for tracking private servers because the total number of unique keys is pretty low, and for some ad hoc analytics because I dont care that the data is out of date when I get it.
To clarify what I meant: suppose a key is initially far back into the ordered datastore, but it is updated while you are reading it, then that key will go to the front in the results. As such, all other keys are pushed one entry further in the ordering. You would be missing one of the keys in your final result, and there would be one duplicate key. (and progressively worse if this happens multiple times while you are reading keys, which will totally happen when used for a production game)
So it’s not so much about the content of the key, but rather that you would be missing/duplicating keys.
You wouldn’t have this problem with your own external server since you can just fetch all of the keys in a few ms, and in a consistent way such that there are no duplicates or missing entries.
Oh! I had not even considered that the sort would be changing while you were iterating through pages! Thats actually horrible.
I need to think about this more.
Hmm so, it’d always work as long as your num keys is less than a page, which is very limited.
It’d also work for slightly larger sets of keys if your number of keys was stable eg: analytics records like num hats purchased, num visitors.
It’d also work if your value wasn’t time.
Interesting things to consider but it should be pretty obvious that this is a stickytape and wishes set up. You can do it but it does have some big old asterixes on it.
This doesn’t work reliably, because datastores don’t offer strong consistency. If there are more than 2 servers running that are concurrently writing to this key, you will probably lose entries at some point because one server will be overwriting the other if they are writing at nearly the same time, without the ability to read the updates of the other server (there is no datastore mechanism for this, you are reliant on the internal caching mechanisms that datastores perform).
Moreover, at scale (which I still don’t know if we care about here), you would probably quickly hit the size limit for the key when you have a high throughput of players. If a key takes 10 characters, you can only store approximately 25,000 keys, which isn’t all that much in the grand scheme of things.
so I’m thinking I should maybe try something where instead of adding a person’s key to a data store I could instead just use a google spreadsheet to store keys and check whether or not someone’s key exists Link
Application center… that’s where I’m stuck, chief. I’m not a big fan of saving all data to a single table though that sounds easier. Perhaps you could save a key as “Applicants” which would contain a table of UserIds for those who have applied, while a UserId key would contain application data.
I’m not well-versed with this kind of stuff. I use external services for applications. When I’m desperate, Trello works as either an actual application panel or a place to archive application data so I can fetch cards in a list if a reviewer clicks “fetch applications” or “refresh”.
Sorry for opening a 2018 topic but have you tried GetSortedAsync() and GetCurrentPage()? You can split your datastore in pages and read it with no keys
