The documentation states that I can use a prefix like “dog” to return multiple keys at a time, this would work like the SQL query: SELECT ... WHERE some_column LIKE 'a%'
Which would return a queryset of each matching key with the search term, I am however unable to get this to work due to lacking documentation.
So no matter if I have keys like “dogwater”, “this key has a dog”, “ht234dog43” these keys would be returned with their respective data. That’s what I want out of this whole ordeal but I can’t get this to work no matter what I do.
I’ve tried to do the following:
local myPageObject = KDS:ListKeysAsync("flower", 20)
local function iterPageItems(pages)
return coroutine.wrap(function()
local pagenum = 1
while true do
for _, item in ipairs(pages:GetCurrentPage()) do
coroutine.yield(item, pagenum)
end
if pages.IsFinished then
break
end
pages:AdvanceToNextPageAsync()
pagenum = pagenum + 1
end
end)
end
for item, pageNo in iterPageItems(myPageObject) do
print(item.KeyName, pageNo)
end
The iterPageItems function is taken from the Roblox “pages” documentation and should work but it doesn’t.
I want to fetch multiple Instances at a time, then call the next page cursor on it.
I need this information as I am working on an IR system that will construct search indexes in the form of tokenized lexemes. I have a Porter Stemmer Algorithm for stemming each word.
What part isn’t working? Your title and content are all in disagreement of which part specifically is the problem and that makes it confusing to understand your real problem. The thread title says ListKeysAsync is broken while the content says the prefix doesn’t work and then that the iterPageItems function doesn’t work.
Strange that you find my issue hard to understand.
I am trying to use ListKeysAsync to return a queryset of all matched “flower” keys (Take a look at the SQL query I sent): KDS:ListKeysAsync("flower", 20) where “flower” is my search term and 20 is the limit per page.
So if my datastore has keys like “flowerpot” and other similar keys to “flowerpot”, I want to return all of them and iterate over that data.
The issue is hard to understand because as I mentioned before, you’re pointing out three different problems, one in the title and two in the content body, and they’re all different. It feels like either you don’t sufficiently understand the source of issue itself or don’t understand how to use ListKeysAsync.
Just so time isn’t wasted trying to figure out that part, I’ll talk about all three, I guess.
ListKeysAsync isn’t broken, it’s just the way it’s being used here. So that’s problem number one out of the way with regards to what the title is saying. If ListKeysAsync is broken, you would get an error about that in the console itself about the engine having trouble communicating with data services.
The first argument of ListKeysAsync is a prefix not a string match, so in your dictionary flower 1, flower 2, flower 3 would be returned but not this sentence has flower. Similarly for your “dog” case, only “dogwater” would be returned because it’s the only string of the 3 that starts with “dog”. This is in the documentation (which it appears you’ve already read and thus should know).
iterPageItems on the Creator Documentation is broken because of the AdvanceToNextPageAsync call which yields. You’ll need to use an alternative that does yielding outside of the iterator. I’ve posted an alternative that converts pages to a table and iterates them, see here (doubles with a documentation update request). You’re responsible for passing along the cursor in the Pages object if you need to further iterate using the cursor parameter.
So because I tried 2 different methods of getting my data out with ListKeysAsync I concluded it wasn’t working correctly. I would still like to see a working snippet of code where ListKeysAsync is used (I will try your page iterator script).
What I am trying to understand overall right now with regards to making datastore requests is how on earth would I construct custom search queries like the one I showed in the SQL example, not just for prefixes with ListKeysAsync but for finding a search term in the whole key? Is this possible? I don’t want to make 100s of separate get requests.
DataStores currently don’t support custom search queries - it’s a feature on the roadmap for the future but for now you’d be expected to iterate your entire DataStore and construct that query yourself out of the results (e.g. turning your entire DataStore key list into tables).
Hm, okay I’ll patiently wait for them to release such features in the future then. Thank you for all the persistent help and information, I appreciate these sorts of discussions on here, you’re very experienced.
Until then I’ll sadly have to use more crude methods like partitioning the entire dataset under certain keys, then fetching those keys in huge chunks and doing the querying logic in cache. It’s either that or my keys are going to be stored in some module which would eat up memory.
I used your pages script and got it to loop through although I am still wondering if ListKeysAsync is not working as intended because the pagination does not look right (my limit is 5 keys per page yet this returns 8 pages with only 2 keys in 2 of the pages):
I’ve saved these values like so:
Saving data (SetAsync) snippet.
local fake_data_import = {
["flowerpot"] = 1,
["flowerface"] = 2,
["flowermate"] = 3,
["flowerfind"] = 4,
["flowerfist"] = 5,
["flowerman"] = 6,
["flowertask"] = 7,
["flowerthrow"] = 8,
["flowercrude"] = 9,
["flowerbase"] = 10,
}
for name, points in pairs(fake_data_import) do
local success, errorMessage = pcall(function()
print(string.format(
'Importing key: %s with weights: %d',
name,
points
))
KDS:SetAsync(name, points)
end)
if success then
print("Import successful.")
else
warn(errorMessage)
end
end
Then I loop through my data with ListKeysAsync like so:
Retreiving data (ListKeysAsync) snippet.
local myPageObject = KDS:ListKeysAsync("flower", 5)
-- Reformat pages as tables
local function pagesToTable(pages)
local items = {}
while true do
table.insert(items, pages:GetCurrentPage())
if pages.IsFinished then
break
end
print("---- AdvanceToNextPageAsync ----")
pages:AdvanceToNextPageAsync()
end
return items
end
local function iterPageItems(pages)
local contents = pagesToTable(pages)
-- Track the current page number starting at 1
local pageNum = 1
-- Get last page number so we don't iterate over it
local lastPageNum = #contents
-- for will resume this coroutine until there's nothing to go through
return coroutine.wrap(function ()
-- Loop until page number is greater than last page number
while pageNum <= lastPageNum do
-- Go through all the entries of the current page
for _, item in ipairs(contents[pageNum]) do
-- Pause loop to let developer handle entry and page number
coroutine.yield(item, pageNum)
end
pageNum += 1
end
end)
end
for item, pageNo in iterPageItems(myPageObject) do
print(
string.format(
'Page: %d has key: %s',
pageNo,
item.KeyName
)
)
end
Yeah, this time it’s actually ListKeysAsync not returning the correct behaviour. Threw the code into Studio and ran it a few times myself along with a number of edits (no prefix, isolating pages to check the page content, so on) and it’s not batching keys.
print(KDS:ListKeysAsync("", 10):GetCurrentPage())
Even this is just one key on the current page. Very likely a bug you found there.
