ListKeysAsync bugged

Bug Reports Engine Bugs
#1

I’ve found 3 bugs in ListKeysAsync.
These are SERIOUS bugs, as they prevent them from working correctly.

"SCRIPT 
local DataStoreService = game:GetService("DataStoreService")
local DatastoreOptions = Instance.new('DataStoreOptions')
DatastoreOptions.AllScopes = true
experienceStore 	= DataStoreService:GetDataStore("Jogo", "", DatastoreOptions)

for i = 1, 50 do -- create 3 keys
	experienceStore:SetAsync('global/' .. i, i*10)
	print("SetAsync key 'global/" .. i .. "'", "Value: " .. i*10)
end

function ListAllKeys(DS, Operation)
	print("\n***ListAllKeys(DS, Operation): ", DS, Operation)

	local Success, Pages = pcall(function()
		return experienceStore:ListKeysAsync(nil, nil, nil, true) -- true = excludeDelete
	end)
	while Success do
		local Itens = Pages:GetCurrentPage()
		for _, Key in ipairs(Itens) do
			if Operation == 'Remove' then
				local Success, Erro = pcall(function()
					local Result = experienceStore:RemoveAsync(Key.KeyName) 
					print('Remove', Key.KeyName)
				end)

				if not Success then
					warn(Erro)
				end
			else
				local value, keyInfo = experienceStore:GetAsync(Key.KeyName)
				print('Key.KeyName, Value', Key.KeyName, value)
			end
		end
		if Pages.IsFinished then
			break
		end

		Pages:AdvanceToNextPageAsync()
		print("Pages.Cursor", Pages.Cursor)
	end
end


print("\ndid not remove")
ListAllKeys('Jogo')
print("removed")
ListAllKeys('Jogo', 'Remove')
print("after removing")
ListAllKeys('Jogo')

1) HTTP 429

  • Where there are about 30 Pages:AdvanceToNextPageAsync(), it gets an error:

502: API Services rejected request with error. HTTP 429 (Too Many Requests)

screenshot

image
image1075×1002 63.1 KB

2) Each key being listed twice

  • Taking the output log and ordering the lines, it can be seen that each key is being listed twice:
Sorted output 
  
  
  502: API Services rejected request with error. HTTP 429 (Too Many Requests)
  Key.KeyName, Value global/1 10
  Key.KeyName, Value global/1 10
  Key.KeyName, Value global/10 100
  Key.KeyName, Value global/10 100
  Key.KeyName, Value global/11 110
  Key.KeyName, Value global/11 110
  Key.KeyName, Value global/12 120
  Key.KeyName, Value global/12 120
  Key.KeyName, Value global/13 130
  Key.KeyName, Value global/13 130
  Key.KeyName, Value global/14 140
  Key.KeyName, Value global/14 140
  Key.KeyName, Value global/15 150
  Key.KeyName, Value global/15 150
  Key.KeyName, Value global/16 160
  Key.KeyName, Value global/16 160
  Key.KeyName, Value global/17 170
  Key.KeyName, Value global/17 170
  Key.KeyName, Value global/18 180
  Key.KeyName, Value global/18 180
  Key.KeyName, Value global/19 190
  Key.KeyName, Value global/19 190
  Key.KeyName, Value global/2 20
  Key.KeyName, Value global/2 20
  Key.KeyName, Value global/20 200
  Key.KeyName, Value global/20 200
  Key.KeyName, Value global/21 210
  Key.KeyName, Value global/21 210
  Key.KeyName, Value global/22 220
  Key.KeyName, Value global/22 220
  Key.KeyName, Value global/23 230
  Key.KeyName, Value global/23 230
  Key.KeyName, Value global/24 240
  Key.KeyName, Value global/24 240
  Key.KeyName, Value global/25 250
  Key.KeyName, Value global/25 250
  Key.KeyName, Value global/26 260
  Key.KeyName, Value global/26 260
  Key.KeyName, Value global/27 270
  Key.KeyName, Value global/27 270
  Key.KeyName, Value global/28 280
  Key.KeyName, Value global/28 280
  Key.KeyName, Value global/29 290
  Key.KeyName, Value global/29 290
  Key.KeyName, Value global/3 30
  Key.KeyName, Value global/3 30
  Key.KeyName, Value global/30 300
  Key.KeyName, Value global/30 300
  Key.KeyName, Value global/31 310
  Key.KeyName, Value global/31 310
  Key.KeyName, Value global/32 320
  Key.KeyName, Value global/32 320
  Key.KeyName, Value global/33 330
  Key.KeyName, Value global/33 330
  Key.KeyName, Value global/34 340
  Key.KeyName, Value global/34 340
  Key.KeyName, Value global/35 350
  Key.KeyName, Value global/35 350
  Key.KeyName, Value global/36 360
  Key.KeyName, Value global/36 360
  Key.KeyName, Value global/37 370
  Key.KeyName, Value global/37 370
  Key.KeyName, Value global/38 380
  Key.KeyName, Value global/38 380
  Key.KeyName, Value global/39 390
  Key.KeyName, Value global/39 390
  Key.KeyName, Value global/4 40
  Key.KeyName, Value global/4 40
  Key.KeyName, Value global/40 400
  Key.KeyName, Value global/40 400
  Key.KeyName, Value global/41 410
  Key.KeyName, Value global/41 410
  Key.KeyName, Value global/42 420
  Key.KeyName, Value global/42 420
  Key.KeyName, Value global/43 430
  Key.KeyName, Value global/43 430
  Key.KeyName, Value global/44 440
  Key.KeyName, Value global/44 440
  Key.KeyName, Value global/45 450
  Key.KeyName, Value global/45 450
  Key.KeyName, Value global/46 460
  Key.KeyName, Value global/46 460
  Key.KeyName, Value global/47 470
  Key.KeyName, Value global/47 470
  Key.KeyName, Value global/48 480
  Key.KeyName, Value global/48 480
  Key.KeyName, Value global/49 490
  Key.KeyName, Value global/49 490
  Key.KeyName, Value global/5 50
  Key.KeyName, Value global/5 50
  Key.KeyName, Value global/50 500
  Key.KeyName, Value global/50 500
  Key.KeyName, Value global/6 60
  Key.KeyName, Value global/6 60
  Key.KeyName, Value global/7 70
  Key.KeyName, Value global/7 70
  Key.KeyName, Value global/8 80
  Key.KeyName, Value global/8 80
  Key.KeyName, Value global/9 90
  Key.KeyName, Value global/9 90
  Key.KeyName, Value MyScope/Key1 Value1
  Key.KeyName, Value MyScope/Key1 Value1
  Pages.Cursor 0yLeYRBLEfxQAVALP1lKEWR7krDMvCg7T9TYAwX/K5A5Iw==
  Pages.Cursor 0yLeYRBLEfxQAVALP1lKEWR7krDMvCg7T9TYAwX/K5A5Iw==
  Pages.Cursor 1w9xkP2tn/AKF8gziaZOqSIiKm3M6WBgjRoGKNbioQMxMiM=
  Pages.Cursor 1w9xkP2tn/AKF8gziaZOqSIiKm3M6WBgjRoGKNbioQMxMiM=
  Pages.Cursor 4gVMler8t8rvbRw0XO151Bq3HY1lFIMd0nBDUu87r/I3Iw==
  Pages.Cursor 4gVMler8t8rvbRw0XO151Bq3HY1lFIMd0nBDUu87r/I3Iw==
  Pages.Cursor 5yW1/2N/5D/anBgoWDYbFq4PUFiYrhZh3QrMjagjIYA0Iw==
  Pages.Cursor 5yW1/2N/5D/anBgoWDYbFq4PUFiYrhZh3QrMjagjIYA0Iw==
  Pages.Cursor b7VcHh8kM6oR2Uwi7eKlmGisrwUjMHDV9M792z7YaiM2Iw==
  Pages.Cursor b7VcHh8kM6oR2Uwi7eKlmGisrwUjMHDV9M792z7YaiM2Iw==
  Pages.Cursor GFauccwnktHwE/stcyfYHlfnsdHJrEIqgxGVYx5JPLIxNCM=
  Pages.Cursor GFauccwnktHwE/stcyfYHlfnsdHJrEIqgxGVYx5JPLIxNCM=
  Pages.Cursor HcUpDRIIx8RsGBuxRsyS4IRkiBP5dnbL2Ku850oTqpAxMyM=
  Pages.Cursor HcUpDRIIx8RsGBuxRsyS4IRkiBP5dnbL2Ku850oTqpAxMyM=
  Pages.Cursor MjCFpfeS+TSP5Qre0Q+ukTsCAGPStXqD/jTh8TKyzAUxMCM=
  Pages.Cursor MjCFpfeS+TSP5Qre0Q+ukTsCAGPStXqD/jTh8TKyzAUxMCM=
  Pages.Cursor nzrxiER/+KVBNMl9B7mbqnzqkKsBZcr9liT+cwLGGf01Iw==
  Pages.Cursor nzrxiER/+KVBNMl9B7mbqnzqkKsBZcr9liT+cwLGGf01Iw==
  Pages.Cursor o2vUnFaVyxDNhITzOgyi8VaakDo6jApixl0yjyrePKEzIw==
  Pages.Cursor o2vUnFaVyxDNhITzOgyi8VaakDo6jApixl0yjyrePKEzIw==
  Pages.Cursor OuCvVpSwkadB8ohNatbwVHe9u0SPHMvTewPgHIBOj30xIw==
  Pages.Cursor pCPc58KpYI2JheWaAzuOg6rIygpkDOwJ38fAkyQtw7MyIw==
  Pages.Cursor pCPc58KpYI2JheWaAzuOg6rIygpkDOwJ38fAkyQtw7MyIw==
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor WXRD7p3hER+YBPmJlOIDik0E+Dym8HVApGS9pxkmY1kxMSM=
  Pages.Cursor WXRD7p3hER+YBPmJlOIDik0E+Dym8HVApGS9pxkmY1kxMSM=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  rogeriodec_games committed a new version of script ServerScriptService.Script
  Script 'ServerScriptService.Script', Line 266 - function ListAllKeys
  Script 'ServerScriptService.Script', Line 272
  SetAsync key 'global/1' Value: 10
  SetAsync key 'global/10' Value: 100
  SetAsync key 'global/11' Value: 110
  SetAsync key 'global/12' Value: 120
  SetAsync key 'global/13' Value: 130
  SetAsync key 'global/14' Value: 140
  SetAsync key 'global/15' Value: 150
  SetAsync key 'global/16' Value: 160
  SetAsync key 'global/17' Value: 170
  SetAsync key 'global/18' Value: 180
  SetAsync key 'global/19' Value: 190
  SetAsync key 'global/2' Value: 20
  SetAsync key 'global/20' Value: 200
  SetAsync key 'global/21' Value: 210
  SetAsync key 'global/22' Value: 220
  SetAsync key 'global/23' Value: 230
  SetAsync key 'global/24' Value: 240
  SetAsync key 'global/25' Value: 250
  SetAsync key 'global/26' Value: 260
  SetAsync key 'global/27' Value: 270
  SetAsync key 'global/28' Value: 280
  SetAsync key 'global/29' Value: 290
  SetAsync key 'global/3' Value: 30
  SetAsync key 'global/30' Value: 300
  SetAsync key 'global/31' Value: 310
  SetAsync key 'global/32' Value: 320
  SetAsync key 'global/33' Value: 330
  SetAsync key 'global/34' Value: 340
  SetAsync key 'global/35' Value: 350
  SetAsync key 'global/36' Value: 360
  SetAsync key 'global/37' Value: 370
  SetAsync key 'global/38' Value: 380
  SetAsync key 'global/39' Value: 390
  SetAsync key 'global/4' Value: 40
  SetAsync key 'global/40' Value: 400
  SetAsync key 'global/41' Value: 410
  SetAsync key 'global/42' Value: 420
  SetAsync key 'global/43' Value: 430
  SetAsync key 'global/44' Value: 440
  SetAsync key 'global/45' Value: 450
  SetAsync key 'global/46' Value: 460
  SetAsync key 'global/47' Value: 470
  SetAsync key 'global/48' Value: 480
  SetAsync key 'global/49' Value: 490
  SetAsync key 'global/5' Value: 50
  SetAsync key 'global/50' Value: 500
  SetAsync key 'global/6' Value: 60
  SetAsync key 'global/7' Value: 70
  SetAsync key 'global/8' Value: 80
  SetAsync key 'global/9' Value: 90
  Stack Begin
  Stack End
  testes @ 22 mar 2023 19:17 auto-recovery file was created
***ListAllKeys(DS, Operation):  Jogo nil
did not remove
  • And that’s because, looking further, you’ll see that Pages.Cursor is being repeated twice.

3) Infinite loop while removing

  • To test the remove option, simply comment the for i = 1, 50 do (to avoid new creations) and the first ListAllKeys('Jogo') (to go straight to the ListAllKeys('Jogo', 'Remove') line;
  • After removing the keys, it keeps repeating two identical Pages.Cursor:
output from the REMOVE log 
   testes @ 22 mar 2023 19:30 auto-recovery file was created
  rogeriodec_games committed a new version of script ServerScriptService.Script
  removed
  
***ListAllKeys(DS, Operation):  Jogo Remove
  Remove global/3
  Remove global/36
  Remove global/43
  Pages.Cursor pCPc58KpYI2JheWaAzuOg6rIygpkDOwJ38fAkyQtw7MyIw==
  Remove global/16
  Remove global/2
  Remove global/23
  Pages.Cursor o2vUnFaVyxDNhITzOgyi8VaakDo6jApixl0yjyrePKEzIw==
  Remove global/1
  Remove global/22
  Remove global/40
  Remove global/41
  Pages.Cursor 5yW1/2N/5D/anBgoWDYbFq4PUFiYrhZh3QrMjagjIYA0Iw==
  Remove global/33
  Remove global/42
  Pages.Cursor nzrxiER/+KVBNMl9B7mbqnzqkKsBZcr9liT+cwLGGf01Iw==
  Remove global/21
  Remove global/32
  Remove global/4
  Pages.Cursor b7VcHh8kM6oR2Uwi7eKlmGisrwUjMHDV9M792z7YaiM2Iw==
  Remove global/10
  Remove global/45
  Pages.Cursor 4gVMler8t8rvbRw0XO151Bq3HY1lFIMd0nBDUu87r/I3Iw==
  Remove global/13
  Remove global/35
  Remove global/37
  Pages.Cursor 0yLeYRBLEfxQAVALP1lKEWR7krDMvCg7T9TYAwX/K5A5Iw==
  Remove MyScope/Key1
  Remove global/11
  Remove global/17
  Remove global/24
  Remove global/26
  Remove global/29
  Remove global/48
  Pages.Cursor MjCFpfeS+TSP5Qre0Q+ukTsCAGPStXqD/jTh8TKyzAUxMCM=
  Remove global/12
  Remove global/15
  Remove global/25
  Remove global/27
  Remove global/7
  Pages.Cursor WXRD7p3hER+YBPmJlOIDik0E+Dym8HVApGS9pxkmY1kxMSM=
  Remove global/20
  Remove global/30
  Remove global/44
  Remove global/9
  Pages.Cursor 1w9xkP2tn/AKF8gziaZOqSIiKm3M6WBgjRoGKNbioQMxMiM=
  Remove global/19
  Remove global/50
  Remove global/8
  Pages.Cursor HcUpDRIIx8RsGBuxRsyS4IRkiBP5dnbL2Ku850oTqpAxMyM=
  Remove global/31
  Remove global/46
  Pages.Cursor GFauccwnktHwE/stcyfYHlfnsdHJrEIqgxGVYx5JPLIxNCM=
  Remove global/18
  Remove global/28
  Remove global/34
  Remove global/47
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Remove global/14
  Remove global/38
  Remove global/39
  Remove global/6
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Remove global/49
  Remove global/5
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  Pages.Cursor PhAVb+TOhkq7qHzGBj5ABCiQidMn0EZSLliVv/z+ti8=
  Pages.Cursor zJIDy/Bg0YfoTmpDzDOJjKAyXe/mPdo8EYLUqiZrpCcxNSM=
  502: API Services rejected request with error. HTTP 429 (Too Many Requests)
  Stack Begin
  Script 'ServerScriptService.Script', Line 266 - function ListAllKeys
  Script 'ServerScriptService.Script', Line 274
  Stack End

I think it’s easy to replicate this error and identify the problem.

11 Likes
New "excludeDeleted" parameter for "ListKeysAsync" doesn't work
#2

HTTP 429 also happens to me

This is my code, it worked fine before march 23 (or 24?), now it won’t break the loop, IsFinished doesn’t work (maybe i did something wrong?)

Code 
function GetEntireList(name, pgsc, mpgs)
	local data = DataStoreService:GetDataStore(name)
	if data == nil then
		return nil
	end
	local ls, pages = pcall(function()
		return data:ListKeysAsync(nil, pgsc, nil, true)
	end)
	if ls ~= true or pages == nil then
		return nil
	end
	local tb, pagec = nil, 1
	while true do
		if mpgs ~= nil and pagec >= mpgs then
			break
		end
		for _, info in ipairs(pages:GetCurrentPage()) do
			local key = info.KeyName
			if key == nil or key == "" then
				continue
			end
			if tb == nil then
				tb = {}
			end
			tb[key] = data:GetAsync(key)
		end
		if pages.IsFinished == true then
			break
		end
		local s, err
		while true do
			s, err = pcall(function()
				pages:AdvanceToNextPageAsync()
			end)
			if s == true then
				break
			end
			task.wait(1)
		end
		pagec += 1
	end
	return tb
end
2 Likes
#3

I also found that DataStore:ListVersionsAsync() has similar issues to bug 1 and 3 reported here.

DataStoreVersionPages.IsFinished is never true despite having no more pages left and DataStoreVersionPages:AdvanceToNextPageAsync() just stays on the same page if it is the last page which together results in indefinite loops followed by an eventual HTTP 429 error.

Minimal example that demonstrates the above:

local DataStore = game:GetService("DataStoreService")
local store = DataStore:GetDataStore("test")

local key = os.time()

store:SetAsync(key, 1)

local versions = store:ListVersionsAsync(key)

while not versions.IsFinished do
    print "advancing"
    versions:AdvanceToNextPageAsync()
end

I am aware that this isn’t exactly relevant to the original bug report but I cannot create my own bug reports for some reason and this is a very serious issue that needs addressing.

4 Likes
#4

Can corroborate all the previous posts here. Our implementation of traversing ListVersionsAsync has been infinitely yielding since around the 20th of March.

For those looking for a quick patch, setting up a hash of Versions/KeyNames previously seen and ending the loop once you see a duplicate entry should do the trick.

Hoping that this can be fixed shortly.

4 Likes
#5

I’ve been receiving this issue for the past few days, making a database system of mines inoperable. It uses ListKeysAsync and I keep receiving the same HTTP 429 issue as the original poster shared.

1 Like
#6

Everything said by the OP is true. It’s definitely bugged, and I won’t be able to continue my WIP game while experiencing these errors.

1 Like
#7

Thanks for the report! We’ve filed a ticket to our internal database and we’ll keep you updated!

5 Likes
#8

The issue seems to be fixed, I no longer receive HTTP 429 errors!

#9

Testing the OP’s script again, it looks like everything is working normally now. Thanks.

(However, once again I ask the Roblox team to NOTIFY that an error has already been fixed, within the forum thread that originated the request.
Not warning is NEGLECT and lack of consideration for developers).

#10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

#11
#12

Started appearing again. Still using same script

adfbdfdf
adfbdfdf720×177 54.3 KB

1 Like
#13

I also have the same issue and i’ve contacted roblox support, i will give them the necessary details to help them fix the issue. I’m currently creating a inspiration version of RetroStudio’s but the datastore seems to be the biggest thing im having issues with, i’m switching to API’s if the issue doesn’t work as it seems that is the last option.