Word Checking is too slow

Hiya there!

I intend to make a scrabble type game with special tile types to work with and passive abilities.

However, I ran onto an issue where my word checking function is insanely slow whenever the tile of the type “*” is found. The idea of the * tile is to work as any letter in the alphabet.

So what I tried so far was to loop through all possibilities with each * you can find in the word, and check each word in the dictionary API I use (https://dictionaryapi.dev/), but checking too quickly returns an error for the http request, thus sometimes skipping existing words.

The estimated minimum delay I found for the checker was 1/12 * 5 attemps.

I wanted to know if there was a work-around this issue, or a way to make the API a lot faster

local httpser = game:GetService("HttpService")

local letters = { -- numbers = scrabble's letter amount
	{"a", 16},
	{"b", 4},
	{"c", 6},
	{"d", 8},
	{"e", 24},
	{"f", 4},
	{"g", 5},
	{"h", 5},
	{"i", 13},
	{"j", 2},
	{"k", 2},
	{"l", 7},
	{"m", 6},
	{"n", 13},
	{"o", 15},
	{"p", 4},
	{"q", 2},
	{"r", 13},
	{"s", 10},
	{"t", 15},
	{"u", 7},
	{"v", 3},
	{"w", 4},
	{"x", 2},
	{"y", 4},
	{"z", 2},
}

local total = 0
for i, v in pairs(letters) do
	total += v[2]
end

local function roll(req: number) -- amount of letters to draw (max of 7)
	local returnedLetters = {}
	for i = 1, req do
		local num = math.random(1, total)
		for j, w in ipairs(letters) do
			num -= w[2]
			if num <= 0 then
				table.insert(returnedLetters, w[1])
				break
			end
		end
	end
	return returnedLetters
end

local function isreal(word: string) -- checks the word
	local url = "https://api.dictionaryapi.dev/api/v2/entries/en/"..word -- api I use
	local success = nil
	local errorMsg = nil
	local attempt = 0
	repeat
		attempt += 1
		success, errorMsg = pcall(function()
			local res = httpser:GetAsync(url)
			local form = httpser:JSONDecode(res)
		end)
		if not success and attempt < 5 then
			task.wait(1/12) -- min delay with 5 attemps that successfully return first existing word from the possibilities everytime
		end
	until success or attempt == 5
	return success
end

local function returnStars(word: string)
	local input = word
	local output, count = input:gsub("*", "*")
	return count
end

local function AllZ(tab) -- ends the while loop if all possibilities have been checked
	local allz = true
	for i, v in pairs(tab) do
		if v ~= "z" then
			allz = false
			break
		end
	end
	return allz
end

local function checkWord(word: string)
	local start = os.clock() -- debugging
	local stars = returnStars(word)
	if stars == 0 then
		if isreal(word) then -- if there are no *, just check the word once
			return true
		end
	else
		local wordCut = string.split(word, "*")
		local newword = word
		local cur = 1
		local starCUR = {}
		for i = 1, stars do
			table.insert(starCUR, "a") -- get a table for each *
		end
		while not AllZ(starCUR) do -- the following ~10 lines will just increase the table's letter position by 1 everytime to check all possible variations
			while starCUR[cur] == "z" and starCUR[cur + 1] do
				starCUR[cur] = "a"
				cur += 1
			end
			for i, v in ipairs(letters) do
				if starCUR[cur] == v[1] then
					if starCUR[cur] ~= "z" then
						starCUR[cur] = letters[i + 1][1]
					end
					break
				end
			end
			newword = ""
			for i, v in ipairs(starCUR) do
				newword = newword..wordCut[i]..v
			end
			newword = newword..wordCut[#wordCut]
			if isreal(newword) then
				print(os.clock() - start) -- time it took to find the word
				print(newword) -- debugging
				return true
			end
			cur = 1
		end
		return false -- no existing word found, return false
	end
end

print(checkWord("*a*e")) -- check for the string *a*e if for any letter replacing

for example in the following image, within the prompt ae, it took 13 seconds to find the word “babe”, so imagine for prompts like “ja**y”.

From my perspective every wildcard "*" increases the letter options by 26. Therefore, several wildcards can create hundreds or thousands of words (e.g. *a*e = 676). Your program verifies each one by sending HTTP requests to a dictionary API. However, sending too many too fast can lead to rate limits or failures. Use of delays (task.wait(1/12)) help avoid this issue, but greatly slows down the entire process.

Not really any way to speed up your requests. External service with single word lookup. Worst case will always be searching every letter for every wildcard. You will be better off finding another API (this seems to be designed to get the definition and stuff rather than to check if the word exists).

As an alternative I know you can download a list of all English words that you may be able to import into your game and then filter through it smartly yourself. There are enough constraints that if you can load these into a table or something you can quickly find only the relevant portions to search and do this locally making it potentially way faster.

If you want to use your current solution though there is little to be done.

Alright so according to your responses I have decided to modify the “*” tile, it will now grant a prompt to the player asking which word is the intended word, thus limiting the word checking requests to 1.

Thank you for helping me.

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.