Is there any way to reduce lag when creating a very large table?

Help and Feedback Scripting Support
, , , ,
#1

  1. What do you want to achieve?
    I would like to find a way to speed up the code so that I can run it without causing a lot of lag. I am creating a table for making a chunk but it needs to set around 2 * 64 * 513 * 64 values (~4.2 mil).

  2. What is the issue?
    The script causes around half a second of lag each time this part of code runs. I have narrowed it down specifically to this part of the code and would prefer that it goes faster because I need to run this function at most a few times per second.

  3. What solutions have you tried so far?
    I honestly can’t think of anything to try since I can’t lower the scale or yScale values.

  • note: scale = 64, yScale = 256
  • it sets every single value in 2 64 * 513 * 64 tables within the ctab (“chunk table”) table
  • ctab[1] is for materials and ctab[2] is for occupancy
  • getMat() checks if cy < height and returns either grass or air (this is a placeholder)
	-- Create table for Terrain:WriteVoxels
	-- [1] = materials, [2] = occupancy
	local ctab = {{}, {}}
	for cx = 1, scale do
		for cz = 1, scale do
			-- Get Height
			local height = htab[cx][cz]
			for cy = -yScale, yScale do
				-- Generate material
				-- note: getMat() does not cause lag, it's a placeholder which
				-- just checks if the material should be air or grass
				local mat = getMat(cy, height)
				
				if not ctab[1][cx] then
					ctab[1][cx] = {}
				end
				
				if not ctab[1][cx][cy+yScale+1] then
					ctab[1][cx][cy+yScale+1] = {}
				end
				
				
				ctab[1][cx][cy+yScale+1][cz] = mat
				
				-- Generate occupancy
				-- note: this doesn't actually work properly but I'm trying to
				-- fix the lag caused by the script first.
				local val = 0
				if cy < height then
					val = math.clamp(height-cy+yScale*2, 0, 1)
				end
				
				if not ctab[2][cx] then
					ctab[2][cx] = {}
				end
				if not ctab[2][cx][cy+yScale+1] then
					ctab[2][cx][cy+yScale+1] = {}
				end
				ctab[2][cx][cy+yScale+1][cz] = val
			end
		end
	end
#2

The first thing you should do is optimize the code. Since everything is being ran in a tight loop, you’ve really gotta make sure it’s running as fast as possible. There’s quite a lot of things that you can optimize, such as by aliasing specific portions of the table so that you don’t have to index for them every single time.

Example: Instead of doing ctab[1][cx] every single time you want to access it, you can alias it to a variable first

local ctab1cx = ctab[1][cx] --use this variable

The same goes with indexing for cy+yScale+1, it is extremely redundant to add up the values to index for it every single time

...
if not ctab[1][cx][cy+yScale+1] then --you could save this table call and use it directly
	ctab[1][cx][cy+yScale+1] = {}
end
...

Another note would be the portion of the code that replaces nil values in the table with empty tables. You could possibly use metatables for that but I’m not sure if that would be more performant, you’ll have to do your own testing.

--THIS IS JUST AN EXAMPLE
setmetatable(ctab[1], {
  __index = function(self, k)
    rawset(self, k, {})
    return self[k]
  end
})

Optimization is all about testing and benchmarking so good luck :slightly_smiling_face:

Summary: There’s a lot of redundancy in your code that can easily be optimized to get better performance. After doing this you should focus on the general logic of your code and see if there’s anything else that you could optimize; perhaps why does the table need to be so damn big and why does each value need to be set with a separate table?