Need help with optimizing my NES picture processor emulator

You can write your topic however you want, but you need to answer these questions:

  1. What do you want to achieve? Keep it simple and clear!
    I need help with optimizing my PPU rendering engine,
    I am trying to make ROBLOX’s first NES emulator, but it runs at 33 FPS and that’s without the CPU emulator running, and I think that the PPU script could be better optimized
  2. What is the issue? Include screenshots / videos if possible!
    the PPU emulator runs very slow
  3. What solutions have you tried so far? Did you look for solutions on the Creator Hub?
    i tried deleting everything inside workspace and making the character invisible and anchored to and turned off canQuery, canCollide, and canTouch, to stop collision scripts from running

This code runs every frame

a snippet of code from the PPU which I think directly impacts the framerate

tilePallet[1] = RAM.getPpuAdress(0x3f00)
	local bgColorPallet = pallet[RAM.getPpuAdress(0x3f00)]
	local vramTileHigh,vramTileLow = {},{}
	local val1
	local val2
	local pxY = 0
	local ppuTileIndxY = 0
	if ppuCtrl[5] == 1 then
		for y = 1, 240 do -- tile drawing (works fine)
			pxY += 1
			if pxY > 8 then
				pxY = 1
				ppuTileIndxY += 1
			end
			local pxX = 0
			local ppuTileIndxX = 0
			for x=1, 256 do
				pxX += 1
				if pxX > 8 then
					pxX = 1
					ppuTileIndxX += 1
				end
				local tileIndx = RAM.getPpuAdress(0x2000 + (ppuTileIndxY * 32) + ppuTileIndxX)
				local shift = 8 - pxX
				local lowByte = RAM.getPpuAdress((tileIndx*16)+pxY)
				local highByte = RAM.getPpuAdress((tileIndx*16)+pxY+8)
				local low = bit32.band(bit32.rshift(lowByte,shift), 1)
				local high = bit32.band(bit32.rshift(highByte,shift), 1)
				
				local colorIndex = low + high * 2
				local colorBytes = pallet[RAM.getPpuAdress(0x3F00+colorIndex)]
				local base = ((y - 1) * 256 + (x - 1)) * 4
				if ppuCtrl[8] == 0 then
					local indx = table.find(pallet,colorBytes)
					if indx then
						if tonumber(indx) > 47 then
							colorBytes = pallet[32]
						elseif tonumber(indx) > 15 then
							colorBytes = pallet[16]
						else
							colorBytes = pallet[1]
						end
					end
				end
				if ppuCtrl[7] == 0 and x < 9 then
					buffer.writeu8(bytes, base, bgColorPallet[1])
					buffer.writeu8(bytes, base + 1, bgColorPallet[2])
					buffer.writeu8(bytes, base + 2, bgColorPallet[3])
					buffer.writeu8(bytes, base + 3, 255)
				else
					buffer.writeu8(bytes, base, colorBytes[1])
					buffer.writeu8(bytes, base + 1, colorBytes[2])
					buffer.writeu8(bytes, base + 2, colorBytes[3])
					buffer.writeu8(bytes, base + 3, 255)
				end
			end
		end
	else
		for i=0,rendSize - 1 do
			local base = (i * 4)
			buffer.writeu8(bytes, base, bgColorPallet[1])
			buffer.writeu8(bytes, base + 1, bgColorPallet[3])
			buffer.writeu8(bytes, base + 2, bgColorPallet[2])
			buffer.writeu8(bytes, base + 3, 255)
		end
	end
	local spriteInLine = {}
	if ppuCtrl[4] == 1 then
		for i=0,63 do
			local spriteRootData = i*4
			local spriteAnchorY = OAM[spriteRootData + 0]
			local att = OAM[spriteRootData + 2]
			local spriteAnchorX = OAM[spriteRootData + 3]
			local tileRoot = 0x1000 + (OAM[spriteRootData + 1] * 16)
			print(tileRoot)
			local pxY = 0
			for y=spriteAnchorY,spriteAnchorY+7 do
				pxY += 1
				if spriteInLine[y] then
					spriteInLine[y] += 1
					if spriteInLine[y] > 8 then
						continue
					end
				else
					spriteInLine[y] = 1
				end
				local pxX = 0
				for x=spriteAnchorX,spriteAnchorX+7 do
					pxX += 1
					if ppuCtrl[6] == 0 and x < 9 then
						continue -- stops the sprite from drawing near the leftmost 8 pixels if ppumask 6 is disabled
					end
					local mirrorX,mirrorY = 0,0
					local ctrlBytes = ctrl.readInp(1)
					
					if ctrlBytes == "11111111" then
						mirrorX = 0
						mirrorY = 0
					else
						mirrorX = 1
						mirrorY = 0
					end
					local shift = 8 - pxX
					local lowByte
					local highByte
					local row = pxY
					if mirrorY == 1 then
						row = 9 - pxY
					end
					lowByte = RAM.getPpuAdress(tileRoot + row)
					highByte = RAM.getPpuAdress(tileRoot + row + 8)
					
					if mirrorX == 1 then
						lowByte = CPU.mirrorByte(lowByte)
						highByte = CPU.mirrorByte(highByte)
					end
					if x >= 1 and x <= 256 and y >= 1 and y <= 240 then
						local base = ((y - 1) * 256 + (x - 1)) * 4
						local low = bit32.band(bit32.rshift(lowByte, shift), 1)
						local high = bit32.band(bit32.rshift(highByte, shift), 1)
						local spriteColorIndex = low + high * 2
						if spriteColorIndex ~= 0x00 then
							local colorBytes = spritePallet[spriteColorIndex + 1]
							buffer.writeu8(bytes, base, colorBytes[1])
							buffer.writeu8(bytes, base + 1, colorBytes[3])
							buffer.writeu8(bytes, base + 2, colorBytes[2])
							buffer.writeu8(bytes, base + 3, 255)
						end
					end
				end
			end
		end
	end
	img:WritePixelsBuffer(
		Vector2.new(0, 0),
		Vector2.new(256, 240),
		bytes
	)

Please do not ask people to write entire scripts or design entire systems for you. If you can’t answer the three questions above, you should probably pick a different category.

This code looks like it could heavily benefit from native codgen. It’d also be worth compiling with O2.

1 Like

Probably won’t optimize much, never hurts to try, but I cleaned it up a little. Little less codelines.

cleaned up some repeats and unused code I could see. Ya may need to add some stuff back in if this all isn’t full code.

function colorer(bytes,base,pallet)
    buffer.writeu8(bytes, base, pallet[1])
	buffer.writeu8(bytes, base + 1, pallet[3])
	buffer.writeu8(bytes, base + 2, pallet[2])
	buffer.writeu8(bytes, base + 3, 255)
end

tilePallet[1] = RAM.getPpuAdress(0x3f00)
	local bgColorPallet = pallet[RAM.getPpuAdress(0x3f00)]
	local vramTileHigh,vramTileLow = {},{}
	local val1
	local val2
	local pxY = 0
	local ppuTileIndxY = 0
	if ppuCtrl[5] == 1 then
		for y = 1, 240 do -- tile drawing (works fine)
			pxY += 1
			if pxY > 8 then
				pxY = 1
				ppuTileIndxY += 1
			end
			local pxX = 0
			local ppuTileIndxX = 0
			for x=1, 256 do
				pxX += 1
				if pxX > 8 then
					pxX = 1
					ppuTileIndxX += 1
				end
				local tileIndx = RAM.getPpuAdress(0x2000 + (ppuTileIndxY * 32) + ppuTileIndxX)
				local shift = 8 - pxX
				local lowByte = RAM.getPpuAdress((tileIndx*16)+pxY)
				local highByte = RAM.getPpuAdress((tileIndx*16)+pxY+8)
				local low = bit32.band(bit32.rshift(lowByte,shift), 1)
				local high = bit32.band(bit32.rshift(highByte,shift), 1)
				
				local colorIndex = low + high * 2
				local colorBytes = pallet[RAM.getPpuAdress(0x3F00+colorIndex)]
				local base = ((y - 1) * 256 + (x - 1)) * 4
				if ppuCtrl[8] == 0 then
					local indx = table.find(pallet,colorBytes)
					if indx then
						if tonumber(indx) > 47 then
							colorBytes = pallet[32]
						elseif tonumber(indx) > 15 then
							colorBytes = pallet[16]
						else
							colorBytes = pallet[1]
						end
					end
				end
                local ppuCtrlandX = ppuCtrl[7] == 0 and x < 9
                colorer(bytes,base,ppuCtrlandX and bgColorPallet or colorBytes)
			end
		end
	else
		for i=0,rendSize - 1 do
			local base = (i * 4)
            colorer(bytes,base,bgColorPallet)
		end
	end
	local spriteInLine = {}
	if ppuCtrl[4] == 1 then
		for i=0,63 do
			local spriteRootData = i*4
			local spriteAnchorY = OAM[spriteRootData + 0]
			local att = OAM[spriteRootData + 2]
			local spriteAnchorX = OAM[spriteRootData + 3]
			local tileRoot = 0x1000 + (OAM[spriteRootData + 1] * 16)
			print(tileRoot)
			local pxY = 0
			for y=spriteAnchorY,spriteAnchorY+7 do
				pxY += 1
				if spriteInLine[y] then
					spriteInLine[y] += 1
					if spriteInLine[y] > 8 then
						continue
					end
				else
					spriteInLine[y] = 1
				end
				local pxX = 0
				for x=spriteAnchorX,spriteAnchorX+7 do
					pxX += 1
					if ppuCtrl[6] == 0 and x < 9 then
						continue -- stops the sprite from drawing near the leftmost 8 pixels if ppumask 6 is disabled
					end
					local mirrorX,mirrorY = 0,0
					local ctrlBytes = ctrl.readInp(1)
                    local allones = (ctrlBytes == "11111111")

					mirrorX = allones and 0 or 1

					local shift = 8 - pxX
					local lowByte
					local highByte
					local row = pxY

					lowByte = RAM.getPpuAdress(tileRoot + row)
					highByte = RAM.getPpuAdress(tileRoot + row + 8)
					
					if mirrorX == 1 then
						lowByte = CPU.mirrorByte(lowByte)
						highByte = CPU.mirrorByte(highByte)
					end

					if x >= 1 and x <= 256 and y >= 1 and y <= 240 then
						local base = ((y - 1) * 256 + (x - 1)) * 4
						local low = bit32.band(bit32.rshift(lowByte, shift), 1)
						local high = bit32.band(bit32.rshift(highByte, shift), 1)
						local spriteColorIndex = low + high * 2
						if spriteColorIndex ~= 0x00 then
							local colorBytes = spritePallet[spriteColorIndex + 1]
                            colorer(bytes,base,colorBytes)
						end
					end
				end
			end
		end
	end
	img:WritePixelsBuffer(
		Vector2.new(0, 0),
		Vector2.new(256, 240),
		bytes
	)

um im sorry if i seem naive but what is 02 and code gen?

Compiler optimization level 2 and native machine code generation respectively. They are tuning options for the Luau compiler.

These are complex concepts, so I am not going to act like I can explain them properly over a devforum post. I already linked to documentation, read that first, and then feel free to do your own research.

1 Like

you should explore some open source Roblox NES emulator source like LuauNES since it is pretty fast

Another option for PPU is change from dot based to scanline-based rendering (lose accuracy inexchange)

Also from ur snippet, there seem to be alot of bad code optimizations there like:

  • Using table.find? I meant, table.find is super slow, like O(n) slow
  • tonumber is a bad practice to use on hot-loops
  • Don’t create tables on these loops, it could be a better way like reuse them instead?
  • Instead of storing three separate 8-bit RGB values in bgColorPalette, why not use a single 32-bit number, so you can just call 1 writeu32 instead of 4 writeu8.
1 Like

oh okay, thank you i put this into consideration and try and implement them

1 Like