How to receive messages in Parallel

Help and Feedback Scripting Support
#1

I am parallelizing unzipping of tables to take up less time.

I am using a messaging system where the main script sends messages to the unzippers, and they send one back while the main script listens for the messages. This works quite well but for some reason the return messages don’t get received until all the unzippers have finished their parallel work. This leads to an unnecessary time waste. It would work better if the messages went through while it is in parallel.

I tried using an event to transmit the data, but they don’t work in parallel, and serializing the code didn’t work out. Using a shared table did resolve the problem, but putting the massive tables into the shared table took more time than sending the chunks back using messages and putting them into a normal table.

image
image1363×537 219 KB

You can see from the micro profiler that the receiving section happens after the entire unzipping part is done. I want the messages to be received during the unzipping. (When the messages are sent)

You can see it more clearly in the output, where all the numbers are printed at once, and then they are received all together, rather than being more spread out.

image
image535×650 56.1 KB

ServerCode

game.ReplicatedStorage.GetChunks.OnServerInvoke = function(Player,ChunksWanted)
	debug.profilebegin("GetToUnzip")
	table.clear(Unzipped)
	local ToUnzip={}
	for i,column in pairs(ChunksWanted) do
		for o,chunk in pairs(column) do
			local ServerChunkX=math.ceil(i/SmallChunks)
			local ServerChunkZ=math.ceil(o/SmallChunks)
			
			if not ToUnzip[ServerChunkX] then
				ToUnzip[ServerChunkX]={}
			end
			ToUnzip[ServerChunkX][ServerChunkZ]="TU"
		end
	end
	debug.profileend()
	
	debug.profilebegin("Send Messages")
	local Messages=0
	for i,column in pairs(ToUnzip) do
		for o,chunk in pairs(column) do
			Messages+=1
			Actors[CurrentActor]:SendMessage("Unzip",world[i][o],i,o)

			CurrentActor+=1
			if CurrentActor>#Actors then
				CurrentActor=1
			end
		end
	end
	debug.profileend()
	
	local MessageReturns=0
	script.Parent:BindToMessageParallel("ReturningChunk",function(Chunk,posX,posZ,num)
		print("Recived "..num)
		debug.profilebegin("SettingChunk")
		MessageReturns+=1
		if not Unzipped[posX] then
			Unzipped[posX]={}
		end
		Unzipped[posX][posZ]=Chunk
		debug.profileend()
	end)
	task.wait()
	
	local ToRender={}
	for i,column in pairs(ChunksWanted) do
		for o,chunk in pairs(column) do
			debug.profilebegin("Getting chunk "..i..","..o)
			local ServerChunkX=math.ceil(i/SmallChunks)
			local InsideChunkX=i-(ServerChunkX-1)*SmallChunks
			local ServerChunkZ=math.ceil(o/SmallChunks)
			local InsideChunkZ=o-(ServerChunkZ-1)*SmallChunks
			local UnzippedChunk=Unzipped[ServerChunkX][ServerChunkZ][InsideChunkX][InsideChunkZ]
			if not ToRender[i] then
				ToRender[i]={}
			end
			
			ToRender[i][o]=UnzippedChunk
			debug.profileend()
		end
	end
	debug.profilebegin("Sending Render")
	return ToRender
end

Parallel Code

local http=game:GetService("HttpService")
script.Parent:BindToMessageParallel("Unzip",function(Chunk,PositionX,PositionZ)
	local UnzippedChunk=http:JSONDecode(Chunk)
	local num=math.random(1,100)
	game.ServerScriptService.Actor:SendMessage("ReturningChunk",UnzippedChunk,PositionX,PositionZ,num)
	print(num)
end)

I used 12 unzippers in this test (the small code above)
image

To reiterate. How do I get the messages to go through during the parallel processing, and if not, what is a suitable work around?
Any Response is appreciated!

I also have more information on this in my code review thread about this: