ReadVoxels() not Working?

Help and Feedback Scripting Support
,
#1

I’m having a problem with ReadVoxels. When I generate a region3, it’s (presumably) positioned in the correct location. The code reads the terrain to be stored in the datastore, but even though the example piece is positioned within the terrain, it always returns the result as if all the materials in the table were “Air.” Any suggestions?

Code:

local RESOLUTION = 8
local MATERIAL_TO_INT = {}
local INT_TO_MATERIAL = {}
local DataStore = game:GetService("DataStoreService"):GetDataStore("Voxels")

local Size = Vector3.new(10,10,10)
local Position = Vector3.new(220,-100,50)
local ChunkSize = 400
local materials = Enum.Material:GetEnumItems()

for i, material in ipairs(materials) do
	MATERIAL_TO_INT[material] = i
	INT_TO_MATERIAL[i] = material
end

local function saveTerrain(position, ss, chunkSize)
	local min = position - (ss/2)
	local max = position + (ss/2)
	local TIME = tick()
	local key = {}
	for x = min.X, max.X - 1, chunkSize do
		for y = min.Y, max.Y - 1, chunkSize do
			for z =min.Z, max.Z - 1, chunkSize do
				local chunkMin = Vector3.new(x, y, z)
				local chunkMax = Vector3.new(math.min(x + chunkSize, max.X), math.min(y + chunkSize, max.Y), math.min(z + chunkSize, max.Z))
				print(chunkMin,chunkMax)
				local region = Region3.new(chunkMin, chunkMax):ExpandToGrid(RESOLUTION)
				local PArtRegion =Instance.new("Part",workspace)
				PArtRegion.Position =region.CFrame.Position
				PArtRegion.Size = region.Size
				PArtRegion.Anchored = true
				PArtRegion.Transparency = .5
				local materials, occupancies = workspace.Terrain:ReadVoxels(region, RESOLUTION)
				local size:Vector3 = materials.Size
				local chunkData = buffer.create(RESOLUTION * size.x * size.y * size.z)
				local chunkOffset = 0
				local Can = false
				for i = 1, size.X do
					for j = 1, size.Y do
						for k = 1, size.Z do
							local occupancy:number = occupancies[i][j][k] * 255
							local material:number = MATERIAL_TO_INT[materials[i][j][k]]
							if material ~= MATERIAL_TO_INT[Enum.Material.Air] then
								Can = true
							end
							print(materials[i][j][k])
							buffer.writeu8(chunkData, chunkOffset, occupancy)
							buffer.writeu8(chunkData, chunkOffset + 1, material)
							chunkOffset += 2
						end
					end
				end
				if Can then
					local keys ="Chunk_" .. x .. "_" .. y .. "_" .. z
					print(keys)
					key[keys]={chunkData,tostring(size)}
				end
				task.wait(0)
			end
		end
	end
	DataStore:SetAsync("TERRAIN",key)
	print("Saved in:",tick()-TIME.."s")
end
1 Like
#2

I know that what causes this “error” is due to my resolution, any ideas to counteract it?

#3

why not use 4 instead of 8? aaa

#4

Using 4 to read 8000, 5000, 8000 of land takes almost 1 minute, whereas using 16 (which is what I use) only takes 8 seconds. Do you have any idea how to make the resolution adjust to that?

#5

image
image934×314 34 KB

It says “must be 4”

#6

If you could provide context as to what you are doing with ReadVoxels, it would be helpful to me to know what you are really looking for.

Depending on what you are doing, advanced optimization techniques include using parallel processing, native code generation, and type checking.

If you post your code in #help-and-feedback:code-review , other developers can help you optimize and reach the performance level you need.

1 Like
#7

As noted, you have to use 4. Roblox never updated smooth terrain to have a multiresolution voxel format. Passing other values is fast because you’re not reading any real data, it’s just returning a block of default values of empty space. Reading and Writing big chunks of terrain is pretty slow, you have to just accept that.

2 Likes
#8

Okay. I didn’t manage to apply parallel, and even after reading more than eight tutorials, I still don’t understand how to apply it. Any ideas about parallel, or do you think it’s too far off topic? What I want is to save the terrain to the cloud (which it doesn’t do anymore, but I can’t correctly add the position parameter due to the resolution).

#9

Any ideas to make the for fast with resolution in 4?

#10

There’s no way to speed up the ReadVoxels or WriteVoxels functions themselves, for a given chunk size, because it’s already native code. Actual generation of terrain and code in nested loops that reads and writes materials and occupancies can often be sped up just by the usual code optimization practices of not wastefully creating and destroying data structures, not redundantly computing loop-invariant values, etc. For some types of generation, if there is a lot of algorithm work in actual Lua code (not calls to C++ APIs) then the --!native directive may give you some speedups where it’s supported (server, plugins).

Doing large terrain generation without hanging studio usually involves breaking up the task into chucks, and yielding between them (or just periodically in general) to keep Studio responsive. Using Parallel Lua (see the API Docs for examples) can also speed up terrain generation calculations (but not the WriteVoxels, since the terrain voxel data does not support parallel writes at this time).

Lastly, it’s worth noting that some terrain generation functions like FillBlock() can be way faster than using WriteVoxels, because it’s all done in C++ code that’s optimized for setting the same value in blocks of memory. So if you can write your code to take advantage of FillBlock to fill regions that have the same material and occupancy, and just use WriteVoxels for the rest, it could end up faster overall.

2 Likes
#11

If you want to save the chunk to the cloud, I would recommend creating maybe 24 Actors and then using SendMessage to tell each Actor which chunk of the terrain to read (reading can be done in parallel). Each actor should read the terrain, and compact the position, occupancy, and material into a buffer.

Finally, you need to send the final buffers to the original script.
One way I can think to do that is to switch to serial and set shared or _G values.

To see the final save data length, do

local http = game:GetService("HttpService")
print(http:JSONEncode(final_buffer):len())

By the way, you call task.desyncronize() to switch to a parallel core and task.syncronize() to switch back to serial, or main core. Within a single Actor, it is safe to assume you only switch between two cores. The only way to have more than 2 cores is to have more actors. Actors are like containers for code.

Note that buffers are the best practice for saving large amounts of data.

1 Like
#12

As @EmilyBendsSpace said, WriteVoxels does not support parallel execution and will use up the primary core. That said, you could still do parallel work while WriteVoxels is executing. Additionally, using task.wait() at certain points will help keep the main core responsive, as it handles everything important.

1 Like
#13

To compact numbers you can use string.pack. Here’s an example:

local http = game:GetService("HttpService")

local function compact(pos : Vector3int16, occu : number, material : EnumItem) : string
	local data = string.pack("i2i2i2i3i2",
		pos.X, pos.Y, pos.Z,
		math.floor(occu * 2^23-1 + 0.5),
		material.Value
	)
	return data
end

local function unpack(data : string)
	local posX, posY, posZ, occu, material = string.unpack("i2i2i2i3i2", data)
	local pos = Vector3int16.new(posX, posY, posZ)
	occu /= 2^23-1
	material = Enum.Material:FromValue(material)
	return pos, occu, material
end

local pos = Vector3int16.new(142,21234,-4123)
local occu = 0.5612
local material = Enum.Material.DiamondPlate

local data = compact(pos, occu, material)

-- the data's size per entry is 11 bytes since 2+2+2+3+2 = 11
local bfr = buffer.create(11)
buffer.writestring(bfr, 0, data, data:len())

local encode = http:JSONEncode(bfr)

print(data:len(), data) -- the string's length
print(encode:len(), encode:sub(1,50)) -- the actual serialized string that is saved.

-- decoding:

local decode_bfr = http:JSONDecode(encode)
local data = buffer.readstring(decode_bfr, 0, 11)

print(unpack(data))
1 Like
#14

My new system.

-Server code

--!native 
--!nocheck
local ServerStorage = game:GetService("ServerStorage")
local RESOLUTION = 4
local MATERIAL_TO_INT = {}
local INT_TO_MATERIAL = {}
local DataStore = game:GetService("DataStoreService"):GetDataStore("Voxels")

local materials = Enum.Material:GetEnumItems()

for i, material in ipairs(materials) do
	MATERIAL_TO_INT[material] = i
	INT_TO_MATERIAL[i] = material
end

local function saveTerrain(position, ss, chunkSize)
	local min = position - (ss/2)
	local max = position + (ss/2)
	local TIME = tick()
	local key = {}
	local Shared = SharedTable.new()
	local results = {}
	local Actores={}
	for x = min.X, max.X, chunkSize do
		for y = min.Y, max.Y, chunkSize do
			for z = min.Z, max.Z, chunkSize do
				local actor = ServerStorage.TerrainProcessor:Clone()
				actor.Parent = workspace
				table.insert(Actores,actor)
				task.wait()
				actor:SendMessage("processChunk",x, y, z, chunkSize, RESOLUTION,Shared)
			end
		end
	end
	local BR = true
	task.desynchronize()
	while BR do
		for i, a:Actor in pairs(Actores) do
			if not a:GetAttribute("Can") then
				BR = false
			end
		end
	end
	for i,a in Shared do
		key[i]={buffer.fromstring(a[1]),a[2]}
	end
	task.synchronize()
	print("Calculated in: ",tick()-TIME.."s")
	DataStore:SetAsync("TERRAIN", key)
	print("Saved in:", tick() - TIME .. "s")
end

-Actor Code

--!native
local MATERIAL_TO_INT = {}
local INT_TO_MATERIAL = {}
local materials = Enum.Material:GetEnumItems()
for i, material in ipairs(materials) do
	MATERIAL_TO_INT[material] = i
	INT_TO_MATERIAL[i] = material
end

local Actor = script:GetActor()
function Generate(x, y, z, chunkSize, RESOLUTION,Shared:SharedTable)
--	warn("Procesando chunk en paralelo")
	local PartData = {}
	local chunkMin = Vector3.new(x, y, z)
	local chunkMax = chunkMin + Vector3.new(chunkSize, chunkSize, chunkSize)
	local region = Region3.new(chunkMin, chunkMax)
	local materials, occupancies = workspace.Terrain:ReadVoxels(region, RESOLUTION)
	local size = materials.Size
	local chunkData = buffer.create(RESOLUTION * size.x * size.y * size.z)
	local chunkOffset = 0
	local CAN = false
	for i = 1, size.X do
		for j = 1, size.Y do
			for k = 1, size.Z do
				local occupancy = occupancies[i][j][k] * 255
				local material = MATERIAL_TO_INT[materials[i][j][k]]
				if material ~= 36 then
					CAN = true
				end
				buffer.writeu8(chunkData, chunkOffset, occupancy)
				buffer.writeu8(chunkData, chunkOffset + 1, material)
				chunkOffset += 2
			end
		end
	end
	if CAN then
		local Key = x..","..y..","..z
		Shared[Key] = {buffer.tostring(chunkData),tostring(size)}
	end
	task.synchronize()
	script.Parent:SetAttribute("Can",true)
end
Actor:BindToMessageParallel("processChunk",Generate)

Am I implementing Parallel Luau correctly? It could be better for me, because I have a piece with a size of 2048, 1000, and 2048, and it takes about 17 seconds. Any recommendations?

#15

Thank you very much, I understood your recommendations, I think the problem here is how Roblox allows you to read the terrain.