Im working on a voxel engine that uses chunks, pretty much just minecraft. I store all the chunk data on the server and i replicate that data to the client. The issue is that the chunk data is really big, the chunk size is 16, 255, 16 with each block in the chunk having a {} attached to it for block data.
What the chunk data looks like:
local chunks = {
[Chunk_Position] = { --> this is what im sending
[Block Position] = {
Name = "BlockName",
Orientation = BlockOrientation,
Tint = TintColor
}
}
}
Currently im converting the chunk data into a json and converting the json into a buffer and then compressing that buffer, but its still allot of data being sent.
Putting that whole table into JSON will waste a ton of bandwidth; it’s not exactly a good format for sending large amounts of binary data. Your approach should be to have an algorithm to generate the buffer yourself.
First, you should look to having a minimal data structure for representing a block. Name should probably be serialized as a number representing the block (which then gets converted back using a table), Orientation I’m guessing is a Vector3? That you can’t do much with, but if it’s just axis-aligned rotation, then just encode it 0-5: one for each direction it could face. Tint can be put into 24/32 bits, RGBA.
From there, you can come up with a way to represent a chunk as an (sparse) array of blocks. This would be creating a data structure that defines the start and end index of a series of blocks, follow it with all of the blocks in that series. If Block Position is not a numerical type, that’s a bit more difficult. Ideally you’d find a way to encode the position into a one-dimensional array so you don’t have to worry about that.
converting a JSON string into a buffer doesnt do anything because they are just the exact same uncompressed data
I think what minecraft does is that it only sends the blocks that are on the surface. For example, if you have a flat chunk with grass on top and stone underneath, you only have to send the stone blocks once the player mines down, not when they just want the grass blocks
-- Pack a chunk into a buffer
function Buffers.PackChunk(ChunkId: number): buffer
-- Get the chunk
local chunk: {[Vector3]: {[number]: any}} = EnvironmentConfig.Chunks[ChunkId]
if not chunk then return buffer.create(0) end
-- Voxels module
local Voxels = EnvironmentConfig.VoxelsModule
-- Count surface voxels
local surface_count: number = 0
for voxel_position: Vector3 in pairs(chunk) do
if Voxels.IsSurface(ChunkId, voxel_position) then
surface_count += 1
end
end
-- Incase a chunk if fully covered
if surface_count == 0 then
return buffer.create(0)
end
-- Buffer variables
local chunk_buffer: buffer = buffer.create(surface_count * 5)
local offset: number = 0
-- Write data to the chunk buffer
for voxel_position: Vector3, voxel_data: {[number]: any} in pairs(chunk) do
if Voxels.IsSurface(ChunkId, voxel_position) then
buffer.writeu8(chunk_buffer, offset, voxel_position.X)
buffer.writeu8(chunk_buffer, offset + 1, voxel_position.Y)
buffer.writeu8(chunk_buffer, offset + 2, voxel_position.Z)
buffer.writeu8(chunk_buffer, offset + 3, voxel_data[1]) -- Voxel id
buffer.writeu8(chunk_buffer, offset + 4, voxel_data[2]) -- Voxel health
offset += 5
end
end
return chunk_buffer
end
I used both idea’s and its allot better now. Went from ~50kb to ~4kb