How to save parts, and the idea of Serialization

Hey, could you expand on this topic?
I’m trying to make a world-saving system, got random generation set up, every part is stored in a separate folder based on type, and its cloned from the preset part with a unique name.
The problem is I still don’t know how to save them efficiently, your module works fine but the number of blocks will quickly fill the memory and I have no idea how to segregate them by name.

Its a 2d mining game of sorts.

1 Like

Hey! So, the serializing I’m doing in the article will probably not be good in your case, since as you already said, you all the parts in seperate folder, which will help in the long run and in our scenario, and you have a preset part for each type of blocks. If I were to quote myself:

Note that, for games that have the same type of objects being used again and again (e.g. a building game, where you just have 3 (or more) types of building parts, like brick, dirt and plastic) you can simply save a table containing table where the key is each object’s name, and that table will hold a long list of Vector3 s that indicate where each block of that certain type is placed. And when unloading, loop through the table, and each time just :Clone() the original object of the current type (assuming it’s stored in ServerStorage or ReplicatedStorage ) and position it at the current position. This is better than having to save properties and extra info we don’t need. Of course this depends on the structure of your game, you might need to store additional info such as color maybe.

A better serializing technique would be like this, when saving, have a table, where each key is the type of the block, and each key holds a table with a lot of serialized positions. When you load the data back, loop through each key, for each key you’re gonna be looping, copying and placing the corresponding preset part for each position. This is a good strategy.

local toSave = {
    Dirt = {loop through all dirt blocks, put the position of each one here},
    Stone = {loop through all dirt stone put the position of each one here},
    ...
}

As for the positions, this is a 2D game, all you need to save is an X and a Y, which could simply be a string containing two numbers seperated by a comma or something else. "x,y".

So, efficiency does matter in your game. When you save something, that something gets JSONEncoded, so the length of the generated JSON string is the limit. Recently, the limit was increased to 1 MB, a character is 1 byte, meaning you can save approxiametly a million character long string. I doubt the game has more than 10 tousand blocks, and for each block you’re gonna need to save a 6 or 7 characters long string, which is its position, you have the comma, and I assume the Y and X directions will go hundreds of blocks in that direction, that’s 7. 10000 * 7, that’s 70000, we’re not even close to the limit. Of course there is more overhead like the [ and , found in the JSON string, let’s just say another extra 20000. Again, these are just my expectations, you can count and see for yourself, if you find issues with hitting the limit, then you would need to do some form of serialization.

3 Likes

So rewriting the module is my only hope.

At first, I thought that was causing stutter while loading blocks.

I guess learning a bit about tables won’t hurt because i don’t know what should i do with this.

1 Like

Ended up with this, it does work well but maybe you have an idea on further optimization.

local tableToSave = {
	Dirt = {},
	Stone = {},
	HardenedStone = {}
}

SaveTriggerEvent.OnServerEvent:Connect(function()
	
	for _,v in pairs(BlocksFolder.Dirt:GetChildren()) do
		table.insert(tableToSave.Dirt, v.Position)	
	end
	
	for _,v in pairs(BlocksFolder.Stone:GetChildren()) do
		table.insert(tableToSave.Stone, v.Position)	
	end
	
	for _,v in pairs(BlocksFolder.HardenedStone:GetChildren()) do
		table.insert(tableToSave.HardenedStone, v.Position)	
	end
end)

LoadTriggerEvent.OnServerEvent:Connect(function()
	
	for _,v in pairs(tableToSave.Dirt) do
		DrawBlock("Dirt",BlocksFolder.Dirt,Vector3.new(v.x,v.y,v.z))
	end    
	for _,v in pairs(tableToSave.Stone) do
		DrawBlock("Stone",BlocksFolder.Stone,Vector3.new(v.x,v.y,v.z))
	end
	for _,v in pairs(tableToSave.HardenedStone) do
    DrawBlock("HardenedStone",BlocksFolder.HardenedStone,Vector3.new(v.x,v.y,v.z))
	end
end)
1 Like

Ik this is a very late question but i was wondering why Unions dont show up when i load them in? i wrote a table for them and everything.

UnionOperation = {"Name", "Position", "Size", "Transparency", "BrickColor", "CanCollide", "CFrame", "Anchored" , "CastShadow" , "CollisionGroupId" , "Material"},

it still displays the output but the union is invisible

1 Like

Looking great. Doesn’t seem to lag when loading nor saving. If you’re planning to make the world bigger there might be a need to optimize it size-wise, else you’re good. As for loading, if you wanna make that faster, perhaps just replace the generic for loop with a renderstep, maybe doing everything under a single for loop is enough to make it slightly faster.

@KreatorKols Unions are a hard thing to save unfortunately! The problem is if you wanted to save a union, you technically need to save the parts that make up the union (the negative parts or whatever), and currently there is no way to get those. So unless the unions are common object that you have presets for like in Czlopek’s case, I don’t think you can do much besides keeping a table for each union specifying details on the parts that make up the union, information used to save and load those parts back, and do the necessary operations (the :UnionAsync() method) to make the union again.

4 Likes

(This is my first post on the forms so sorry if its bad) I just wanted to start by saying this is really well worded! I do have a question though. I have been trying to serialize a RodConstraint and its attachments, I have gotten the attachments to work but I’ve gotten stuck on RodConstraints, specifically the “Attachment0” and “Attachment1” part of RodConstraints. Any ideas on how serialize it?

2 Likes

for me roblox should add byte arrays to roblox

1 Like

Not a terrible idea, you can create something like that on your own. One does already exist though, check this out

1 Like

Hey! Sorry for the late answer (and what are the odds, as soon as I noticed your reply you made a topic about the question).

I didn’t actually cover this, and I should to be honest! There are many ways to take care of properties set to instances, but I think the easiest way would be to include a separate dictionary inside of the saved table, where each key is an id, set to a serialized instance.

When you save an object which has a property set to an instance, say Attachment0, you will generate a random id, in the serialized form set the property to that id.
This id will serve as a lookup in the separate dictionary I mentioned. In this dictionary you will save the serialized form of the instance the property was set to. I think you know where this is going.

When deserializing, if you come across Attachment0, you will grab its stored id, look for the part corresponding to its id inside of that dictionary, deserialize the part, and set Attachment0 to the part!

1 Like

Alright, I have to admit I’m a little confused but hey its the only way we learn! My main question is how would I identify the id? I guess I don’t really understand the whole id thing.

1 Like

The serialized property contains the id. The id is randomly generated when serializing, and the instance the property was set to will be serialized and inserted into a dictionary (that is saved too) under a key, where that key is the randomly generated id.

When deserializing, when you deserialize the property, you grab the saved id, go to the saved table containing the ids and corresponding instances, and grab the instance corresponding to the id you have, and deserialize it, and set the property to the instance!

This is the processs in short:
Serialization:

  • Generate ID
  • Set serialized property to the ID
  • Serialize instance the property is set to
  • Inside of a saved dictionary containing ids, insert the serialized part under the generated id

Deserialization:

  • Grab the ID from saved property
  • Index the saved dictionary containing the IDs to get the serialized part
  • Deserialize
  • Set the property to the deserialized part

And one thing, if the instance the property is set to is a child of some object (or maybe the object indexing the property itself), it might be serialized twice, and two copies might exist when you deserialize. To circumvent this, you might need to add some checks, or maybe change how the ID is saved.

1 Like

Could you give a code example? I really want to learn how to do this, but I simply don’t think I’m going to be able to do it without some help. I also understand writing a code sample is typically more time consuming then just a normal reply so don’t feel any pressure to do so.

Hi! So I have a building game and I am making a save/load feature. I have followed your instructions and everything is working, expect whenever you make for example a small house and save it, make some changes and save it again, all the blocks that were already saved are saved again. So you will have two houses in one because all the blocks are saved twice. I tried to fix this by going through the database with a for loop and check all the positions of the already saved blocks. Whenever the position of the new block is already saved in the database, it shouldn’t save. But it does. I have been trying to fix this for the last two days without success. Maybe you can help me?

This is the code of the InitProps function (everything else is the same as in the topic):

function InitProps(objects)
	local DS = game:GetService("DataStoreService"):GetDataStore("Slot")
	local ency = DS:GetAsync(tostring(game.Players:FindFirstChildOfClass("Player").UserId))

	local decoded = HttpService:JSONDecode(ency)

	local tableToSave = {}

	for _, obj in pairs(objects) do
		local class = obj.ClassName
		local t = tableToSave[class]

		local alreadyExists = false 

		if not(t) then
			tableToSave[class] = {}
			t = tableToSave[class]
		end

		for _, others in pairs(decoded) do
			for prop, value in pairs(others) do
				for data,values in pairs(value) do
					if data == "Position" then
						local posx = obj.Position.X
						local posy = obj.Position.Y
						local posz = obj.Position.Z
						
						if posx ~= values.X or posy ~= values.Y or posz ~= values.Z then
							
						else
							print("NO")
							alreadyExists = true
						end
					end
				end
			end
		end
		
		print(alreadyExists)
		
		if not alreadyExists then
			local add = {}

			for _, Prop in pairs(Properties[obj.ClassName]) do
				add[Prop] = Serialize(obj[Prop])
			end

			local children = obj:GetChildren()

			if #children > 0 then
				add["Children"] = InitProps(children)
			end
			table.insert(t, add)
		end
	end
	return tableToSave
end
1 Like

Why don’t you instead of saving what was newly added, re-save the entire house again. Or are you looking for efficiency? In that case, perhaps a simpler solution is having an array of anything added after the last save. When saving again add it to the previous save, and empty this array for it to be refilled again.

Why don’t you instead of saving what was newly added, re-save the entire house again.

I tried to do this first, but I found on the Forum that you can’t delete any data from the Datastore for some reason, so I thought it was impossible.

1 Like

Well you don’t have to delete it, if you save the serialized data under the same it will overwrite the old data.

Could you please show me an example of what you mean?

A little overthinking on saving EnumItem, just save EnumItem.Value and it will return the number and it will still work. Some parts could have been done better, but we appreciate your contribution to the community anyway.

1 Like

I’m getting the error invalid argument #1 to 'pairs' (table expected, got nil)
my code is

local ourFolder = workspace.Places[player.Name]:GetChildren()

    for _, v in pairs(ourFolder) do
        print("got item")
    end

    local result, Err = pcall(function()
        DS:SetAsync(player.UserId, Save.Encrypt(ourFolder))
    end)

it does print “got item”