Data integration

Hey there,

I am currently working on a roblox game and with every game … comes data.

My biggest concern of making a game is having people get data when they join, but then we add the new data to the template and the new players will get it but the people who already got data, will not add it into their data.

I dont want them to miss out on different aspects of the data that the player has already vs the new player where it is already importated into the template and they can just get it.

Here is an script of how i normally store my data:

local DataStoreService = game:GetService("DataStoreService")
local DataStore = DataStoreService:GetDataStore("PlayerData")

local SessionData = {}
local TemplatedData = {
	-- Values
	["Value 1"] = 100,
	["Value 2"] = 300,
	
	-- Tables
	["Table 1"] = {},
	["Table 2"] = {}	
}

function loadData(ID)
	local Attempt = 1
	local Success = false
	local Data = nil
	
	repeat
		local success, err = pcall(function()
			Data = DataStore:GetAsync(ID) or TemplatedData
		end)
		
		if success then
			Success = true
			SessionData[ID] = Data
		end
		
		if err then
			warn(err)
		end
		
		Attempt += 1
	until Success == true or Attempt == 3
end

function saveData(ID)
	local Attempt = 1
	local Success = false
	local Data = SessionData[ID]
	
	repeat
		local success, err = pcall(function()
			DataStore:SetAsync(ID, Data)
		end)
		
		if success then
			print("Success!")
			Success = true
		end
		
		if err then
			warn(err)
		end
		
		Attempt += 1
	until Attempt == 3 or Success == true
end

game.Players.PlayerAdded:Connect(function(player)
	loadData(player.UserId)
end)

game.Players.PlayerRemoving:Connect(function(player)
	saveData(player.UserId)
end)

If for some reason I add a feature into the game but I need to also add it into there data like this:

local TemplatedData = {
	-- Values
	["Value 1"] = 100,
	["Value 2"] = 300,
	["Value 3"] = 1050,
	
	-- Tables
	["Table 1"] = {},
	["Table 2"] = {},
	["Table 3"] = {}
}

They will now not have table 3 or value 3 as it was not in there information when they first joined.

Is there a fix or something to resolve this issue?

(Sorry if i am talking all crazy, i dont really understand how I can do this, i also suck at gra

1 Like

When you first load the players data (when they join the game). Loop over the TemplatedData and check if the playersData has each item. If they are missing said item → add it and set it to the default tables value.

2 Likes

How would I add this check into the function tho?

function loadData(ID)
	local Attempt = 1
	local Success = false
	local Data = nil
	
	repeat
		local success, err = pcall(function()
			Data = DataStore:GetAsync(ID) or TemplatedData
		end)
		
		if success then
			Success = true
			SessionData[ID] = Data
             for Name, Value in pairs(TemplatedData) do -- loop through all values in ur default data dictionary
                 if not SessionData[ID][Name] then -- check if said value exist in the players data
                    SessionData[ID][Name]  = Value; -- if not, add and set it to the default
end;
            end
		end
		
		if err then
			warn(err)
		end
		
		Attempt += 1
	until Success == true or Attempt == 3
end

      

this is just an example. I’m on mobile so there may be typos/formating issues.

1 Like

I’ll check with this when I get home, thanks :slight_smile:

This is called Reconciling data, and is a feature provided by datastore wrappers like ProfileService. Nonetheless, you can implement it by:

local function Reconcile(data, template)
    for key, value in template do
        if not data[key] then 
           data[key] = value
        end
    end
end

Reconcile(Data, TemplateData)
1 Like

oh, lol thought it was called integration (thats what i hear everyone call it). also, would I just use the reconcile function with the data (the player has loaded) and the templated? Also, would I need to find another way to add it to the session data?

You just call the function after getting your data:

if success then
	Success = true
    Reconcile(Data, TemplateData)
	SessionData[ID] = Data
end
2 Likes

@Mystxry12

Reconciling does not work:

local Template = {
	["Player Information"] = {
		["Cash"] = 100,
		["Cash2"] = 200,
		["Inventory"] = {
			["Pets"] = {["Troll"] = {["ID"] = 1000}}
		}
	},
	["Backend Information"] = {
		["FirstTimeJoining"] = true,
		["JoinCount"] = 0,
		["UserId"] = "",
		["AccountAge"] = 0,
		["Usernames"] = {}
	}
}

function EzSave.Reconcile(ID, Data, Template)
	for key, value in pairs(Template) do
		if not Data[key] then
			SessionData[ID][Data][key] = value
			print("Added:", key, "with", value)
		end
	end
end



function EzSave.PlayerJoined(player)
	local Attempt = 1
	local Success = false
	local Data = nil

	repeat
		local success, errorMessage = pcall(function()
			Data = PlayerDataStore:GetAsync(player.UserId) or Template
		end)

		if success then
			Success = true
			EzSave.Reconcile(player.UserId, Data, Template)
			SessionData[player.UserId] = Data
			Data["Backend Information"]["UserId"] = player.UserId
			Data["Backend Information"]["AccountAge"] = player.AccountAge
			table.insert(Data["Backend Information"]["Usernames"], player.Name)

			if Data["Backend Information"]["FirstTimeJoining"] then
				Data["Backend Information"]["FirstTimeJoining"] = false
			end

			Data["Backend Information"]["JoinCount"] += 1

			SessionData[player.UserId] = Data
		end

		if errorMessage then
			warn(errorMessage)
		end

		Attempt += 1
	until Attempt == 3 or Success == true
end
1 Like
SessionData[player.UserId] = Data
EzSave.Reconcile(player.UserId, Data, Template)

P.S:- Seems like you don’t understand how tables work; when you reference Data it is returning the table at that memory address and when you edit Data, all references to that memory address are automatically updated to the new value. So you don’t need to do SessionData[player.UsedId] = Data every time you update Data.

Also you might need to expand upon your Reconcile function to deep copy tables when necessary. Here’s how ProfileService did it.

local function DeepCopyTable(t)
	local copy = {}
	for key, value in pairs(t) do
		if type(value) == "table" then
			copy[key] = DeepCopyTable(value)
		else
			copy[key] = value
		end
	end
	return copy
end

local function ReconcileTable(target, template)
	for k, v in pairs(template) do
		if type(k) == "string" then -- Only string keys will be reconciled
			if target[k] == nil then
				if type(v) == "table" then
					target[k] = DeepCopyTable(v)
				else
					target[k] = v
				end
			elseif type(target[k]) == "table" and type(v) == "table" then
				ReconcileTable(target[k], v)
			end
		end
	end
end
1 Like

Yep, I saw my issue before and forgot to delete the post. my fault

(Also yes, I do understand how tables work)