Cutting back on the amount of Datastores used in my game

[Eh_Canadian0]

I made an in-game cosmetic shop where players can buy Trails, Auras, and Gear, however I had made each cosmetic it’s own datastore and as such I was presented with the this warning:

DataStore request was added to queue. If request queue fills, further requests will be dropped. Try sending fewer requests

So as means of curbing this I decided to try to merge all the data stores into one massive store for all the cosmetics when the player leaves the game. However I am now presented with a new problem, that being how I would reload all the players assets from their datastore. This is as certain items use the same names as others (ie. an orange trail is being saved as well as an orange aura).

Here is the script to save the player's data upon leaving (I had a previous script already saving the players Cash, Wins, and 'ChestTime' so I just decided to combine the save function with that). I have also obfuscated some parts of the code so it would be easier to read.

local PlayersService=game:GetService('Players')
local DataStoreService=game:GetService('DataStoreService')
local ds = DataStoreService:GetDataStore("DATA")

local DataStore=DataStoreService:GetDataStore(":ChestData")
local CurrencyStore=DataStoreService:GetDataStore(":Currency")
local WinsStore = DataStoreService:GetDataStore(":Wins")

PlayersService.PlayerRemoving:Connect(function(Player)
	if Player:FindFirstChild("ChestData") and Player.ChestData.Value==true then
		if not ChestSettings.CreateNewCurrency then return end
		local Success,Errormsg=pcall(function()
			CurrencyStore:SetAsync(Player.UserId, Player.leaderstats[ChestSettings.CurrencyName].Value)
			WinsStore:SetAsync(Player.UserId, Player.leaderstats.Wins.Value)
			
			-- AURAS
			
			local ownedTrails = {}
			for i, trail in pairs(Player.OwnedTrails:GetChildren()) do -- CHANGE THIS
				table.insert(ownedTrails, trail.Name)
			end
			
			-- GEARS
			
			local ownedGears = {}
			for i, trail in pairs(Player.OwnedGears:GetChildren()) do -- CHANGE THIS
				table.insert(ownedGears, trail.Name)
			end
			
			-- TRAILS
			
			local ownedAuras = {}
			for i, trail in pairs(Player.OwnedAuras:GetChildren()) do -- CHANGE THIS
				table.insert(ownedAuras, trail.Name)
			end
			
			-- COMBINE TABLES
			
			local ownedEverything = {}
			for i, TableChild in pairs (ownedTrails) do
				table.insert(ownedEverything, ownedTrails)
			end
			for i, TableChild in pairs (ownedGears) do
				table.insert(ownedEverything, ownedGears)
			end
			for i, TableChild in pairs (ownedAuras) do
				table.insert(ownedEverything, ownedAuras)
			end
			
			local success, err = pcall(function()
				ds:SetAsync("allitmes-" .. Player.UserId, ownedEverything) -- CHANGE THIS
			end)
			
		end)
		if not Success then warn(Errormsg) else print("Saved Currency") end
	end
end)

.
.
.

And here is my (albeit unfinished as I didn't know how to continue on) script which would load the players auras (again some bits obfuscated)

local dss = game:GetService("DataStoreService")
local ds = dss:GetDataStore("DATA")

local SS = game:GetService("ServerStorage")
local RS = game:GetService("ReplicatedStorage")

local Events = RS:WaitForChild("Customization")

-- CHANGE THIS

local auras = SS:WaitForChild("Cosmetics"):WaitForChild("Auras")
local Choose = Events:WaitForChild("AuraChooseRE")
local Buy = Events:WaitForChild("AuraBuyRE")

-- player added

game.Players.PlayerAdded:Connect(function(plr)
	
	local aurasOwned = {}
	pcall(function()
		aurasOwned = ds:GetAsync("allitmes-" .. plr.UserId) or {} -- CHANGE THIS
	end)

	
	local ownedFolder = Instance.new("Folder", plr)
	ownedFolder.Name = "OwnedAuras" -- CHANGE THIS
	
	for i, owned in pairs(aurasOwned) do
		
		if auras:FindFirstChild(owned) then
			
			auras[owned]:Clone().Parent = ownedFolder
		end
	end

end)

The script managing the players auras, gears, and trails have all more or less the same implications.

What practice should I go about doing so as to load the player’s data?

I have thought about so far:

• Changing the names of items (OrangeTrail, OrangeAura, BlueAura, BloxyColaGear), however I would not know how to occlude and index “Gear” / “Trail” / “Aura”
• Whether the object’s class would be saved or not and if I could just modify my scripts to work around that

[Zolyphony]

I would split it into two datastores with one being “Stats” and the other being “Items”. Depending on the type of stat or item (as in one scope can be “Auras” or “Trails”) you can label it under a different scope

[Eh_Canadian0]

Alright I’ll try a shared save file for Gears + Trails and one for just Auras (In case I’d like to add a “bloxy cola” aura or something like that lol)

[Eh_Canadian0]

So I made some changes to the saving script (This is what it looks like now):

PlayersService.PlayerRemoving:Connect(function(Player)
	if Player:FindFirstChild("ChestData") and Player.ChestData.Value==true then
		if not ChestSettings.CreateNewCurrency then return end
		local Success,Errormsg=pcall(function()
			CurrencyStore:SetAsync(Player.UserId, Player.leaderstats[ChestSettings.CurrencyName].Value)
			WinsStore:SetAsync(Player.UserId, Player.leaderstats.Wins.Value)
			
			-- AURAS
			
			local ownedTrails = {}
			for i, trail in pairs(Player.OwnedTrails:GetChildren()) do
				table.insert(ownedTrails, trail.Name)
			end
			
			-- GEARS
			
			local ownedGears = {}
			for i, trail in pairs(Player.OwnedGears:GetChildren()) do
				table.insert(ownedGears, trail.Name)
			end
			
			-- TRAILS
			
			local ownedAuras = {}
			for i, trail in pairs(Player.OwnedAuras:GetChildren()) do
				table.insert(ownedAuras, trail.Name)
			end
			
			-- COMBINE TABLES
			
			local ownedEverything = {}
			for i, TableChild in pairs (ownedTrails) do
				table.insert(ownedEverything, ownedTrails)
			end
			for i, TableChild in pairs (ownedGears) do
				table.insert(ownedEverything, ownedGears)
			end
			
			local success, err = pcall(function()
				ds:SetAsync("Gears and Trails-" .. Player.UserId, ownedEverything
				ds:SetAsync("Auras" .. Player.UserId, ownedAuras)
				
				print (ownedEverything) -- *** See note
				print (ownedAuras) -- ***See note
			end)
			
		end)
		if not Success then warn(Errormsg) else print("Saved Currency") end
	end
end)

***However I noticed that when I rejoin the items do not reload in so I added the two prints at the end of the script and this is printed into the output:

  20:15:48.284   ▼  {
                    [1] =  ▼  {
                       [1] = "Green",
                       [2] = "Orange"
                    },
                    [2] =  ▼  {
                       [1] = "Green",
                       [2] = "Orange"
                    },
                    [3] =  ▼  {
                       [1] = "Fidget Spinner",
                       [2] = "Chez Burger",
                       [3] = "Ice Cream",
                       [4] = "Spork"
                    },
                    [4] =  ▼  {
                       [1] = "Fidget Spinner",
                       [2] = "Chez Burger",
                       [3] = "Ice Cream",
                       [4] = "Spork"
                    },
                    [5] =  ▼  {
                       [1] = "Fidget Spinner",
                       [2] = "Chez Burger",
                       [3] = "Ice Cream",
                       [4] = "Spork"
                    },
                    [6] =  ▼  {
                       [1] = "Fidget Spinner",
                       [2] = "Chez Burger",
                       [3] = "Ice Cream",
                       [4] = "Spork"
                    }
                 }  -  Server
  20:15:48.284   ▼  {
                    [1] = "Green",
                    [2] = "Blue"
                 }  -  Server

Do you know what I could do to fix the mess with the tables?

[weakmetatable (metatable)]

Hi,

Personally I like to use a download → cache system. Imagine everything being stored within a single large table, like such:

{
	["trails"] = {...},
	["effects"] = {...},
	["swords"] = {...},
	["experience"] = 152,
	["level"] = 23,
}

I use the term “download” because it implies that I attain the data, and then store it somewhere. My download function looks like this:

local function cacheAsync(player: Player, retryOnFail: boolean)
	local success, result = xpcall(
		function ()
			return DATABASE:GetAsync(DATABASE_KEY .. player.UserId)
		end
		, function (...: any)
			warn(...)
		end
	)

	if (not success and retryOnFail) then
		local attempts = 0

		while attempts < MAX_TRIES do
			attempts += 1

			success, result = cacheAsync(player)

			task.wait(RETRY_TIME)
		end
	end
	
	if (success) then
		if (not result) then
			result = default
		end
		
		cache[player] = result
	end
	
	return success
end

This function is only ran once for each player that joins the experience. The result is stored within a cache table within the main module scope. Then if any arbitrary script requires the data, instead of requesting the data again via :GET, it’ll fire a BindableFunction. Example:

requestDataEvent:Invoke(player, "swords", "experience")

It would take in tuple arguments, here being “swords” and “experience”.

---

Keep in mind, roblox allows you to store 4.19 million (I believe UTF-8) characters. This means that as long as you don’t have a plot system (Such as Bloxburg) you can save almost a seemingly endless amount of data.

Aditionally, if you’re wondering how to edit the data via methods such as :UPDATE, :SET or :INCREMENT. You can again, use a BindableFunction. And since the :Invoke already yields, it acts the same as a regular call to the datastore. Keep in mind you DO NOT want to make a call to the datastore every single time you use :SET, :UPDATE or :INCREMENT. You want to edit the cache table instead. When the player leaves, or when :BindToClose is called, that’s when you’ll want to make an actual call to the datastore.

To add on further, if you’re worried about data loss, you can simply add an auto save system. So in the unlikely event the servers are down, the player will only lose ~5 minutes worth of progress. And additionally if the servers are down, you can instance a “Servers are currently down; saving data. Don’t leave.” screen just in case.

To add on even further. When a player joins, you can create a loading screen which yields until the data has entirely downloaded. Then remove the screen after the data has downloaded successfully.

There are many other strategies when it comes to loading/saving data without getting rate limited. But this simple design doesn’t fail . (1/1,000,000 chance maybe )

I’m not sure if this was what you were implying as a whole, but I like my style of downloading data via :GET → caching/storing to a cache table. It’s slick and easy to manage. It’s my bad if I misinterpreted what you inferred.

Good luck

[Eh_Canadian0]

Ok I’m almost near the end of my modifications to the script, the save data works perfectly fine but now I just want to know how to retreive + apply the data.

This is the load script I have so far:

game.Players.PlayerAdded:Connect(function(plr)
	
	local items
	
	local aurasOwned = {}
	pcall(function()
		items = ds:GetAsync("Items-" .. plr.UserId) or {} -- CHANGE THIS
	end)
	
	print (items) -- ***
	
	local ownedFolder = Instance.new("Folder", plr)
	ownedFolder.Name = "OwnedAuras" -- CHANGE THIS
	
	for i, owned in pairs(aurasOwned) do
		
		if auras:FindFirstChild(owned) then
			
			auras[owned]:Clone().Parent = ownedFolder
		end
	end

end)

***The print function prints out this:

  18:13:57.607   ▼  {
                    [1] =  ▼  {
                       [1] = "Indigo",
                       [2] = "Orange",
                       [3] = "Yellow"
                    },
                    [2] =  ▼  {
                       [1] = "Fidget Spinner",
                       [2] = "Ice Cream",
                       [3] = "Spork"
                    },
                    [3] =  ▼  {
                       [1] = "Green",
                       [2] = "Indigo",
                       [3] = "Orange"
                    }
                 }  -  Server - AuraApplier:26

How can I get the reloading script to access the 3rd subtable of “items”?

[Katrist]

Just index it as you would with any other table:

game.Players.PlayerAdded:Connect(function(plr)

	local items

	local aurasOwned = {}
	
	pcall(function()
		items = ds:GetAsync("Items-" .. plr.UserId) or {
			{},
			{},
			{}
		} -- CHANGE THIS
	end)

	aurasOwned = items[3]

	local ownedFolder = Instance.new("Folder")
	ownedFolder.Name = "OwnedAuras" -- CHANGE THIS
	ownedFolder.Parent = plr
	
	for i, owned in pairs(aurasOwned) do
		if auras:FindFirstChild(owned) then
			auras[owned]:Clone().Parent = ownedFolder
		end
	end
end)

[Eh_Canadian0]

Works like a charm, thanks for all the help!