How to use DataStore2 - Data Store caching and data loss prevention

Resources Community Tutorials
#1700

So now I have an issue that I been trying to fix for weeks now but all my attempts seem to not work.

My problems:

  • When someone purchases a vehicle, their money gets spent but the item value occasionally fails to append to the OwnedItems table. I know there’s gotta be a better way of doing this. The :Set function is kinda suspicious since it’s known to overwrite things.

  • When a player teleports, I even included a measure where when a player requests the server to teleport to another place, I tried using this technique where I would call the save function and pause the teleportation script until ConfirmSave event gets fired. Then the AfterSave gets called to fire the ConfirmSave event. The teleportation script then proceeds once ConfirmSave has been fired to now teleport the player. The issue here: It takes as much as 6+ minutes until AfterSave gets called before the teleport begins.

The game autosaves every 200 seconds and the second problem gets much worse on a place with 77,000 parts. Is it the part count? Mesh/unions? Too many server scripts? (I kept while loops with a wait time of at least 1 second up, but one script uses a server sided RunService loop), or collision groups? Or server memory? 1000 to 4000 moving parts? What is going on here?

local DataStore2 = require(script.Parent:WaitForChild("DataStore2"))

local defaultValue = 0
local defaultValue2 = -0.499
local defaultValue3 = 0
local defaultValue4 = {}
local defaultValue5 = 0

local Main = "MainStats"

DataStore2.Combine(Main, "DistanceTravelled", "OrbitsCompleted", "Money", "OwnedItems", "EXP")

local AllItems = {
	['LunarCruiser'] = {48000, 1},
	['Oppy'] = {17000, 2},
	['VIPER'] = {9000, 3},
	['LunarTerrainVehicle'] = {30000, 4},
	['Perseverance'] = {80000, 5},
	['Curiosity'] = {60000, 6},
	['Sojourner'] = {4000, 7},
	['DyneticsLander'] = {80000, 8},
	['SpaceExplorationVehicle'] = {100000, 9},
}

local PlayerService = game:GetService("Players")
local MarketplaceService = game:GetService("MarketplaceService")
local BadgeService = game:GetService("BadgeService")

PlayerService.PlayerAdded:Connect(function(plr)
	local DistTravelled = DataStore2("DistanceTravelled",plr)
	local OrbitsComp = DataStore2("OrbitsCompleted", plr)
	local Money = DataStore2("Money", plr)
	local OwnedItems = DataStore2("OwnedItems", plr)
	local EXP = DataStore2("EXP", plr)

	local folder = Instance.new("Folder",plr)
	local hiddenstats = Instance.new("Folder",plr)
	hiddenstats.Name = "HiddenStats"
	folder.Name = "leaderstats"

	local DistTravelled2 = Instance.new("IntValue",folder)
	local OrbitsComp2 = Instance.new("IntValue",folder)
	DistTravelled2.Name = "Distance Travelled"
	OrbitsComp2.Name = "Orbits Completed"

	-- Multipliers

	local EXPMultiplier = Instance.new("NumberValue",hiddenstats)
	EXPMultiplier.Name = "EXPMultiplier"
	EXPMultiplier.Value = 1
	local MoneyMultiplier = Instance.new("NumberValue",hiddenstats)
	MoneyMultiplier.Name = "MoneyMultiplier"
	MoneyMultiplier.Value = 1
	local MoneyMultiplier_Raw = 1

	if MarketplaceService:UserOwnsGamePassAsync(plr.UserId, 13494489) then
		EXPMultiplier.Value *= 1.6
		MoneyMultiplier_Raw *= 1.4
	end
	if MarketplaceService:UserOwnsGamePassAsync(plr.UserId, 14703981) then
		MoneyMultiplier_Raw *= 2
	end
	if MarketplaceService:UserOwnsGamePassAsync(plr.UserId, 12713201) then
		EXPMultiplier.Value *= 1.6
		MoneyMultiplier_Raw *= 1.4
	end
	if plr:IsInGroup(8333210) then
		MoneyMultiplier_Raw *= 1.2 
	end
	if BadgeService:UserHasBadgeAsync(plr.UserId, 2124605245) then
		EXPMultiplier.Value *= 10
		MoneyMultiplier_Raw *= 10
	end
	MoneyMultiplier.Value = MoneyMultiplier_Raw

	-- Load

	DistTravelled2.Value = DistTravelled:Get(defaultValue)
	OrbitsComp2.Value = OrbitsComp:Get(defaultValue2)
	game.ReplicatedStorage.MonetaryFunctions.UpdateCurrency:FireClient(plr, 
    Money:Get(defaultValue3))
	game.ReplicatedStorage.GainEXP:FireClient(plr, EXP:Get(defaultValue5))

	--Update
	DistTravelled:OnUpdate(function(UpdatedDistance)
		DistTravelled2.Value = UpdatedDistance
	end)

	OrbitsComp:OnUpdate(function(UpdatedOrbits)
		OrbitsComp2.Value = UpdatedOrbits
		MoneyMultiplier.Value = MoneyMultiplier_Raw * ((UpdatedOrbits/32) + 1)
	end)

	Money:OnUpdate(function(UpdatedMoney)
		game.ReplicatedStorage.MonetaryFunctions.UpdateCurrency:FireClient(plr, UpdatedMoney)
	end)

	EXP:OnUpdate(function(UpdatedEXP)
		game.ReplicatedStorage.GainEXP:FireClient(plr, UpdatedEXP)
	end)
	
	Money:AfterSave(function(newMoney)
		game.ServerStorage.ConfirmSave:Fire(plr)
	end)

end)

game.ReplicatedStorage.GainDistance.OnServerEvent:Connect(function(plr,speed,orbitspeed)
	if speed > -10 and speed <= 12 and speed == speed and orbitspeed == orbitspeed and orbitspeed > -0.005 and orbitspeed <= 0.005 then
		local DistTravelled = DataStore2("DistanceTravelled", plr)
		local OrbitsComp = DataStore2("OrbitsCompleted", plr)
		DistTravelled:Increment(speed,defaultValue)
		OrbitsComp:Increment(orbitspeed,defaultValue2)
	end
end)

game.ReplicatedStorage.Donate.OnServerInvoke = function(player, amount, amount2, recipient)
	if player.UserId == 30683604 then
		local rec = game.Players:FindFirstChild(recipient)
		local DistTravelled = DataStore2("DistanceTravelled", rec)
		local OrbitsComp = DataStore2("OrbitsCompleted", rec)
		DistTravelled:Increment(amount, defaultValue)
		OrbitsComp:Increment(amount2, defaultValue2)
	elseif player.UserId == 71338532 then
		if amount < 211500 and amount2 < 5 then
			local rec = game.Players:FindFirstChild(recipient)
			local DistTravelled = DataStore2("DistanceTravelled", rec)
			local OrbitsComp = DataStore2("OrbitsCompleted", rec)
			DistTravelled:Increment(amount, defaultValue)
			OrbitsComp:Increment(amount2, defaultValue2)
		end
	end
end


--------------- MONEY STUFF ------------------

function ReceiveItemAfterBuying(plr, item)
	local OwnedItems = DataStore2("OwnedItems", plr)
	local Cache = OwnedItems:GetTable(defaultValue4)
	if table.find(Cache, item) == nil then
		table.insert(Cache, item)
		OwnedItems:Set(Cache)
		print(unpack(Cache))
		return 'Purchased'
	elseif table.find(Cache, item) ~= nil then
		return 'Already Owned'
	end
end

function BuyItem(plr, item) -- item is a string. Precondition: CheckIfOwned is invoked.
	local Money = DataStore2("Money", plr)
	if AllItems[item] ~= nil then
		if Money:Get(defaultValue3) >= AllItems[item][1] then
			Money:Increment(- AllItems[item][1] , defaultValue3)
			return ReceiveItemAfterBuying(plr, AllItems[item][2])
		else
			return "Can't Afford"
		end
	end
end

function CheckIfOwned(plr, item)
	local OwnedItems = DataStore2("OwnedItems", plr)
	local Cache = OwnedItems:GetTable(defaultValue4)
	if table.find(Cache, AllItems[item][2]) ~= nil then
		return 'Item Owned'
	elseif table.find(Cache, AllItems[item][2]) == nil then
		return 'Item Not Owned'
	end
end

function Earn(plr, amount) -- Keep this server sided.
	if amount == amount then
		local Money = DataStore2("Money", plr)
		local MultiplierFolder = plr:FindFirstChild("HiddenStats")
		local FinalAmount = amount
		if MultiplierFolder then
			FinalAmount = amount * MultiplierFolder.MoneyMultiplier.Value
		end
		Money:Increment(FinalAmount, defaultValue3)
	end
end

game.ReplicatedStorage.MonetaryFunctions.Buy.OnServerInvoke = BuyItem
game.ReplicatedStorage.MonetaryFunctions.CheckOwnership.OnServerInvoke = CheckIfOwned
game.ServerStorage.CheckOwnership.OnInvoke = CheckIfOwned
game.ServerStorage.EarnMoney.Event:Connect(Earn)


--[[
Call CheckIfOwned first before calling BuyItem
]]

------------------------------------------------

game.ServerStorage.EXP_Server.Event:Connect(function(plr, EXPAmt)
	if EXPAmt < 10000 and EXPAmt == EXPAmt then
		local EXP = DataStore2("EXP", plr)
		EXP:Increment(EXPAmt, defaultValue5)
	end
end)
------------------ SAVING ----------------------
function Save(plr)
	DataStore2.SaveAll(plr)
	print("Saved All Data")
end

game.ReplicatedStorage.AutoSaveRemote.OnServerEvent:Connect(Save)
game.ServerStorage.Save.Event:Connect(Save)
#1701

Retrying isn’t for when nil is returned, it’s for when data stores error.

#1702

What? You have to call :Set in the end, even methods like :Update and :Increment are just wrappers over :Set.

My guess is that DataStore2 is being throttled hard (auto saving for everyone every 200 seconds sounds extremely likely to hit the very low limits on ordered data stores). If your game is not yet released, I recommend switching to the Standard saving mode which has much laxer throttling.

#1703

oh wow thats kind of bad then

#1704

Ohh I see. But when it comes to tables, is there a better method to call when appending something on a table just like we call :Increment on numerical values or is that the only way?

And for saving every 200 seconds, it’s not at the same time. It’s based on when the player joined. If throttling was the case, the 2-6+ minute delay also happens when I’m the only one on the server, especially I just joined (even less than 200 seconds ago). I also don’t see any error messages of it throttling on the developer console, which is what kept me for weeks finding the cause.

#1705

I’m trying to use DataStore2 in my game. But i’ve run in to an issue, here’s the code:
(issue explained after code)

local players = game:GetService("Players")
local InventoryLibrary = require(game:GetService("ReplicatedStorage"):WaitForChild("InventoryLibrary"))
local dataStore2 = require(1936396537)
 
     dataStore2.Combine("DATA", "inventory")
     players.PlayerAdded:Connect(function(player)
     	
     	local InventoryDataStore = dataStore2("inventory", player)
     	local inv = InventoryDataStore:Get(InventoryLibrary:New(player.UserId))
     	print(inv)
end)

The problem lies in the middle part where i try to run print(inv)
It prints sometimes, sometimes it doesn’t. And InventoryLibrary:New(player.UserId) returns a table.

So to my understanding this script should see if the player had an inventory. If the player does not, run InventoryLibrary:New(player.UserId) which returns a table. Save that, then print it.
The problem isn’t because it yeilds, see I tried with :GetAsync instead of :Get yet it was still inconsitent.

#1706

Do not require by ID. Read the documentaiton.

My money is that requiring by ID is yielding, so the players load before PlayerAdded is ran. Don’t require by ID.

#1707

Doing it the way you suggest wouldn’t counteract when Roblox returns nil (which would be a consistent failure), and would just throttle harder for the 99.9% scenario of “new player joined”.

1 Like
#1708

No, just :Get(), write, then :Set().

2 Likes
#1709

By the way, DataStore2 is gonna go down and up, and down and up.

The DataStore 1.1 Update seems to do the exact same as DS2 in the Ordered Backups method, except in a raw matter, main difference being: The versions get wiped after 30 days. Which is pretty good given that a lot of people have problems deleting data when roblox requests data deletion for certain players, and the option to really mess with these backups. Which is good.

This also means the Standard Method is not the odd one out there. If I were to use DS2, I would use it for the OrderedBackups method anyway. Using Standard feels wrong for a module which aims at data loss prevention, it doesn’t feel right to me to use Standard just because there’s no backups. With datastore 1.1, that’s not a issue.

I have a question, what do you think about supporting datastore 1.1 backups? I understand DS2 has not been updated since 2019, and that you have other things to do outside DS2. It’s completely fine. I don’t want you to feel forced to support it.

#1710

When using :Get() to retrieve the data, OnUpdate() to update the data, if the data is in form of a table, the result we get is still a table right? And do its indexes(dictionary) stay the same?

#1711

Thanks for the help:)
Where in the documentation can I find it?

#1712

I hadn’t seen this part as clear yet, but how do I make a datastore using DataStore2?

#1713

The data store is what is constructed using DataStore2().

2 Likes
#1714

Ah, thank you for letting me know!

#1715

I am currently trying to make a datastore just for the server, and so on player leave is not working for me.

#1716

You never had access to backups as a developer anyway unless you rummage through the code.

As I’ve said before, I’m not convinced that Ordered Backups is why DataStore2 doesn’t have data loss. I am more convinced that the lack of data loss from DataStore2 simply comes from the fact that I am an experienced developer who has battle tested this code across the world and back to solve every nook and cranny from a notoriously hard to stabilize API.

I have no idea what this is.

That’s because my projects move with me. I haven’t felt any need to update DataStore2 as it works perfectly for my use cases. With the new data store APIs that are coming, I plan on updating it to support that once I start working on data stores for my next game (which I have no ETA for).

1 Like
#1717

The installation process is detailed here: Installation - DataStore2

#1718

DataStore2 is currently only for player data stores, as I have not felt a need for server data stores and nobody has PR’d it.

#1719

OnUpdate() to update the data

No. You have Set to update the data. OnUpdate tells you when the data is updated.

Yes. There is no reason it would be any different.