Need help rewriting / fixing datastore script

Help and Feedback Scripting Support
#1

Hello, everyone

I have about 5 of these - each one controlling it’s own DataStore. There’s one for Credits, Cosmetics, Armor, Settings, and then this one - which contains the “System” datastore, which controls things like if the player is banned, their total kills, total playtime, etc.

I recently tried to fix the problem with it becoming exhausted, but my efforts seem to have failed me, especially with full servers. I don’t want to hire anyone currently to rewrite it currently as I don’t have the funds. Still though, if it’s a simple fix, put the solution in a response to this. I’m not very good with DataStores, and I think this was probably something I shouldn’t have gotten into in the first place. :stuck_out_tongue:

Code:

local stuff={
--<<------------------------------------------------------------------------------------------------>>--
"premiumMultiplier","NumberValue",										1;
"isBanned","BoolValue",													false;
"numBans","NumberValue",												0;
"numWarnings","NumberValue",											0,
"storeDiscount","NumberValue",											1,
--<<------------------------------------------------------------------------------------------------>>--
"MatchesWon","NumberValue",												0,
"HighestSpree","NumberValue",											0,
"TotalKills","NumberValue",												0,
"TotalDeaths","NumberValue",											0,
"KDR","NumberValue",													0,
"TotalHeadshots","NumberValue",											0,
"TotalMatchesWon","NumberValue",										0,
"TotalTimePlayed","NumberValue",										0,
"ChallengesCompleted","NumberValue",									0,
"TotalCreditsEarned","NumberValue",										0,
"FirstTimePlaying_TEST","BoolValue",									true,
}


local DS=game:GetService"DataStoreService":GetDataStore("SYSTEM","global")
game:GetService"Players".PlayerAdded:Connect(function(p)
local leaderstats=p:WaitForChild("SYSTEM",.3)or table.foreach({''},function(i)i=Instance.new("Folder",p)i.Name="SYSTEM"return(i)end)

for x=1,#stuff,3 do
    local val=Instance.new(stuff[x+1])
    val.Parent=leaderstats
    val.Name=stuff[x]
    
    local save;
    delay(6,function()
        save=true;
    end)

    local data=DS:GetAsync(p.UserId..stuff[x])
    val.Value=data and type(stuff[x+2])==type(data)and data or stuff[x+2]

    print("Creating SystemService for "..p.Name)

    val.Changed:Connect(function()
        if save then
            save=false;
            DS:SetAsync(p.UserId..val.Name,val.Value)
            delay(6,function()
                save=true;
            end)
        end
    end)

end


end)
#2

I think you should step back and rethink how you are handling this. You are using a lot of different keys for one player’s data, individual for each value. In reality you should save all the data as a table and read/write only when needed.

I cant think of a reason to save on each value.Changed, this will just flood SetAsync requests.

Here is an example on how you can slim this down.

local saveables = {
    Stats = {
        Coins=0, Kills=0,
    };
    Misc = {
        TotalTimePlayed=0
    }
}

Now you can save and load that whole table (Dictionary) at once, saving at important times only, such as play leave, server shut down, and regular intervals as a backup (every 2 mins is more than enough)

As for loading, you can handle it as such

local data = DataStore:GetAsync(player_unique_key)
For category, list in pairs(data) do
    for name,value in pairs(list) do
        -- make your folders and int/string/bool values as needed
   end
end

Add in error handling and such

1 Like
#3

Problems:

Too many requests.

You’re storing each value in a separate datastore key, so with 16 different values, you’re sending 16 Get requests every time a player joins the game. You throttle your saves so that each value can only be updated in the datastore every six seconds, but still you can be sending 16 Set requests at the same time if all of the values update around the same time. This is waaaaay too many requests.

Solution:

Instead of storing each value in a separate key, store them all in one key. E.g.

SomeDataStore[p.UserId.."PlayerData"] = {
    PremiumMultiplier = 1;
    IsBanned = false;
    NumBans = 0;
    TotalKills = 0;
}

This way you use one request for loading, and one request for saving. I also recommend saving data on an interval of every 1 minute and whenever the player leaves the game, rather than potentially every 6 seconds. Players can use up to 10 requests a minute for saving on Changed with throttle, versus 1 (+1 if they leave) a minute with an interval save.

No error handling

What happens if a DataStore request errors? If a Get request fails, the player’s data will never be loaded, and anything they do in the game will not save because the Changed event is never hooked into. If a save fails, it’s less bad – only that save will fail and it will be fixed when it is saved again, but this is still bad.

Solution

Build error-handling into your game. I recommend adding all requests to a queue – keep retrying if the request fails with a Roblox issue, and scrap the request and handle accordingly if it is an issue with your code (e.g. value in SetAsync is nil). You can catch errors with pcall, and determine what kind of errors they are with DataStore Errors.

4 Likes
#4

So with the key, is it possible to update my current datastore without player dataloss? That’s the only thing I’m worried about as some of my playerbase have already made purchases that are saved in a datastore, i.e. “Credits.”

#5

Yes. To migrate data to the new format, you can do something like this:

playerAdded:
    local data = tryLoadNewData()

    if not data:
        local newData
        if hasAtLeastOneKeyOfOldData(): -- e.g. totalTimePlayed != nil
            local oldData = loadAll16Keys()
            newData = convert16KeysToNewFormat()
        else: -- new player
            newData = getDefaultData()

        saveData(newData) -- player data is now in new format
        data = newData

You can remove this check later down the road when you’re certain substantially most players have been converted to the new format (a few people will lose data, but you can manually restore per-request, or just tell them tough luck), or leave it in indefinitely (but know you’ll be using an extra request to check for old data every time a new player joins).

3 Likes
#6

Will look into doing this. I can’t thank you enough! :slight_smile:

Also, with LoadAllKeyes, would I just run a for loop that goes through the current datastore titled “SYSTEM,” obtain all of the keys within “SYSTEM” except for the table key, and then update the values within the table key with the keys present directly under SYSTEM? Haven’t messed with tables in datastores, so this’ll be new.

#7

You’re going to need to manually hardcode the key names into your script. If you reuse the key names from the then-current datastore, the format may have been changed (key removed or added). Then you just do

function loadAll16Keys()
    local oldData = {}

    local oldKeyNames = { "totalTimePlayed", "Credits", ... }
    local oldDataStore = datastoreservice:GetDataStore(oldDataStoreName)
    for _,keyName in pairs(oldKeyNames) do
        local value = oldDataStore:GetAsync(p.UserId..keyName) -- make sure to do error handling here
        oldData[keyName] = value
    end

    return oldData
end
4 Likes
#8

This is really helpful.

Will come back here if I need anymore help. :slight_smile:

#9

Great news, but also bad news:

The datastore now saves a table. However, when opening the DataStore with @Crazyman32’s datastore editor, the table is there, but Credits aren’t. I’m probably doing something wrong here.

local DataStoreService = game:GetService("DataStoreService")
local Currency = DataStoreService:GetDataStore("Currency","global")
local currencyTypes = {}

local saveData = function(player)
	for _, currency in pairs(player.Currency:GetChildren()) do
		
	end
	
	
	
	Currency:SetAsync("CurrencyTypes"..player.UserId, currencyTypes)
end

game.Players.PlayerAdded:Connect(function(player)
wait(6)
saveData(player)
end)

Edit: Nevermind.

I completely forgot to put in “currencyTypes[currency.Name] = currency.Value” on line 7.

Question though, @EchoReaper
How do developers have leaderboards for kills & stuff like that? I haven’t been able to figure that out yet, but it’s probably super simple.

#10

You have to use OrderedDataStores: https://developer.roblox.com/articles/Data-store#ordered-data-stores

1 Like
#11

Thank you!

On another note, I’m having problems with GetAsync() currently. Pretty much the last thing I’m having problems with and then I’ll be good to go. I intend on running both systems for about two weeks or so, notify players about the integration, and then remove the previous old buggy system.

	credits = Currency:GetAsync(("CurrencyTypes"..player.UserId), "Credits")

The datastore is structured like this:

  • CurrencyTypes
    ---- *Credits
    -----*Redshiftium
    I think I’m doing something wrong here.
#12

Something similar has been asked before. You can check out my response on this thread: Using one datastore to save multiple tables?

#13

Read the post I made in this thread: Using one datastore to save multiple tables?

#14

Neither of those options seem efficient enough to me.

I pretty much just need to be able to view the information for both values, Credits & Redshifitium. Perhaps I’m just not understanding the two solutions. :frowning:

#15

Actually, I played around with it a bit more and I finally figured out how to use it.
Didn’t realize “return fldr” meant it was giving me a folder with the table.convertr.

This does exactly what I need it to! :slight_smile:

#16

Would like to thank @EchoReaper, @Nightrains, @Fm_Trick and @AllenTheNomad for assisting me in rewriting my datastore system. If I could, I’d accept all four of you as a solution, but I don’t have that option sadly. :frowning:

1 Like
#17

No problem bro.

1 Like
#18

One more question:

Is this a good way to save data every minute? Or would this cause the datastore to crash like it is currently?
This would also work alongside a function to save data when a player leaves.

while true do
	
	wait(60)
	for _, plr in pairs(game.Players:GetPlayers()) do
		saveData(plr)
	end
	
end
#19

“saveData” doesn’t complete instantly I’m presuming, so you should do a check whether “plr” is still in-game (i.e. is parented under Players) since you obtain the player list at some point in time before the call happens. In other words suppose every call takes 1 second and there are 20 players in-game, the call for the 20th player happens 20 seconds later, and they may have already left the server (and thus had their data saved through the PlayerRemoving handler).

local Players = game:GetService("Players")

while wait(60) do
	for _, plr in pairs(Players:GetPlayers()) do
		if plr.Parent == Players then
			saveData(plr)
		end
	end
end
2 Likes
#20

Does this not work for Studio?

I have BindToClose() and PlayerRemoving() functions, but my data doesn’t save in Studio.
Is there a reason for this?

Also, my while loop is throwing the datastore resource error for some reason.