Help me improve the code for reading and saving data

When I was testing the game, I noticed that sometimes the data will not be saved because the write limit has been exhausted. That’s why I wrote this script.

Provide an overview of:

  • What does the code do and what are you not satisfied with?

    • The script adds all incoming data to the table, this allows you to:

      • Read data quickly
      • Bypass write and read restrictions
      • Reading new data before writing it to the DataStore
      • Data will not be lost if the DataStore does not write the data
      • If the DataStore does not write the data, the script will try to write the data until it writes
        Other data will be written even if the script fails to write any
      • Write only new data, rather than storing old data one by one.
      • Less dependency on constant access to the DataStore (this works if the Datastore writes data from time to time)
      • Providing a faster, more reliable, customizable alternative to UpdateAsync
      • Data that is not in the table will be automatically read from the DataStore
    • Error messages and about data not suitable for writing, saving data about errors, in particular time, error, DataStore, key. Ability to get a table with errors

    • You can add settings and player’s UserId to the data

  • What potential improvements have you considered?

    • Write player data only when player exits
    • Adding new features
    • Using DataStore 2
  • How (specifically) do you want to improve the code?

    • Make it more optimized
    • Remove bugs in code
    • Improve the while loop if possible
local module = {}

local dss = game:GetService("DataStoreService")
local Players = game:GetService("Players")

local ErrorData = {} -- [time] = {["Name"] = "", ["DDS"] = "", ["Key"] = key
local DataForSave = {}
local Data = {}
-- ! Check the data that is returned, because it may be an error

function WriteErrors (Error, DDSName, Key, Time)
	
	if Time == nil then Time = os.time() end
	
	table.insert(ErrorData, Time, {["Name"] = Error, ["DDS"] = DDSName, ["Key"] = Key})
	
	warn(Error)
	
end


-- Read
function module.ReadEvent (dssName, key, ReadAgain)  -- DataStoreName, Key, Request from the DataStore (true/false)

	-- Checking the correctness of the data
	if typeof(dssName) ~= typeof("") and key == nil then		WriteErrors("The name of the DataStore is not a string and the key is nil", nil, nil) 		return "The name of the DataStore is not a string and the key is nill" end
	if typeof(dssName) ~= typeof("") then		WriteErrors('DataStore name is not a string', nil, key) 		return  "DataStore name is not a string" end
	if key == nil then		WriteErrors('Key is nil', dssName, nil) 		return "Key is nil" end
	
	print("Reading: " .. dssName, key)
	
	if ReadAgain == true then
		local success, Value = pcall(function()
			local GDS = dss:GetDataStore(dssName)
			return GDS:GetAsync(key)
		end)
		
		if not success then		   WriteErrors("Failed to read data from DataStore, error: " .. Value, dssName, key)			return "Failed to read data from DataStore, error: " .. Value end
		
		return Value
	else
		if Data[dssName] == nil  then Data[dssName] = { ["Keys"] = {}, ["Warns"] = 0, ["Errors"] = 0 }  end
		
		if Data[dssName]["Keys"][key] == nil then

			local success, Value = pcall(function()
				local GDS = dss:GetDataStore(dssName)
				return GDS:GetAsync(key)
			end)

			if not success then WriteErrors("Unable to read data from DataStore to write to table and return data, error: " .. Value, dssName, key) return "Unable to read data from DataStore to write to table and return data, error: " .. Value end

			Data[dssName]["Keys"][key] = { ["Data"] = Value, ['Time'] = 0 }

		end
	end
	
	return Data[dssName]["Keys"][key]["Data"]

end


-- Write			DataStoreName, Data, Key, {"UpAs" - Save with function (data), "Fun" - the same as UpdateAsync(), but takes data from a table or accesses the DataStore, "InAs" - IncrementAsync()}
function module.WriteEvent(dssName, data, key, tegs, userIds, options) 
	
	--Checking the correctness of the data
	local NonCorrectData = ""
	
	if tegs == nil then tegs = {} end
	if typeof(dssName) ~= typeof("") then		NonCorrectData = NonCorrectData .. 'DataStore name is not a string, '  	end
	if key == nil then		NonCorrectData = NonCorrectData .. 'Key is nil, '  	end
	if typeof(userIds) ~= typeof(8) then NonCorrectData = NonCorrectData  .. "The fifth variable userId must be a number, i.e. the player's Id, " end
	if table.find(tegs, "UpAs") or table.find(tegs, "Fun") then	if typeof(data) ~= typeof(function ()end) then 		NonCorrectData = NonCorrectData .. "Second variable data should be a function or remove the UpAs/Fun tag" 		end	end
	
	print("Writing: " .. dssName, data, key, typeof(tegs), userIds, typeof(options))
	
	if NonCorrectData ~= "" then	 WriteErrors(NonCorrectData, dssName, key)	return NonCorrectData end
	
	if Data[dssName] == nil  then Data[dssName] = { ["Keys"] = {}, ["Warns"] = 0, ["Errors"] = 0 }  end

	if Data[dssName]["Keys"][key] == nil then

		local success, Value = pcall(function()
			local GDS = dss:GetDataStore(dssName)
			return GDS:GetAsync(key)
		end)

		if not success then WriteErrors("Failed to read value:" .. Value) return "Failed to read value:" .. Value end

		Data[dssName]["Keys"][key] = { ["Data"] = Value, ['Time'] = 0 }

	end

	if table.find(tegs, "Fun") then
		local success, value = pcall(function()
			return data(Data[dssName]["Keys"][key]["Data"])
		end)
		
		if success then
			data = value
		else
			Data[dssName]["Errors"] += 1
			WriteErrors("The function threw an error: " .. value, dssName, key)
			return"The function threw an error: " .. value
		end
		
	end

	Data[dssName]["Keys"][key]["Data"] = data
	Data[dssName]["Keys"][key]['Time'] = 0
	
	DataForSave[dssName] = { ["Key"] = key, ["Data"] = data, ["tegs"] = tegs, ["userIds"] = userIds, ["options"] = options }
	
end


function module.ReturnErrors () return ErrorData end


-- Save
delay(0,function()
	while wait(1) do
	
		for i in pairs(DataForSave) do

			wait(Data[i]["Keys"][DataForSave[i]["Key"]]["Time"])

			local dssName = i
			local v = DataForSave[dssName]
			local tegs = v["tegs"]
			local data = v["Data"]
			local key = v["Key"]
			local userIds = v["userIds"]
			local options = v["options"]

			local pcallSuccess =  nil
			local pcallError = 0

			if table.find(tegs, "UpAs") then

				local success, Error = pcall(function()
					local GDS = dss:GetDataStore(dssName)
					GDS:UpdateAsync(key, data)
				end)

				pcallSuccess = success
				pcallError = Error

			elseif table.find(tegs, "InAs") then

				local success, Error = pcall(function()
					local dssPlus = dss:GetDataStore(dssName)
					dssPlus:IncrementAsync(key, data, userIds, options)

				end)

				pcallSuccess = success
				pcallError = Error

			else

				local success, Error = pcall(function()
					local GDS = dss:GetDataStore(dssName)
					GDS:SetAsync(key, data, userIds, options)
				end)

				pcallSuccess = success
				pcallError = Error

			end

			if pcallSuccess ~= true then	ErrorData("An error occurred while recording: " .. pcallError, dssName, key) return end

			if v == DataForSave[dssName] then DataForSave[dssName] = nil end

			delay(0, function()
				for i = 6, 0, -1 do
					Data[dssName]["Keys"][key]["Time"] = i
					wait(1)
				end
			end)

		end

	end
end)

return module 
Changes
  • Detailed error information
  • Error reading capability
  • Incoming data is now better checked for errors

Reading time (arithmetic mean per 100 requests):

  • Reading from DataStore (first reading) - 0.05117469999997411 (During the test, each time the reading time increased from 0.6 to 5.1)
  • Reading from a table (second reading) - 0.0001611970004159957

I will be glad if you help me improve the code

If you use the script - write in the comments, I will be interested

Would you use my script?

  • Yes
  • I don’t use data store
  • Scripts have already been written not for the script
  • I am already using another script
  • If there are more features, then yes
  • No

0 voters

1 Like

NEVER give the client write access to anything, it will be abused by exploiters! DataStore writing should only be controlled by the server.

In all, I feel like your trying to reinvent the wheel here. ProfileService and ReplicaService are both great modules that were designed to work together to give you secure datastoring and easy dynamic read access to clients; both of which fit your needs here.

1 Like

Not a bad approach but i think directly using remote events is definatly a big no no, for sure turn it into a function that can be use though, perhaps a module would be better for you.

Aside from that I noticed that failed attempts are put back into the que which ok yes does allow for repeated attempts but it also clogs up your saving que, it would be better to have a priority rating when saving from ques to ensure important data gets saved first.

The warning system is a nice touch but i think adding checks upon pcal fail e.g if something like a model is trying to be saved you can give a message like “Cannot Save Models, try saving properties instead”

It looks like the script is tailored to your scripting format which is good practice and nice to see but id say it is worth adding to and rewriting when you learn more in the future, i practice redrafting myself and it has helped me improve my skills and my scripts a lot.

Keep at it though, just dont forget that dss stores and identifiers are both string, no matter how you get the string, keeping this in mind will get you further when thr need comes for it.

I am very confused in the DataStore and in these scripts. I will definitely check out these scripts. The script uses a BindableEvent which can only be accessed by the server.

Thanks for the feedback, I’ll definitely listen.
My script has a table in which all data is written that will be saved in the DataStore and will be deleted only when overwritten. This will allow you not to wait for the previous versions to be saved and will allow you to read information very quickly directly from the table without using the DataStore and will not face restrictions. The read and write functions write these values to the table, and then the while loop writes them after 6-0 seconds.
If I use ModuleScript, which returns functions to the script. How will the script write to the ModuleScript table? Or is there something I don’t know?

It would work for the base of your dss i.e save and load. This just means it can quickly be accessed from other areas in your game and makes code neater, you would still need to properly manage stores on the scripts that call upon the base functions though. I always keep a script with a few functions for accessing my dss module and copy and paste them where needed.

Using a module isnt that important if your just using basic data saving though, unless your making a part saving function or something then its a must use.

I would personally have a local table for storing data
then when a player joins, read data from the datastore and put it in the table (and remove it when player leaves)
when a player wants the data, return it from the table
every 15 seconds try to save the players data (and when player leaves)
that way the client cannot to control when the datastore read’s or write’s

for me it “just works”

Now I understand how Module Script works, I’ll go improve the script.

I also thought about this, what would be saved on exit and deleted from the table. The client has no control over the script. BindableEvent only works from server to server and from client to client.
https://developer.roblox.com/en-us/api-reference/class/BindableEvent