Hello everybody, I have been trying to think what is the best way of saving data to very large datastore, I expect the datastore size to be near the datastore limit.
So I was currently thinking about two methods.
Method
Keys will be the clans UUID and I can just index the clan easily by doing data[UUID]
local data = {}
data["UUID"] = {
Name = "",
Members = {},
}
Method
The whole datastore is going to be an array - indexing of the clan is going to be done by pairs loop
local data = {}
table.insert(data,{
clanId = "UUID"
Name = "",
Members = {},
})
Please let me know if there’s better approach to do so or which method should I use.
How willing are you to wait for DataStore V2 to face stable release or completely set up your DataStores with experimental features with a way to handle cases of potential failure? Additionally, are you willing to not query DataStores frequently and only if you need them?
If you’re willing to run these assumptions then save clan data in separate keys and use ListKeysAsync if you want to get a list of all existing guilds in your experience. As for how you should display them, you’ll probably want to save on request budget and only display them if the user searches for them or you have, for example, a guild recommendation feature.
Any static information should be cached on the server. If you’re caching any changing data, you may want to let the player know when you’re stalling for more request budget or when the cache was last updated so they understand if they’re working with updated information or not.
Generally you should not collate data like this into a single key. Keys should be unique to a data set.
My project is far from release so I am willing to wait for a bit.
I just realized it after I wrote this topic that saving under one key won’t be a good idea.
My idea is to update the info about the clan on change so for example I want to change clan’s name and then I will call updateasync. The thing with this is the probability of abuse because more than one player are going to be able to change clan’s info but this can be probably prevented by storing clan’s UUID to table and set change info cooldown.
All active clans in the server will be cached in table and Info replicated via folders in replicated storage with attributes.
Additional question:
Am I going to be able prevent “duplicated names” of clans with the ListKeysAsync? Because this is probably my biggest concern right now.
My concern as well since I’m working on a guild system as well: you probably might have difficulty here unless you actually fetch clan data since ListKeysAsync only returns keys in your DataStore at the moment and additional querying options are still in development. In my case, I intend to use guild names as the identifier and it can’t be changed without creating a new guild. Not the best user experience but I’ve got to work with what I have access to and this seemed the most sound.
That’s unfortunate but I don’t really want to use guild names as a key but as I am seeing your reply there’s no probably no other efficient way to do it.
My original idea was to make second key in datastore with the name of the clan that will be holding the clan’s UUID and upon rename I can just Remove the key from datastore and create new one.
Check for if the guild name exists would work like that:
local guildName = "somethingx1"
local guildNameRequest = dataStore:GetAsync("@"..guildName) --// the @ is there to prevent conflict with randomly generated UUID
if (not guildNameRequest) then
return true
else
return false
end
I don’t really get what benefit would this have, because creating new key in datastore is probably faster than caching received data from MessagingService to table and check if anybody isn’t trying to make guild with same name.
But when I am thinking about it now it can be a good idea, but I don’t really know if MessagingService is fast enough for it to be even good.
I don’t really see the issue here since you are going to call the GetAsync and UpdateAsync here only once and the updates are going to be done only per certain time and on change.
Well that’s a good idea but how I would go about it?
Something like this?
local currentlyAddedNames = {}
local function getValidName(guildName)
local guildNameRequest = dataStore:GetAsync("@"..guildName) --// the @ is there to prevent conflict with randomly generated UUID
--// Messaging Service implementation
local MessagingConnection = MessagingService:SubscribeAsync("guildNames",function(message)
table.insert(currentlyAddedNames,HttpService:JSONDecode(message.Data))
end
if (not guildNameRequest and not table.find(currentlyAddedNames,guildName)) then
MessagingService:PublishAsync("guildNames",guildName)
MessagingConnection:Disconnect()
return true
else
MessagingConnection:Disconnect()
return false
end
end
You should connect this when the create Guild menu is opened.
Multiple servers contribute to an experience limit (time between request never mentioned) meaning if two clients created a Guild with the same name, two Get requests would be issued possibly causing one to throttle.
