DataStores replaced DataPersistence to give developers more control over data saving. While appreciative of this, I don’t understand why DataStoreService doesn’t automatically queue up save requests and instead just drops them if it has hit the request limit. I see “DataStores are evil they always lose data!”, “Why would I switch over to DataStores? DataPersistence is actually usable.”, “Help me with X on DataStores”, Help me with Y on DataStores", etc. DataStores are not easy to use. You can’t use them properly unless you go beyond the default API and make something yourself. In every project I’ve worked on before that took advantage of DataStores, I’ve had to do something similar to this:
[code]
local updateQueue, dataCache = {},{}
while wait(60) do
–save all items in updateQueue and delete entries when saved; if unable to save for some reason (e.g. out of requests), break and don’t delete current entry
return {
set = function(key, value)
updateQueue[key] = value
–update = modified version of set
get = return dataCache[key] or datastore:GetAsync(key)
}[/code]
If a game developer doesn’t have something like that then they will be plagued with data loss due to request limits. This type of module is essential for DataStores to work properly. If a game has any sort of shop or inventory system, the game has to expand on DataStores or there is a 100% chance of data loss. Why is this something we have to worry about though? This seems more like back-end functionality that should happen already. We shouldn’t need to finish the implementation of DataStores just so we can use them, and not even every game developer is capable of finishing DataStore implementation in their games.
All of this stuff should happen behind the scenes and be easy for novice scripters to use. There isn’t anything to be gained from having to worry about request limits and update queues on our side.
I would suggest new methods and functionality added to the old API. Changing how it currently works has the potential to create issues with system already made. I think an additional method or two for the DataStore API would be more beneficial:
Examples of what we would use:
:SetQueue(key, value, urgency) --Urgency is a value between 0 and 10 (or some other range) that will put some requests before others. Call the method with higher urgency for actual purchases and players leaving with data to save. Use lower urgency for autosaves that don’t need to really be saved ASAP.
Note: Async replaced with Queue because all methods return instantly because they are just putting the request into queue.
As you said, since it returns instantly we can get rid of that obnoxious Async naming convention (yes, I know that it is there to denote that it doesn’t return instantly, but you kind of know that already if you’re using it so it’s redundant and annoying to type out).
I have never run into any limit issues with DataStore. I don’t see why other people do, it’s not that hard to optimize your code to make way for these limits.
“It’s not hard to optimize your code to make way for these limits”
As I said earlier, DataStores can’t be used without completing the functionality of them. Thank you for confirming that.
Everyone I’ve seen who is running into data store request limits is either:
A) Trying to call SetAsync to store data for every player at set intervals
B) Trying to store a bunch of data in separate keys instead of a single key
If you are still running into data store limits and you aren’t doing either of those things, let me know and we can figure out a solution.
Can confirm you need the knowledge to set up a Cache system to fully use the power of DataStores.
If you don’t you will hit the Limit, as I did. However, now that I have implemented a Cache based system I don’t hit the limit anymore and have even added more datastore related stuff and don’t run into limits.
[quote] Everyone I’ve seen who is running into data store request limits is either:
A) Trying to call SetAsync to store data for every player at set intervals
B) Trying to store a bunch of data in separate keys instead of a single key
If you are still running into data store limits and you aren’t doing either of those things, let me know and we can figure out a solution. [/quote]
A global auction market where players can put items up for sale / buy items (each requiring a datastore request to purchase / post the item) would quickly exhaust the request queue very quickly as players usually make transactions in bulk (speaking from experience with WoW and Runescape)
Updating a player’s bullet inventory each time a bullet is shot (DataStores shouldn’t be a tool exclusive to experienced scripters – anyone should be able to use them. This is ROBLOX. If the updates queued then there would be no problem with this.)
Updating the global leaderboard each time someone gets a kill/dies
DataStores are a replacement for DataPersistence. They should be easier or as easy to use as what they replaced, and in this aspect they come up short. There is no denying that ROBLOX developers in general have a hard time with DataStores – that shouldn’t happen.
If you consistently violate the data limits queuing won’t help you. It will just make the problem less apparent - instead of failed writes you’ll get growing queues and writes that take way too long to complete.
E.g. you can’t solve the “updating player’s bullet inventory each time a bullet is shot” by queuing every single write.
change goes to cache
change goes to queue
fire bullet
change goes to cache
change goes to queue
fire bullet
change goes to cache
change goes to queue
And then a minute later that bullet count in the queue is saved to the DataStore. That’s how I’ve done it in my past projects, and that’s pretty much what you have to do if you want to save the bullet count to the DataStore. I’m not talking about a queue on the network where the server holds requests and then sends them when it has available requests – when I say queue I’m referring to keeping a local copy of the data and then being able to modify that without any request limit whatsoever and then every minute or so that new value in the queue is updated in the DataStore.
There are certainly cases where that wouldn’t be desirable, so instead of replacing SetAsync and UpdateAsync with that queue functionality, Set and Update could be added as methods that automatically use the default queue system. It would be much more user-friendly (novice developers wouldn’t have to worry about request limits), we could ditch that obnoxious async naming convention most of the time, and if it wouldn’t break any existing functionality.