Recent Datastore Issues

Help and Feedback Scripting Support
, ,
#1

I’m not sure if it only affects games with lower popularity, I suppose it would due to the nature of datastore limits. But as of recently (May 2021), Roblox services have become increasingly unreliable. They are constantly timing out, or just working ALOT slower. This is causing me serious datastore problems.

My game averages about 300-600 players online at any given time. I have 6 datastores for player data (I use some heavy compression methods, but even still I separate out different types of data to avoid limitations. So one datastore is for stats, one is for settings, etc). Roblox’s reliability used to be such that players could load in, play the game, leave (at which point they’re data is saved) and everything would save instantly no problem. I also saved item data anytime a purchase was made.

Starting this month (May 2021) this just stopped working… Players would take an extra 5-10 seconds to load in, and they would start dropping save data left and right. The datastores weren’t hitting a limit, they were just too SLOW. Players would leave a server, and hop to a new one, before their data saved in the old server. Leading the game to load in old data, they would report lost progress, and when they left that server, it would overwrite permanently.

So in order to fix it, I started saving data periodically, and if the pcall for saving failed, it would yield, and then try again a second or 2 later. I hoped this would make it so that when players server hopped too quickly, they’d only lose about 15 minutes of progress tops. This made things much WORSE.

Now I’m constantly hitting the call limit on my datastores, and it’s still taking years to save when it doesn’t error. I even have a condition to save data when the game closes, just to ensure everything is saved, and even that’s not working. Someone bought a product in a private server, and the game never saved the purchase before the server shutdown when they left.

How am I supposed to fix this? It’s not like I can just call the datastores less, it’s the slowness of the datastore that’s the core issue here.

TL:DR DataStores are too slow, nothing I can do when players server hop, or try to save things in a private server. Any ideas?

2 Likes
#2

Datastores are not too slow, you are probably handling it poorly.

My tip is to use ProfileService, because it firstly makes saving data for players super easy and secondly handles all the queueing and loading and saving, so you don’t have to think about it.

Or you can make your own Module to handle datastore calls.

1 Like
#3

it would be easier to use profile service then to make ur own module

1 Like
#4

Looking over the ProfileService API it appears to be prone to the same issues I’m already having. It’s core methods run on Roblox’s Datastorage API like mine, and DataStore2’s. So I can’t really see how it could operate better or faster than mine already does.

#5

It’s not prone to the same issues you are having. It has an automatic queue and it spreads out the calls evenly depending on the players in the server so it never exceeds the limit.

loleris said himself (creator of ProfileService), each profile makes a little over 2 UpdateAsync calls per minute. You are allowed 60 + numOfPlayers * 10, so if you give each player 1 profile you will never exceed the limit.

#6

It’s prone to the same issues, because of it’s queue. In fact it’s queue would make things worse for server hoppers.

I have 6 datastores per player. Let’s assume ProfileServer will/can scale for that. So because of it’s queue it’d save on average 2 players per minute. Players in my game usually server hop because of 2 reasons. 1. Someone in the game is better than them, making it hard for them to dominate. 2. It’s a small server and they want more people to play against. Because of those two reasons it leads to players server hopping in waves.

ProfileService has no support for communicating between servers, so for some of these people their data won’t save in the old server (assuming it’s even still open) well before they join a new one BECAUSE of the queue.

#7

Now aside from that, I noticed ProfileService yields 7 seconds between retries, and I looked at roblox’s API and noticed the write limit per player is once every 6 seconds.

So mine was trying once per second, that would explain the reason why I’m getting limit warnings. So I can fix that problem easily, but it doesn’t fix the overall issue.

#8

I don’t know what you want from me, you wrote this topic, I gave you an answer. With 5 seconds of reading ProfileService (not ProfileServer) API you tell me it’s not good.

I shouldn’t of said anything about the queue (the queue was a recent addition, the reason it’s there is to stop the default queue from giving warnings)

The fact that you have 6 datastores per player is already a problem, you have to make 6x the calls if you would use one datastore per player.

ProfileService does this and abstracts it, so that it always loads and saves the data with haste and without data loss.

You clearly haven’t done research on ProfileService, because one of ProfileService’s biggest selling points is it’s session locking. Do research and learn what race conditions are.

If you don’t want to use it, that’s fine don’t use it, but don’t argue about it with me when you have done 5 seconds of research.

#9

I’ve come up with a solution with my own DataStore to prevent issues; essentially, there is a table named BlacklistedPlayers (or bp for short) and whoever is on that list cannot save their data. When a player joins the game, they are automatically inserted in that list. In order for them to get off the BlacklistedPlayers list, the following have to be checked:

  • Their data folders are loaded in
  • Their data from GetAsync has loaded in

If any of them failed, then they can’t have their data saved in that session and are prompted to reload or try again later.

1 Like
#10

It session locks based on previously saved data. It saves the time and session id within it’s data packet. I understand how session lock works, and I’m telling you it WOULDN’T work under say this example:

2 players leave the game… the queue is now occupied. Another player buys a product, to illustrate how much of a problem it is, let’s say it’s one of the really expensive ones. Game tries to save data due to the purchase, it’s queued. Now that purchasing player, server hops. New server loads their data in, because the last save was recent enough that it’s not a dead session lock.

Boop, that player now has lost some progress and alot of robux.

I know it doesn’t communicate between servers, at least not effectively, because the entirety of ProfileService calls only RunService and DataStoreService. And it specifically mentions the dead session lock timer of 30 minutes, and how that can be unreliable.

I’ve always kind of had this issue for advocates of Datastore2, and now ProfileService. It’s equivalent to:

Me: “I can’t drink milk because I’m lactose intolerant”
You: “Oh just drink a milkshake, it’s better than milk”

Arguably that’s true, but they’re still made from the same base product, which is where the problem lies. A better solution would be “try drinking lactaid”. Which takes the base issue, finds the problem with it, and solves it.

#11

That’s not a bad idea, but doesn’t that have a negative bias against saving data, instead of loading it?

Like the blacklisted players can still load in data, but in one of their previous sessions it fails to save?

#12

I’m Recoil 30 Level player And I don’t have any problem with databases

#13

I think what I’ll do instead, is alter my SetAsync function (my own system I use has a function called SetAsync to make the calling easier), so that it uses UpdateAsync to check old data. This will make saving slower of course, since it’ll need to decompress the old data to verify it. I’ll also store the data that was first loaded in somewhere. That way I can accurately measure progression. So whenever it saves, it’ll save progression onto the old data. So if someone levels up to say, level 12. Then server hops and it loads in level 11, if they level up to level 12 again, it’ll save as level 13. I’ll do something similar with purchased items.

This could still lead to me getting complaints about losing progress, but at least the real damage and data loss will be minimal.

#14

Alot of people don’t, but if you look in our #bug-reports channel, there’s quite a few that do.