Tips on saving large amounts of data

My game has been experiencing issues with DataStores since it was released. Currently, we save our user’s info using a large table containing various different values and one nested table for vehicle information.

This is proving to be problematic, as we seem to exhaust DS requests quite frequently. The game autosaves every 10 minutes (saves are staggered over this timeslot) and when a player purchases a vehicle. Data loss reports have rapidly increased lately to a point where the game is borderline unsustainable. I’m intending to create a whole new data persistence system for an upcoming update and I’m looking for tips on how to make this as robust as possible. We currently run our game on a single 80-player server which is usually full for 1-3 hours at a time.

The data we’re looking to save for each player is as follows:

-The player’s character’s first + last name
-The player’s character’s gender
-The player’s health, hunger and thirst
-The player’s wealth in three different stores - welfare, bank and wallet.
-All of the player’s vehicles’ information such as model, colour, plate, registration date etc along with all items stored in the trunk
-The time remaining of a player’s jail sentence, if any
-The contents of the player’s inventory
-The player’s in-game house address / reference, if they own one
-The player’s reputation with various NPCs, stored as an integer

I also need to save the following information in a separate datastore so that it can be accessed when a player is absent from the game:

-Arrest records containing various fields of information
-Citation records containing various fields of information
-Information about each vehicle linked to the plate - while this is saved in relation to the player’s actual car data, a copy needs to be stored elsewhere to allow the information to be fetched without knowing who owns the vehicle

…So yeah, a lot of data. We’re currently using Trello to store non-player linked data. While useful, this presents HTTP throttling issues, so I’d much rather store a majority of it in-game. I’ve looked a fair bit into this issue and seen a lot of people suggesting DataStore2, but I wouldn’t know where to start with using it. I’ve taken a look and had a go but it was fairly confusing, especially in relation to the data I need to save.

Any advice is appreciated.

I would for sure suggest you ProfileService, I find it very handy for saving tables etc.

Look on it here: Save your player data with ProfileService! (DataStore Module)

3 Likes

You could try serialising stuff if you’re concerned about the amount of data you’re storing, but other than that, I suggest you only call DataStoreService when the player leaves the game, if you aren’t already.

I’ll look into this, I’d never heard of ProfileService before - it sounds to be exactly what I need, but I’ll have to experiment. Thank you.

No problem glad that I helped good luck with your game and fixing this issue I was in same situation as you ;).