I own one of those “Leave a note” games (you could play it here), where players could place a note with custom text and It would sync across all servers and would stay forever in the datastore.
I currently store each note’s content and cframe (and other data like date, userid, etc) in a large table with every note in the game. Currently, this large note table has became over 1000 entries in length and has become really inefficient to just change one value or add a note.
The datastore structure is like this:
Datastore
NoteData
NoteTable
1
Note Content,
Note CFrame,
2
Note Content,
Note CFrame,
etc
Every time I want to place a note, edit it’s text, or delete a note, I have to get the NoteTable, make the changes, and save it back again. You could see how this could be very inefficient for the server
What would be the best approach to make this more efficient? Do I save each note as it’s own datastore entry? How would I load every note and place it in the server when doing that?
Here’s something you could try. Get all the notes, and store them into a table on the server. When you want to make changes, change that queue. Then you can have another thread that periodically updates the datastore. Say maybe every 5 seconds. This would be less expensive in big servers, but I’m not sure if your data store is set up in a way for you to be able to get all the notes in the game at once.
Yeah that’s exactly what I am currently doing. My question is how I could make this process more efficient. All the notes are stored in a table in the datastore and the table is over 1000 in length. It’s fine when getting all notes at once, but very inefficient when for example making changes to one note only.
Ah, well, the actual process of saving a datastore, it’s impossible to speed up, that’s on Roblox’s end. One little nuance, I would have a system so you know if the datastore has changed in the first place. Every time a note changes, the queue knows that it has to save in the next interval, but nothing changes then don’t save during that period.
If your code isn’t broken, and you’re not experiencing any actual LAG, I think you should be fine. You say it’s becoming inefficient, but is that because you just don’t like thinking about iterating large numbers of time? The server might just be fine.
Also, wouldn’t doing this cause conflicts with multiple servers? What if I place a note in one server and another server saves to the datastore before the server where I placed the note in. It would take the player’s one note away and never place the note because the datastore is overwritten
If you want to avoid you need to have your data stores set up in a way to where you can access one at a time. Not have to retrieve all the data just for one. You should probably look into scopes. For each note you can create one.
Yeah, I am currently. This is one of the reasons why I am posting this thread, to find an alternative method to what I am currently doing. Since my game is small, it never became a problem for me, but if it grows it would become a huge problem
I have eventually solved this issue by storing notes in “chunks”, these chunks are stored as different datastore keys which only contain 300 entries of notes. When the server starts, these chunks are all loaded in at once, appearing the same as before I made this change. When a user creates or deletes a note, it gets the note’s ID which now contains its chunk id and note id inside the chunk, queries the relevant chunk, modifying it, and then sets the datastore.
Here’s a diagram of the new datastore structure if it helps you understand
NoteStore
ChunkAmount = 5
Chunk1:
Notes: [content, cframe, date, userid]
Chunk2:
Notes: []
Chunk3:
Notes: []
…