Rather complex data structuring issue

[Rhaoke]

This data issue is very specific to my game and the way that it has to work, so I will spend a good bit of time prefacing what I am attempting to do.

The project that I have been working on and am nearing completion to is a realtime interactive map. If you have ever played games such as Hearts of Iron IV or Crusader Kings III, you would understand sort of what I’m attempting to do.

The data structure of the game has always been a bit messy, but I managed to make a solid system that was working, until I came across this issue today that is derived from the limits on MessagingService messages.

Let me explain the issue;

To make a solid and persistent data structure, I knew that I would have to use a mixture of MessagingService and DataStores. A lot of the issues from the data structure have been derived from data limits, but this one is the first one that I haven’t been able to circumvent.

NOTE: I am using 1 player servers.

The first issue that I came across was the 6 second limit on writing to the same DataStore key.

My solution to this was that when a server wanted to save (on close), it would ping the other existing servers using MessagingService, and if there were other servers, it didn’t need to save. The reason that it didn’t need to save, is because EVERY action on this map is automatically synced to the other existing server instances for the ‘realtime’ effect.

After this, I didn’t encounter much issues with data limits until now.

Here is a picture explanation of my issue:

You can see towards the bottom the explanation of my issue, and to be honest, I have no idea of how to approach solving this issue.

I realize now that 1 player servers crippled me into this issue, but it’s too far into the project now to change from the concept of 1 player servers.

Does anyone have any ideas of how to approach this?

Any tips would be appreciated.

[kingerman88]

You’re using messaging service to transfer data between servers, which I mean I wouldn’t really do just generally do to all the topics I see about messaging service.

Datastores transfer between servers, so “theoretically” you could keep doing :GetAsync() from the datastores to get the information and have it continuously updated.

Also datastores have a max cap of 4mb

DataStore Data Limit Increase

Since launching Data Stores years ago, the service limited storage to a maximum of 256KB data per key. Since then, games and experiences have only grown to be bigger and more immersive, and developers have had to implement some workarounds in order to save an ever-growing amount of player data. To address this, as of today, we are increasing the DataStores per-key data limit from 256KB to 4MB! Note: Request limits for reads and writes are going to stay the same, as is the per-key throughput li…

[Rhaoke]

The limits on datastore requests are much lower than the limits on MessagingService requests, and additionally datastores do not update in realtime like MessagingService does.

[kingerman88]

Hmm, that is an interesting problem. I think the most probable thing that you will need to do is really relax on definitions. For instance:

Instead of doing this {Town1Money = 100, Town2Money = 200, Town3Money = 300} etc
{100, 200, 300} and then your scripts will need to decompile them
Becoming more efficient on your usage of what needs to be updated in real time and what can be updated every so often.

[Rhaoke]

I appreciate your efforts, but this is not helpful whatsoever. The way the map works, all of the actions need to be replicated for the strategy of the map to be effective.

If one player moves an army, and another player doesn’t see that the army was moved on their screen, the second player is at a disadvantage.

[JigglyWiggles]

Now this is VERY out of my realm of knowledge, but I had an idea. Could very possibly be the dumbest thing I have said, but I’ll say it anyways. Could you use some sort of external program or API? I don’t know exactly what the limits of these are but I am hoping to start some sort of idea or thought process. I guess what I am thinking is you have something that stores this data somewhere else, and then each server updates with that. Like how a router works maybe? Is this even possible with what we have?

[Rhaoke]

It’s definitely possible, but not viable in my situation.

That would require external hosting, which costs money, which I’d rather not spend on this project.

[ThatTimothy]

To reliably do this, you’d need one server to be designated as the “central” server. That server would be responsible for actually saving the world data to the datastore. That way, you dont have to worry about different servers writing at the same time, and all the other servers would send updates to it. However, getting a central server reliably isn’t possible on Roblox as of now. Cloud Scripts may provide this functionality in the future.

I suggest you change the aspects of your games design to be more single server vs multiple servers based, or make the game turn based so that players dont have the “army appeared out of nowhere” issue.

[Rhaoke]

The problem is how deep into the project I am centered around the existing data structure.