Introducing MemoryStore - High Throughput, Low Latency Data Service!

Basically, the total amount of requests that can be made in total is * 1000 + 100 ⨉ [number of users]. My requests to, however, MemoryStoreService:GetSortedMap(“Matchmaking”) will be limited to 100k requests? Or only a key into that map will be limited to that number? Sorry if I didn’t understand this correctly.

EDIT: Also, there seems to be a bug when Queues have 0 invisibility timeout. After trying to remove the items after getting them, the item would still remain in the memory. This thing doesn’t happen if invisibility is something greater than 0 seconds. Is this intended?

local MemoryStoreService = game:GetService("MemoryStoreService")
local Queue = MemoryStoreService:GetQueue("MATCHMAKING", 5)
Queue:AddAsync({
	Id = 500
}, 300)
Queue:AddAsync({
	Id = 500
}, 300)
Queue:AddAsync({
	Id = 500
}, 300)

local Data, Id = Queue:ReadAsync(10, false, 0)
print(Data)
Queue:RemoveAsync(Id)

task.wait(10)

local Data2, Id2 = Queue:ReadAsync(10, false, 0)
print(Data2)

EDIT 2: MemoryStoreQueue:ReadAsync()'s waitTimeout parameter states that if you can wait indefinitely by putting the value to -1. Is it truly indefinitely or is there a limit?

I have a question. I want to use memory store for matchmaking in my fps game, but I am worried about possible latency issues between player and server. Memory store works global, isn’t it? If I teleport some player to server which there is a high delay between the player and the server, this will affect gameplay quality. Is there a solution for this or am I lacking knowledge about it?

The expiration parameter of MemoryStoreSortedMap:SetAsync() and MemoryStoreSortedMap:UpdateAsync() desperately needs some clarification.

On the Developer Hub, it states that the expiration parameter takes an Int64 value, which can technically get up to 9,223,372,036,854,775,807

When you set a value that high, no error is thrown but no value is set either (very cool, not frustrating at all). When you set a somewhat lower value however, you get this error:

image

This is still a lie however, by a few decimal places. After testing different values, I found that they actually cap the expiration value to 2,592,000, not 2,592,000,000. It was only after I put in a value less than 2,592,000 that it actually started saving my data.

Poor documentation of new features is happening way too often. Love the new stuff, but it’s critical that we have the ability to learn how to use it properly through official channels.

11 Likes

Anything about the memory size limit that’s in bold still being wrong? This issue still seems to be ongoing and values still seem to be capped it seems as I am getting “The provided value is too long” for just over 1kb in a sorted map which makes this incredibly difficult to work with because 1kb is nothing.


Here’s an image, as you can see the error occurs when the length of the entire data is just over 1kb (1024). This is affecting a lot of people that use my module as the documentation is incorrect when it makes this claim:

I know that the entire memory isn’t over 65kb as well. There’s maybe 2kb in there at max.

Here’s also confirmation. I made a new game with 0 memory usage. And I made a test script and confirmed these suspicions:

It’s fine on the first one, which is (just) under 1kb. But it errors on the second one which is over 1kb. And the total memory limit with 1 player is 65kb, which this is definitely not hitting.

And before you say it’s because I’m in studio, it’s not:

Simple test code, literally just a string of characters
local MemoryService = game:GetService("MemoryStoreService")
print("Memory limit size test:")
local test = ""
for i = 1, 1020 do
	test = test .. "1"
end
print("Length of data being added: " .. tostring(#test))

local map = MemoryService:GetSortedMap("Map")

map:RemoveAsync("Test")
map:RemoveAsync("Test1")

map:SetAsync("Test", test, 86400)

test = ""
for i = 1, 1100 do
	test = test .. "1"
end

print("Length of data being added: " .. tostring(#test))

map:SetAsync("Test1", test, 86400)

Please either fix the memory size limit on sorted maps or document it correctly.

8 Likes

I could be wrong but I think the error message for exceeding the expiration time is wrong.

I tried to use that exact time and it wouldn’t let me. That’s 30,000 Days rather than 30?

Nice, This is going to be so useful. Question: Does MemoryStore Data save accross places? And how can I use AddAsync? Do you type in the name of the value, The value itself then its priority then its experition? Because I need this to temporarily store player data for a game.

so don’t use Queues but SortedMaps

This service is useless for matchmaking. You’d expect to implement this using Queues, but this currently has no functionality to remove a specific item. You can’t account for players exiting the queue.

You could probably implement a hacky solution using SortedMaps, but this seems counterintuitive and seemingly makes Queues futile for matchmaking systems, which is what I was personally excited for.

Not sure if this was an oversight or some sort of limitation but seeing such a basic crucial feature missing is really disappointing for such a seemingly powerful service. :frowning_face:

3 Likes

[v1.0.0] MatchmakingService - Resources / Community Resources - DevForum | Roblox look this guy

To be fair, I do use a hacky method using sorted maps, so he isn’t wrong. It would be so much better if we got more control over memory queues.

Perhaps my frustration was a bit too over-emphasized in my original post. It just sucks that this service went through a closed beta period and I’m failing to see a proper way to utilize Queues without all the previously mentioned drawbacks. It just doesn’t make life easier :frowning:

There are still games that are using MessagingService for matchmaking systems, which is honestly quite embarrassing. If it wasn’t for your effort of writing almost 2,000 lines of code, I’m not sure how many people would’ve wanted to put in this kind of effort to make their own system using SortedMaps.

I’m not sure how stress-tested this module is, but I’d assume it’s a great alternative.
Thank you for your contribution :slight_smile:

I’m right there with you, I would much more prefer to use Memory Queues, and as I’ve mentioned in my matchmaking service post, they’re just not in a suitable position to be useful, and it’s rather unfortunate as well. Especially when they added the limit on sorted maps to 1kb from any amount that was within quota… That made my service that much more difficult to maintain. The 1kb limit in sorted maps is completely arbitrary as well and I don’t understand why it was added. I do hope that they add more features to memory queues because right now I would say that they’re pretty useless.

1 Like

Could anyone help with this?

i didn’t said that he was wrong just a link to your works but yes there are too much things to fix with MemoryStore Service like the size limit etc…

Hey @V_ChampionSSR, thanks for the feedback. Can you explain more details about the challenge here? Why removing a specific item is critical to your use case? For matchmaking, we think you can validate if the player has dropped after reading from the head of the queue.

Are you talking about the key length limit? Can you elaborate your challenge and what you want to achieve?

When this service was initially announced, singular SortedMaps did not have a size limit as long as it did not go over the quota. Since then it was lowered to 1kb. I made a bug report about this before the docs were updated:

This just made it very challenging to use SortedMaps because 1kb is not a lot to work with.

As for memory queues, I would just like the ability to control them a bit more, such as getting specific values from the queue, or just added a Skip parameter to ReadAsync that will skip the number you give it and only return the values after. There also needs to be a way to remove specific items from memory queues.

The example for memory queues you guys give is a matchmaker. However, if a player leaves after they queue, there’s no way to remove them unless you read them first, which is rather inefficient because you also can’t just remove a single value while reading, it’s either you remove everything you read or nothing.

3 Likes

I will also share my opinion on this topic.
SortedMaps are fine (except the 1kB limit), but Queues need to be reworked from scratch.

Here are the API members they should have:

:AddAsync(value: any, expiration: number, priority: number?)


This function stays the same.

:RemoveAsync(value: any)


Removes all items that are equal to value.

:UpdateAsync(transformFunction: (oldValue: {any}) → {any}, expiration: number)


Retrieves the queue and lets you update it via a callback function. The queue is a sorted array.

:GetRangeAsync(count: number?, direction: Enum.SortDirection?): {any}


Returns a sorted array of queue items. count specifies the number of items (if nil, all are returned). direction specifies whether it starts at the end of the queue (descending) or at the beginning (ascending). The default is Enum.SortDirection.Descending.
It would be nice if this function didn’t have a limit of how many items it can return (or at least bigger than 300).

.Added / .Removed / .Changed


We need some sort of event when the queue got updated.


Please make ReadAsync deprecated and remove it after some time. It is extremely inefficient.

I hope that you will realize at least a few of these functions. If you need a more detailed explanation, feel free to ask me. :slightly_smiling_face:

1 Like

Hello. Thank you for the response. As was mentioned by @Extuls and @steven4547466 there is no efficient method including the one you have described. A thread that has more detail can be found here.

As for your question:

Let’s use ReadAsync for example, 20 results. 10 of those results are players that have left the matchmaking queue. That leaves us with 10 valid results. How can I remove those 10 invalid results, so that I only get 20 valid results?

There is no way to tell which results are valid or not. Even if there was a way to tell if a player was “dropped”, RemoveAsync would only let you remove all 20 results from the queue.

The reason most of us would prefer to have a specific amount of valid results from a Queue is so that we can make sure that there are enough players to start a match. Not to mention there is also that chance that TeleportService may fail, and some matched players aren’t able to make it. Being able to account for the margin of error would be incredibly useful.

All of this is not only to make life easier for us developers trying to make a matchmaking system but, will greatly improve user experience. We want players to be matched as efficiently and swiftly as possible.

What would be a potential solution is that instead of AddAsync() returning void, it could return an id that you can use in conjunction with RemoveAsync(). That way, if a player leaves the game, the queue, or whatever else we could handle this logic ourselves.

1 Like

Thanks for the details. Can you share more about your use case? What type of data you’d like to store in sorted map? What’s max size do you expect?