Introducing MemoryStore - High Throughput, Low Latency Data Service!

Ephemeral data stores, at last! I can see so many opportunities being opened up. Clans in an FPS or streets game, maybe a mod workshop for community created content in an RPG, or hell, even just being able to create global chat networks.

This is perfect.

5 Likes

This must be a dream, this is one of the best updates I have ever seen come to Roblox. I am blown away by the incredible groundbreaking updates created by Roblox staff, and MemoryStores have exceeded my expectations.

I can’t wait to see what’s next.

8 Likes

Could you explain the use cases for both #1 and #2? Why they are important to you? It’ll help us understand the needs before deciding the future improvements.

7 Likes
  1. Max expiration is 30 days.
  2. Pls see Q2 in the FAQ section. We’ll consider adding it in the future.
11 Likes

It’s already been reported as a bug multiple times in the past to no avail. I’ve been left hanging for ages now. I was hoping that Roblox was finally announcing a fix for the issue! It’s a major one! :slightly_frowning_face:

4 Likes

Ah, I missed that part, thanks!

4 Likes

Awesome update!

Limit clarifications:

64KB + 1KB ⨉ [number of users]
When users join the experience, the additional memory quota is available immediately. When users leave the experience, the quota is not reduced immediately; there’s a grace period of 24 hours before the quota reevaluates to a lower value.

Imagine that 10 users join and soon leave, followed by a different set of 10 users joining and leaving (so that the experience has had 20 users join but never more than 10 users online at once). Is the the storage limit for the next 24h or so 74 KB (64 + 1*10) or 84 KB (64 + 1*20)?

I expect a KB is 1024 characters? How much overhead is there per key/value? If we theoretically put all the contents of a MemoryStore in a table and then JSON encoded it, would the length of that string be equivalent to the current experience data usage?

[Edit: I answered my last paragraph’s questions in one of my posts below where I discuss many limits I’ve discovered, including how the maximum value size is 1kb, not unlimited!]

9 Likes

Is this eventually going to make OrderedDataStores obsolete? If we have the datastore itself be a single source of truth throughout all servers, could we simply have a queue that has a priority set by, e.g. the amount of coins they have and the loop through the first 50 in effect making a more customizable and quicker ordered store.

(side note: Will this be supported in cloud scripts? That would be heavenly)

6 Likes

Really impressed with the quotas actually. It looks like I can probably comfortably use this for a server browser + multiple other features. That’s really awesome.

10 Likes

Is is possible to renew the expiry date of an entry? For example, I may set the expiration time of a key to be 24 hours, but 20 hours later decide that the entry should last another 24 hours (for a total of 44 hours). Is this possible, or do I need to query the current value then set it again with a new expiration time, hoping that it hasn’t changed in the meantime.

4 Likes

For now you should probably just make a new memory store with the memory value.

3 Likes

This basically takes a huge strain off relying on MessagingService to transmit data between servers. Now, instead of transmitting data through MessagingService, we can use it to alert a server that a value has been updated.

6 Likes

Oh my God, I needed this so bad for my dueling game, as I wanted for cross-server joining, where users could be put in the same match even if they are in different servs. Thanks!

2 Likes

Ordered datastores are more of used in leaderboards, and if their record is reset in 30 days they will be VERY mad.

2 Likes

I found a bug (to let those wanting to use this service immediately know, since I can’t make a public bug report - though I’ve just sent a message to the Bug Support group so hopefully they’ll publish the report later…)

If two UpdateAsync requests run at the same time when the initial value of the key is nil, neither request will be executed a second time.

Sample code:

mss = game:GetService("MemoryStoreService")
test = mss:GetSortedMap("Test")
s = os.clock()
timeout = 3
task.spawn(function()
	print("spawn update", test:UpdateAsync("0", function(v) print("spawn received", v) return (v or 0) + 2 end, timeout))
	print("spawn", os.clock() - s)
end)
print("main update", test:UpdateAsync("0", function(v) print("main received", v) return (v or 0) + 1 end, timeout))
print("main", os.clock() - s)

Sample output when it goes wrong:

spawn received nil
main received nil
spawn update 2
spawn 0.1831044999999
main update 1
main 0.1833445999946

(The rest of the time, one transform function will finish executing before the other one starts.)
Once the value exists, it seems to work as expected.

2 Likes

Just tested it: you can refresh the expiry with UpdateAsync (you can just return whatever the current value is in the transform function) or SetAsync (assuming you already know the value).

1 Like

This is amazing. Thank you for listening.

This is pretty cool too. It’s nice that I don’t have to do it myself.

This is also amazing. I was scared about this but, seems like we’re in the clear.

The quotas are actually hella impressive. Doesn’t feel as limiting.

1 Like

It would be sycned with the actual data stores as I mentioned in my post (or I implied it atleast)

1 Like

DataStores and MemoryStore will stay separate data spaces. OrderedDataStores will not become obsolete as a result of MemoryStore but it may help you create periodic leaderboards more easily. Some experiences may choose to base their leaderboards in MemoryStoreService alone which is fine!

Just do be mindful of the per-experience quota and that you leave enough room for other use cases of MemoryStore if you base your leaderboards in MemoryStoreService. The quotas are very generous but if you abuse them too badly you might find yourself running out.

Always remember to remove data you no longer need, very important point to using this service.

5 Likes

This is actually really cool. A few months ago, I made a server list system similar to how dedicated server browsers work. Instead of using my frankestein data store + messaging service spaghetti code, I can finally use one store to save both volatile and static data and have much shorter server matchmaking startup times.

1 Like