MemoryStore HashMap [Beta]

Hi Creators,

We are excited to announce a new Data Structure in the MemoryStore Service - the Hash Map. The Hash Map is a key-value store with no sorting guarantees, making it the ideal data structure for a simple high-throughput data cache. Use this to implement a shared inventory, live auction house, or to store player metadata for your experience (such as an active server list).

MemoryStore Hash Maps have no data structure item limits or data structure memory limits. You are free to store as many items as you want, so long as you do not reach the Universe memory size quota. It does, however, maintain the individual item size constraints (128 bytes for a key, 32 kilobytes for a value). Additionally, it provides the same basic functions as a Sorted Map - GetAsync(), RemoveAsync(), SetAsync(), UpdateAsync() - you can effectively hot swap a Sorted Map to a Hash Map if the ordering of items is not needed for your use case. If you are using the GetRangeAsync() function but not utilizing the ordering of items in your Sorted Map, you can replace it with a call to the Hash Map’s ListItemsAsync() function (this method lists items in a paginated fashion).

In addition to the Hash Map Data Structure, we have also decided to revamp the MemoryStore Rate Limits! We will be replacing the per-data structure request limit with a new per-partition limit. A partition is a subsection of storage in which MemoryStore data is cached. Data is distributed across these partitions differently by data structure type. This limit will be higher than the old per-data structure limit, providing extra flexibility in how you use the MemoryStore data structures. To minimize the chances of hitting the partition limit, see the updated Best Practices guide.

The API guide has been updated with details about this new data structure, including a section on Hash Maps and one on the new partition limits. We hope you find this useful!

As always, your feedback is invaluable for us to improve our services and fit your needs. Feel free to share any comments or questions below, and we’ll do our best to answer. We can’t wait to see what you build with this new feature!

Happy building!
The Roblox Creator Services Team

211 Likes

This topic was automatically opened after 9 minutes.

:star: This new Hash Map looks fantastic!

The unlimited items and flexible memory limits will help us all to streamline data management.
The partition-based rate limits are a smart improvement, giving us more flexibility.
And the hot-swap feature and ListItemsAsync() function should prove to be very useful.

Loving the focus on allowing developers to streamline data management more!
What a great update!

Can’t wait to see what everyone builds!

29 Likes

Nice! :star:

Really thrilled about what these MemoryHash map can offer, I really hope the memory stores improvements will come quickly.

21 Likes

This is pretty cool, but unfortunately I am still too cautious about using memory store for any kind of dynamic input because the behavior of what happens when the cache fills up, which is common in my existing workflows that use an external Redis, is not documented.

25 Likes

We were migrating millions of backend objects last week this would’ve been perfect omg :sob: THANK YOU

15 Likes

Perfect timing! Almost coincidental… We were just about to release a global mutex lock system that implemented SortedMaps as a temporary implementation.

11 Likes

Will definitely use this for my marketplace and Auction house update for my game, Much Appreciated!

11 Likes

can someone please drop the tldr

8 Likes

TL;DR: SortedMap variant with less features but in return you get more generous limits.

25 Likes

Thanks for your feedback, I’ve responded to your original post (Apologies for noticing it now, appears it wasn’t triaged correctly).

9 Likes

Absolutely splendid.
can’t wait for it to release. :face_with_monocle:

9 Likes

Yoooo, this is awesome!
So if I understand correctly…

This is basically a datastore with much simpler features / less functions, but also less limited rate-wise?

8 Likes

Something I kinda want is native compression, maybe zstd since thats already in the engine (lua apps package).

Referencing this because the docs for MemoryStores mention using LZW without providing such impl.

9 Likes

Not quite, data in a hash map still has to expire, and you are still held to the MemoryStore memory limit, which is much more limited than DatastoreService (which supports theoretically infinite storage space). Hash maps are more suited for temporary, small amounts of data that need to be read / written to very often.

9 Likes

The most impressive thing I’m reading here is the rate limits. Hope the server hamster doesn’t get tired.

14 Likes

That’s fantastic!
I really appreciate seeing that Roblox is getting even better.

7 Likes

I’m a little bit confused on what the ListItemsAsync() “count” parameter means by “max possible number of items returned”. Does this mean the max amount of pages returned, or the max amount of total entries within all of the pages it returns?

5 Likes

Good question, and apologies if the docs aren’t clear! This refers to the max number of entries on one page. So if your max is 50, each page would have up-to 50 items

4 Likes

Howdy. I may not use MemoryStore, this seems like a great update to it. Since it can be used for live auctions or shared inventory. Which seems like a easier way of getting a players data to other players. Without risking people finding weak points. I LIKE SHARDING. (Sharing data from any player with set conditions.)

I’m thinking Turn Based Games, more accurate abilities with stamina steal, etc. In Adventure games where you have a bag of items. You can drop it and it will track all what you had in it and other players can look in the bag.

6 Likes