Announcing Observability Dashboard for Memory Stores

[dragonknightflies]

Hi Creators,

We are excited to announce the Memory Stores observability dashboard today, which allows you to monitor & debug issues with this service in real-time!

Previously, there isn’t much visibility about the performance and issues for your live experiences. It could be challenging to detect, triage, and investigate production issues. To streamline your workflow, we now have a new dashboard that offers insights for Memory Stores in real time with minute level granularity.

The Memory Store observability dashboard has 5 charts:

• Memory Usage: tracks the memory usage of the experience, and also shows the quota over time.
  
  
• API Request Units: shows the total request units consumed by the experience and the quota. You can also break it down by each API method to identify which uses the most units. Learn more about request units here.
  
  
• Request Count by API: breaks down the API request count per minute by API method, e.g. SortedMap:Get() so you can understand which API methods are called the most.
  
  
• Request Count by Status: groups the API requests based on response type, e.g. Success, Invalid Request, etc. You can use this information to identify the top issues for this service.
  
  
• Request Count by API x Status: allows you to analyze the response distribution for a specific API or response type.
  
  

We understand that for you to identify the root cause of a particular issue, it’ll probably be useful to know which data structure causes most errors. This will be possible when the Error Reports become available. Stay tuned!

Tips to Handle Limits

Similar to all other services, we recommend building retry logic when API requests are throttled. A good approach is to use exponential backoff. If you are running out of memory, remove the items that are not needed. Generally, we suggest setting the shortest expiration time needed for your items so that your memory can be automatically freed up.

Check out our documentation to learn more. To get started, go to Creator Dashboard, choose the experience you want to monitor, and click the “Memory Stores” section on the left navigation under “Real Time”. We look forward to hearing your feedback and iterating on it! Based on the learnings, we’d love to repeat the pattern for other services in the future.

If you haven’t already, give Memory Stores Service with newly enhanced observability a try. We can’t wait to see what amazing experiences you can build using this fast and reliable in-memory storage!


Cheers for more insights,
The Roblox Creator Services Team

[V_ChampionSSR (V_Champion)]

this is a fantastic update! paired with the 6s datastore cooldown removal, it really has significantly improved the development experience for us devs who use these apis!

[OneEDM (EDM)]

Awesome! Could we get dashboards for tracking usage of other APIs such as MessagingService, OrderedDatastores, Datastores as well?

[iGottic]

@dragonknightflies

Wow… that is a lot of data structure requests over the limit…

In all seriousness though, good job, gamers👍

[Burgundy2014]

@V_ChampionSSR

this is a fantastic update! paired with the 6s datastore cooldown removal, it really has significantly improved the development experience for us devs who use these apis!

Thanks V_Champion! Look for more updates to come in Q3!

[Burgundy2014]

@gillern
Data Stores Observability is on the roadmap.

[apenzijncoolenleuk1 (Apenz1)]

Are both client and server crash logs and client framerate statistics planned for the performance dashboard?

Additionally script usage statistics and network bandwidth usage would be very handy.

I’m not sure how/if it’s possible to represent this data in charts, but it would definitely be helpful for debugging crashes.

For now the only client crash metric I’m aware we have is the “device crash percentage” on developer stats. It’s definitely helpful to see what (mobile) devices are too weak to run our game, but without physical testing access to the device it’s not really possible for us to know why the crashes happen.

More informative crash metrics would be very handy for our high-memory game with realistic assets. And I’m sure a lot of other games would also benefit from this.

[Midnightific]

Oh my gosh, thank you!

My team and I will be using this feature almost everyday, thanks so much and paired with the removal of the 6s limit, it’s been a fantastic week so far!

[SendByte (Send)]

This is a great feature for maintaining limited memory consistency, whereas we can make changes to scripts inside of experiences to see how that effects total memory usage. All-in-all, this is a great monitorization update! Though there is one question that I can’t seem to find on the documentation: how often would the data update?

[Pikachu023456]

Fantastic! Can’t wait to start using this.