MessagingService/MemoryStoreService not working this morning. Failing to send

Players are unable to join the game out of nowhere this morning. My experience relies on MessagingService and MemoryStoreService for managing servers and joining maps.


MemoryStores seem to be have issues based on the errors. ‘Osm’ stands for OrderedSortedMap here.
20230707_092613_185_ApplicationFrameHost

I have a warning message in the corner that appears when API calls like this fail.
20230707_092644_566_RobloxPlayerBeta

image


It seems resolved now. – Nevermind. It just went through one time for me. Everyone still appears stuck waiting for a server.
Based on player reports it was is broken from 4:54 AM to 9:38 AM PDT.

4 Likes

Hi, I can confirm that this issue is also happening to us.

Our training center game uses MessagingService to talk with other places within the experience, such as updating it. Since ~8 AM we’ve been experiencing the error “There was an error invoking Hub method 'MessageRouterHub.Publish'”, even after trying to republish 30 times but it ultimately just fails.

This also can happen with MessageRouterHub.Subscribe, with basically the same error. (“There was an error invoking Hub Method 'MessageRouterHub.Subscribe'.”). This happens randomly and is not constant, though annoying.

(Images removed for privacy reasons)

1 Like

Hey @Tomarty and @ddust1n thank you for the report. We had an infra issue with MessagingService this morning which is now resolved. Can you confirm that everything is back to working on your side?

2 Likes

It’s resolved now.

During the outage I determined that the max payload size is actually 991 bytes when sending a string (not 1kb.) My manager is set up so I can request to send messages on a key, and it gets sent in a batch with other messages. I can also disconnect a message (like a join request when a player leaves), to prevent buildup during outages. When there’s an long outage it starts to attempt sending batches at or just under the request size limit. It’s important that this limit is consistent and well-documented. I would prefer if it were possible to send non-unicode messages using string.pack, so that my messages can be more compact, have more predictable sizes, and so it’s easier/more-performant to encode and decode data.

3 Likes

Yes, it is resolved now. Thank you!