MessagingService 'Messages received for entire game' limit is incorrect

@Seranok The wiki states the current limit for ‘Messages received for entire game’ is (100 + 50 * number of servers) per minute. Instead, it appears the game does not increase this budget for each additional server, so the budget instead remains at (100 + 0) per minute

You can replicate the bug with the following code for one-player servers. With two players it runs fine, however as soon as you go above it begins throttling and throwing the error The rate of publish requests exceeds the allowed limit, despite being within the limits.

local messagingService = game:GetService("MessagingService")
local topicBaseName = "Topic"
local totalTopics = 3
local messagesSent = 0
local publishesPerMinute = 50
local publishInterval = publishesPerMinute /60

for topicId = 1, totalTopics do
	messagingService:SubscribeAsync(topicBaseName..topicId, function(message)
		print(message.Data)
	end)
end

while true do
	wait(publishInterval)
	local topicId = (messagesSent%totalTopics)+1
	local topicFullname = topicBaseName..topicId
	local success, errorMessage = pcall(function() messagingService:PublishAsync(topicFullname, "Test"..topicFullname) end)
	if not success then
		warn(errorMessage)
	end
	messagesSent = messagesSent + 1
end

I’ve setup a place here as an alternative: Messaging Service Testing - Roblox

8 Likes

I’ll just piggyback off your post since it’s tangentially related, but I have another issue occurring in my game:

This appears to only affect one topic at a time, but it also seems like this topic gets locked out for the entire game for an indeterminate amount of time. I’ve seen it start working again only to start throwing these errors once more.

Then again, these might even be the same root cause. Hope that helps anyone debugging this issue.

EDIT: Hit another seemingly incorrect/undocumented limit:

The game server has reached the allowed limit of active subscriptions.

It’d be nice if there was a way to see the remaining throttle budget like with DataStores. Regardless, I only had 14 topics subscribed for a 1-player server when this error was encountered. That seems pretty low.

2 Likes

Would like to also include that servers will occasionally shut itself down when hitting the publish request limit.

1 Like

If I understand what you are doing, with 1 server, 50 messages/minute are sent and 50/minute are received. (Global limit = 100 + 50*1 = 150)

With 2 servers, 100 messages/minute are sent (50/server) and 200 messages/minute are received (since every message is received once on each of the 2 servers). (Global limit = 100 + 50*2 = 200)

With 3 servers, 150 messages/minute are sent (503) and 450 messages/minute are received (3150 since each message is received on each server). (Global limit = 100 + 50 * 3 = 250).

It gets worse after that since the way you’ve set things up the number of received messages scales as n-squared. Based on my math, then, the system is performing as expected and begins throttling for 3 servers and above since starting there you are trying to receive too many messages.

If I’m misunderstanding things or can do anything to help clarify further, let me know.

3 Likes

I’m not sure why you are seeing the “retry after -2billion”. We will investigate this.

2 Likes

Thanks for clarifying, that make sense now.

Surely this method of calculating the budget limits scalability and puts larger games at a significant disadvantage though?

For example, lets say a game publishes a message twice every minute:

  • With 10 servers, the total messages received will equal 200/minute (10^2 * 2) , with a budget of 600/minute (100 + 50 * 10)

  • With 1000 servers, the total messages received will equal 2,000,000/minute (1000^2 * 2) , with a budget of 50100/minute (100 + 50 * 1000), exceeding the limit.

With the current budget, surely games can not practically use MessagingService as they grow in size?

A budget of (100 + 50 * number of servers squared) per minute would resolve this.

2 Likes

I think it makes sense. Each server isn’t really going to be able to receive messages from 1,000 other servers the same as it could from 100 other servers. Which is why I think that you should limit broadcasts to one single server, and everything else should be more “point-to-point.” For example, register a topic for each user ID in your server and only send messages to specific users.

This way, it seems that bigger games wouldn’t be at a disadvantage assuming the API is used as intended, but rather smaller games because they can go longer without encountering this pitfall, which can make it more confusing to debug.

When I worked at Roblox, I know we tried pretty hard to design APIs that were impossible to use incorrectly (which isn’t always possible), but I don’t know what I’d do with this one. It’s pretty difficult to disseminate information like this to all the developers that might use it. Interesting problem.

1 Like

Sorry for the long delay in responding to this. I haven’t had a chance to come back and check the forum since before the Thanksgiving Holiday here.

What mr_smellyman alludes to is pretty much correct. Since the message traffic scales as the square of the number of machines if everybody is broadcasting at a constant rate to everybody else, you can’t use a code/design pattern for your game like this since we only let you scale number of messages linearly based on the number of game instances (servers). From a backend perspective, it would be hard and expensive to engineer the system to handle the millions of messages per minute (or per second for our larger games) that would be required to support every server broadcasting to every other server.

The system should still be useful if you set up channels to listen to messages per player to use for a chat like system where players only receive messages sent to them by other players. Or, you can have an “achievement” channel where you broadcast when a player accomplishes something so you can display a congratulations to all players across your game instances. These are just a couple examples that don’t require you to send a ping from every server to every other server several times a minute.

2 Likes

FWIW, I’m still seeing the -2 billion timeout issue.

I was not certain I should create a new topic or not, so I decided to revive this topic to see if I could get some information about the -2147483648 seconds issue.

Currently I’m creating a game where the player rooms are made so it sends a message to the main menu list every 5 seconds, however after very few sucessful messages sent and the game starts outputting this error:

I tried changing the wait from 5 to 60 seconds to see if it would solve the problem, but it would still happen after 2-3 minutes.
Not too sure if this is a useful information or not, but every player room is a reserved server, here’s the topic data that is sent between the room and the main menu:
Topic_Data

2 Likes

We have a system where each server publishes a message once a minute to topic to announce how many players are on the server and are really surprised to be receiving quota errors with such a small volume. I was expecting to be able to send 20 messages per server per minute, based on this doc.

So based on this thread, it appears that each server is not publishing one message a minute but actually a value equal to the number of servers in our game. And each server receives every message even if it wasn’t intended for that server. That is quite a surprising implementation!

Can we ask for the docs to be updated to explain this? Right now, these docs describe our use case as one of the examples of what you can do with the MessengingService. However, this isn’t actually possible for a game with a large number of servers running.

Does each server also subscribe to the same topic?

The problem discussed in this thread is that if you have N servers running, even if you use 1 send budget, you are spending N receive budget because each server is receiving that message.

For N servers sending a message every minute, you are using N*N receive budget per minute. The limit only scales by N, not by N*N, so you are going to hit limits that way.

It can be implemented just fine – you just need to limit the amount of servers acting as server browsers, or unsubscribe from the topic if you are not needing such events (e.g. mid-game in servers where a game is being played).

You need to do some effort here to reduce unneccessary communication and to reduce the complexity of the send-receive graph to avoid the N*N scaling.


Btw, I recommend filing a new bug report if you are seeing issues, this one has been marked as solved by the author.

1 Like

The use-case documentation doesn’t make that clear - it gives a fair limit to how many players could chat with each other (20 in the whole game), but doesn’t extend that limit to 20 servers when it mentions the server browser. An accurate description would be “Compile a list of all the game’s servers and who is in them (every minute) and display it on a maximum of 20 servers.”

2 Likes

That’s a good request for a change to post in #bug-reports:developer-hub

Okay; I sent a message to the Bug Support group about it.

2 Likes

This topic was automatically closed after 6 days. New replies are no longer allowed.