I suspect that Analytics > Performance > Client Frame Rate is incorrect

Here’s the rundown:

On Pet Simulator 99 we have a custom fast-flags system that allows us to dynamically change the behavior of all servers/clients on the game, live in real-time. One of these flags allows us to disable the rendering of other pets for players, (i.e a user can see their own pets, but not pets belonging to other players.) I can confirm that this system works perfectly - we’ve been using it for over a year to change game behavior live and have had no issues.

The trading plaza is a place under Pet Sim 99 (place ID 15502339080) that supports 50 players at once. The biggest performance bottleneck in this server (by FAR) is the pets system, because the client must render pets for 50 players. Plus, there are no other serious mechanics going on. It’s pretty much a baseplate of pets.

When flipping the render-other-pets flag (note, this flag applies only to mobile players, i.e Phone and Tablet), I’d expect to see significant framerate increases on the anaytics dashboard (showing Place=Trading Plaza(15502339080), Percentile=P10). However, I see no changes at all. The chart carries on as normal. To demonstrate this, I left the flag enabled for about 12 hours last night. Here are the timestamps:


Just to reiterate, the rendering of other pets was OFF from 4:34PM EST 2/20/24 → 4:52AM EST 2/21/24, meaning the framerate of mobile players should be much higher.

Here’s what the analytics shows during that timeframe, showing AVG framerate:

I’d like to note: without further context it might appear the flag helped. This isn’t true. If you extend the analytics out 7 days you can see that there are fluctuations in framerate throughout the day (pretty interesting, must be a correlation between timezone:device_quality):

I’ve ensured our flag system works properly by sitting in the trading plaza on low-end mobile devices and changing the flag from a dev account on a different server. Framerate significantly improves.

The point of this post is just to say: please look into this. I suspect something fishy is going on. Flipping our internal fast-flag applies a significant framerate boost to mobile devices, and that significance is not reflected on the analytics page. We’ve been relying on this analytics dashboard to gauge how well our performance improvements have helped players (i.e we ported to Streaming Enabled recently) and I get the sense the data isn’t reliable.

6 Likes

Thanks for the report! We’ll follow up when we have an update for you.

3 Likes

Will also add that we’ve been tracking framerates ourselves with our own analytics and it’s a stark contrast.

3 Likes

Without using Roblox’s Analytics how else can you determine a clients frame rate?

Send some data using HttpService about user FPS?

For instance, you could use Heartbeat signal (or similar) and calculate how many times it triggers per second.
Then after the client gathers enough data, send it to the server.

Hi:
We are working on debugging what the issue could be from our end right now.
Could you help turn on the flag for a few hours again and let me know when it is on?
Therefore we can capture some sample data and analyze offline. Thanks!

I track FPS independently of Roblox and send it to my own servers, and I can tell you this stuff is pretty accurate from our research. However, we haven’t moved data quickly, so maybe Roblox’s isn’t very responsive?

1 Like

I’d prefer not to flip the flag again as it worsens the player experience. Are you unable to use existing data & the date range I gave you? If not I’ll flip it again, please lmk.

Unfortunately I cannot use historical data since I did not set up the debug framework.
It does not have to be a very long time. 1 hour probably should be enough for me to gather some data and analyze.