(2017-11-07) New Mobile Performance Stats

Hey Developers,

We’re happy to announce an addition of a Performance Filtering section to the Developer Stats page. This section will show you a more detailed view of how your game is performing across various device types.

Specifically, this will show how often your game is crashing on these devices. This is important because we use this same information to filter our mobile game sorts. If a game is crashing too much on a particular device, we will hide that game from sorts on that device until the game has been updated. This is to ensure that players have the best experience possible on the device they are using.

Here is an example of what the new crash statistics will look like:
filtered devices|300x

We hope this information will help you identify problems in your games, and allow you to strategically optimize for mobile experience. You can learn more about optimizing your Roblox games by reading about our Memory Analyzer and Developer Console features.

We’d also like to stress that this is just the beginning of the type of stats we want to expose and provide for developers. Please let us know what you think of this feature and what stats you’d potentially like to see!

EDIT: New Wiki article has been posted. We are looking to make some changes to this system in the future, but for now have a peak: Documentation - Roblox Creator Hub

Thanks,
Developer Relations and Developer Services Team

74 Likes
  • Will all devices be listed, not just 4. All models of iPad, iPhone and android? (just checking)
  • What if there’s a bug in the ROBLOX app itself which will make the user crash?
  • What if the crash is ‘self inflicted’ - I know sometimes playing ROBLOX on the mobile app on my iPad and listening to music with the brightness up causes the app to crash (possibly due to getting hot)
  • What’s the range in % for the filter to take place? e.g. over 80% crash rate game is removed from all listings, <40% has no affect
  • What if the user has a jailbroken device and modified their iDevice in a way that can cause the device to boot into “safe mode” (basically restart) whilst playing (it might be a dodgy extension) would this count as a crash, or would jailbroken devices be void from the statistics outright?
  • Will we get this for other platforms and operating systems in the future?
9 Likes

Ooh, pretty nice! What exactly does Roblox consider as an update which removes the filter for those devices? Does it count any update, or would we need to inform someone about any sort of changes which would help with crashes on that device?

1 Like

Woohoo!!

3 Likes

Thanks for adding this, now I can probably try my best not to build houses too detailed!

2 Likes

Will we see similar statistics for other platforms in the near future?

1 Like

Great question! We track every iOS device, unfortunately Android devices are much harder to track and its very basic information such as “High-end Android” and “Low-end” Android. We really want to stress that this was a tactical change for us and we have so much more we want to deliver to developers in the future.

We are able to separate this as we have specific items that we track in the back-end in which we consider a “Crash”.

We consider an update to the Place as an update that will reset your crash filter. You will not need to inform anyone.

2 Likes

This can change based on how many users have played your device and how many have crashed. We would like to expose how many users are playing the device so you can gauge impact on addressing the issue.

We would like to expand upon this, as to the scope we are not too sure yet. Right now we are focused on Mobile devices because those are the ones that get filtered.

1 Like

When do you think this will be added? Right now I see 75% of our iPad Mini 1g users are crashing, but I don’t know if the total userbase of that device is larger than the other devices that are crashing.

2 Likes

How long has roblox been collecting the data for our games so far? All I see are percentages with no sample size.

Is it going to be similar to how top earning is sorted for example, that the percentage shown is a weekly rolling average? Or is it all time crash rate?

How often is this figure updated on dev stats?

3 Likes

My crash stats are very similar to the ones in OP.
I’m starting to wonder why mobile devices over half a decade old are still supported anyway.

image

2 Likes

We are shooting for early next year.

Over a year and a half, but these are recent with Place updates. As stated above we want to provide information about how many users are using those devices so that you can better gauge the impact of the crashes are your player count.

As stated above, this figure is reset on place update. We are looking into whether they have an expiry date and will probably add one in a later change.

It loads on page load. The data is collected each time you refresh the page.

Yup, this is to be expected. All of those devices are running the A5 chip, and as you can imagine those are very old devices. The problem with discontinuing support is they are a very large percentage of our mobile player base and we can’t just stop supporting them. As this number dwindles we will definitely stop supporting such devices.

6 Likes

Nice, this is very helpful! Will we get any details about what is causing crashes in the future?

1 Like

I think most of the crashes are simply RAM related, so your first approach to stopping crashes is to reduce memory usage.

1 Like

I guess it is a good thing if it shows this?
https://gyazo.com/81a4ba97d3ce05c579b8bf872a418695.png

Absolutely, this could be a ways out but we will definitely do our best to improve upon on this system.

2 Likes

But the real question is, why is 1 in 2 users or worse crashing??? That sounds like an enormously very large extremely big problem!!

4 Likes

What exactly is a “crash” on mobile anyway? iirc, if you run out of RAM (which your phone doesn’t have a lot of), the app just gets force closed. Not sure if that counts.

how would they measure a crash anyway? check a lockfile next time roblox runs? server logs when a player leaves without sending a disconnect signal? …

1 Like

Yes, the app getting closed by the OS counts as a crash (which is also mostly why they happen). Yes, we check a lockfile.

2 Likes

Can this be updated to show unfiltered devices that don’t have a high enough crash rate to filter them out? Like if something’s just at 4% crash rate or approaching 19%+

4 Likes