Bad past game performance causes irrelevant ratings to negatively impact player's expectations

TheAmazeman · May 13, 2019, 1:56am

As a Roblox developer, it is currently impossible to showcase our game’s most recent ratings to the games page, instead, the all time rating is shown.

I believe ratings are an incredibly important tool to gauge how well a game, or even how a gamepass is performing.
I believe a certain segment of potential high value players might not checkout a game if the displayed rating is under their arbitrary threshold.
As a game receives more ratings, it takes more and more ratings to change the math for the number % to change for the players viewing it.
I believe developers can and do improve their games over time, and that it is unfair to display the all time rating on the games page, when what matters is the ratings going forward.

What I mean by going forward, is what the past X number of ratings have been, let’s say 1000. That’s a good example sample size to gauge the current rating of the game by the players, and small enough that updates to the game will have an impact on the ratings displayed on the games page. This could also make the top rated sort a bit more sorted by the ratings, assuming the sort goes by recent ratings, and not all time ratings. This could also be done with gamepasses if their ratings were displayed to players on prompt purchase, though 1000 may be too high of a sample size to capture change in player’s opinion about a game’s gamepasses. The all time rating could remain on the game’s detail page.

tl;dr
Don’t show the all time ratings on the games page, instead showcase the rolling average of the last 1000 ratings.

If Roblox is able to address this issue, it could improve developer morale and increase work to maintain high recent game ratings, upping the quality bar on Roblox to be even higher.

BSlickMusic · May 13, 2019, 4:56pm

If only it were smart enough to actively gauge what the current median rating was, because perhaps a more recent average would show a better score, meaning to say that there had been improvements, and adjust accordingly; that would be really cool.

32bitPC · May 13, 2019, 5:06pm

When I used to seek out who disliked my games and ask them how to improve my game, I reached a startling conclusion: 9/10 the person that disliked the game had literally no idea why they disliked it.

I support this idea, but I do have some issues with it. If the proposed system is to remove/ignore old ratings from the public viewing after a certain amount of time, I feel there should be a system to prevent a game from losing all of its ratings if it ever becomes inactive.

One possible way to have this system work is to have it be based on the last time a user played the game rather than when they applied their rating. This way if a user played during launch and then two years later, they came back, the ratings will re-apply/update. It would also prevent a rating “shortage.”

Sorry if any of these points come off as clumsy, I’m using the excuse of sleep deprivation.

TheAmazeman · May 13, 2019, 5:06pm

The idea of the past 1000 is just one potential solution that may even have downsides I haven’t seen yet. We don’t need to have that or the idea of a current median rating to be the exact solution, but I wanted to get it out there in the dev forum that this is a problem developers are having with the rating system

TheAmazeman · May 13, 2019, 5:09pm

No, I don’t think we should be proposing the idea to get rid of the all time ratings on the game’s detail page entirely.

This idea should only impact the numeric rating percentages on all game tiles from all places on the games page and home page. These highly advertised and visible ratings before a player even clicks a game, should showcase the recent ratings, not the all time rating.

32bitPC · May 13, 2019, 5:21pm

Wasn’t suggesting they get rid of the ratings, I’m saying that they get rid of the old ratings so they don’t effect the calculations.

It’s like calculating the average of an array while adding new numbers to the start (new ratings coming in) and removing the old ones.

callmehbob · May 13, 2019, 6:00pm

Coincidentally enough, (because of the number you mentioned) a new game just got dislike botted for 1k dislikes. I usually wouldnt have anything against this idea but I guess this might give trolls a bigger opportunity to sway things. But I mean, trolls get bored, even if this did happen it probably wouldn’t last long? (The new game is already climbing back up nicely from yesterday’s bot).

I have a few weird feelings about the like/dislike system in general. Some users do not know how to rate a game based on what it is, like showcases, clothing homestores, or even a kid’s first game ever. I have come across a few that were clearly made by a beginner, trying their hardest, with a rating below 50%. I personally felt bad for these people and I hope this didn’t prevent them from developing more. (We all react differently to negativity, sometimes it gets the fire under us to do better)

On that funny note, I have been hovering 89% fooreevverr. I have a long term goal of getting dat 90%!! I plan on achieving this through many updates… nothing wrong with a long term goal I guess? …right…?! (Still convincing myself of this)

When using the search system i probably don’t click on games under 40%.

I think the like/dislike systems only uses right now are…

a goal for the dev (or a potential beatdown to never dev again because you see how brutal people are?!?! Maybe? I haven’t actually talked to anyone who had this case, maybe it’s me just feeling bad for games that get heavily disliked for no legitimate reason)
a way for users to gauge if the game is decent before going in (I think there should probably be a different system for this than the like/dislike thing honestly)

Dislikes are a weird thing. As roblox is the imagination platform, I don’t think people should be able to come into your brain and shut down your creative ideas. If YOU enjoy the game, and you want to see your imagination come to life, strangers shouldn’t be able to make you feel bad in a vague dislike system where you do not even know why they disliked.

When I first started out and published a game, I got my first dislike around 40 likes. I was so crazy nervous to get my first dislike and ruin my perfect stat. Of course, dislikes are inevitable and I knew it would probably happen sometime. I covered that portion of the screen with my hand when opening my game page to prevent from seeing the like/dislike. I really just didn’t feel like potentially getting disappointed from some vague dislike that I have no way of knowing why it happened. (Even if we could know the reason, as someone stated in a different comment, 9/10 people don’t have a valid reason why they disliked. coincidentally my like ratio is nearly 9/10 )

I have learned A LOT more valuable feedback via players telling me on twitter. The like/dislike system hasn’t done a single thing to let me know how to improve my game

As far as roblox stats goes, I gauge how well the game is doing from how many people are playing, average play time, crash rates etc

Twitter / Facebook / idontuseanythingelsetoknow do not have dislike functions. YouTube does, I’m not sure if it belongs there or not and there’s other people that can talk about YouTube with much more creditability than I.

There does need to be like a “legitimacy” tag or something on games I think, or something that gives players an indicator that they aren’t going to get scammed in this game or something along those lines.

There’s a lot to talk about on this subject

TheAmazeman · May 13, 2019, 6:36pm

I had this in mind actually, and I think that if this 1000 rating (still admittedly arbitrary) thing was in-place, it could be incredibly easy to report botted ratings to Roblox. It would be too obvious to ignore something is up if a game had a 0% rating (or very low because of a recent botting attack) even while being well liked while chatting with players.

I agree with this post that they need to secure the rating system, even if it’s a captcha, that majorly helped my group wall when it came out, I don’t see it not working for giving a game a rating either.

callmehbob · May 13, 2019, 6:47pm

Didn’t realize this was common knowledge that the voting system was vulnerable (honestly I didn’t even know bots could do this until the bot attack yesterday) that deff needs to be fixed for sure.

My husband mentioned something kinda funny, I was just thinking about the up-side to this system, updates = upvotes basically. What happens when players do not like the update, such as maybe temporarily disabling a feature, roblox dataloss happening, etc?! But tbh like you said before, this just gives developers more accountability to keep up with good updates.

AND might even help with giving some clues if the audience in general is enjoying the game rn

HoudiniDev · May 13, 2019, 7:10pm

If there’s no major changes at all to game ratings, in my own opinion something as simple as a users rating expiring after 3 months would allow for better and more accurate results to be displayed and would give the user the opportunity to fairly rate their experience from a more recent point in time as games are improved and developed. Maybe a game hasn’t improved and has since poorly been improved, they could negative rate their experience. Etc.

TheAmazeman · May 28, 2019, 7:36pm

Since I posted this, my game has received 1257 ratings at a 83.77% approval, 4 points higher than the all time ratings show. This is hurting the image of my game as the all time 79% rating has no relevance whatsoever to the current iteration of the game.

The last 7 days the game has received 519 ratings at a 84.58% approval, trending upwards. Due to the large amount of votes already cast, it would take 32,000 thumbs up and 0 thumbs down for my all time ratings to catch up with an 84% rating.

The idea that my game (and many others) is getting what it deserves with the current rating system, and that it will eventually reach it’s actual “going forward” rating is ludicrous. There are many other games that have had rough histories, such as Mad City (which all time rating has slowly risen from below 70% to 79% since establishing its place as a main-stay). Who knows what its actual rating is now? Being able to rise that far after that many votes is incredible.

The fact that the all time rating is advertised on the games page, instead of a more recent sampling of the ratings such as the OP, is asinine.

Tomarty · May 29, 2019, 4:18am

Ratings should also be per-platform. A game could easily have an awful experience on console but a great experience on mobile, and the game’s displayed ratings should reflect this.

TheAmazeman · August 23, 2019, 3:44am

I am very pleased that this idea got talked about at RDC, let’s continue the conversation!

One thing I am not sold on that was mentioned in this video is resetting the ratings after each update, that seems messy and sample size is a big deal which is not dealt with in that potential system. Roblox already has a potential solution inspired after their implementation of RAP for limited items (Recent Average Price). Roblox could develop a system “RAR” (Recent Average Ratings) as mentioned in the OP, display those on the games page so the ratings are never based solely on low sample sizes.

BULK_BROGAN · October 23, 2019, 1:57pm

I agree in principle, but I don’t think a rolling average of X of votes is the best way to do it, rather something like the average of the last two weeks. This would help prevent bot attacks setting it to basically 0%, as well as being a more meaningful time range for bigger games. i.e if the servers are down, a big game could be smashed in the ratings over a small time period

TheAmazeman · April 14, 2020, 6:33pm

I wanted to bump this as every year developers of games with smaller total ratings put their game on the line in Roblox’s official events. Roblox egg hunts for example (with few exceptions) are notorious for people claiming they don’t like it and that “X year was better”. This creates a blurred hatred of most games involved, and ratings are shot down. The smaller amount of ratings, the more these egg hunt ratings affect the games that are involved. The OP still stands as an incredible idea that needs to be continuously talked about. An event that lasts about a month of a game’s lifetime should not negatively effect the game indefinitely.

I agree with Nicolas_Caged’s X amount of votes as the best way to do it.

I don’t support removing anybody’s past ratings from the game’s detail page display, the purpose is the advertised ratings on the games page.
A total ratings and recent ratings section on the game’s detail page would be awesome.