Guide: Effectively QA Testing Your Roblox Experience

[Abcreator]

Hello there, I’m @Abcreator,

You may or may not know me but I’m a QA tester for some of largest experiences on the Roblox platform, for example, DOORS and Rainbow Friends. I’m making this guide to inform developers on some of the best-practices when deciding to bring people together to test your Roblox experience.

First, let’s dispel some common misconceptions when it comes to QA testing on the Roblox platform:

• QA testing is not necessarily just “bug testing”, it may also include game-design feedback as-well as suggestions for possible new-features.
• Following on from the last-point, QA testing shouldn’t be the last step in your development process, when everything is completed and ready for release. A tester may inform you that UX (user-experience) in your game is confusing and you may have to re-design the entire thing!
• QA testing is not just for “big-experiences” with thousands of players, many small Roblox experiences get tested and it can be incredibly important for their future-success.
• QA testing is a job, make sure to treat QA testing that way, don’t down-play testers as regular players, treat them as you would someone who you are commissioning.

Why is testing important?

• Decreases critical release-day issues.
• Makes your game more user-friendly and decreases the chance of players getting confused.
• Helps you discover new features for your game that you may have never thought about.

If you’ve decided that testing is important for your Roblox experience, that’s great! But how are you going to get testers, is there a grading-system you should be working on, how important is past-experience for QA testers? I’ll try to answer as much as possible in the remainder of this topic.

What options are there for testing?

You have two options when it comes to testing your Roblox experience, both with their own drawbacks:

• “Free QA testing” groups:
  These groups provide free testers to QA test your Roblox experience, you can find many of these amazing groups with posts in #resources:community-resources, these groups are great for smaller Roblox experiences that may find it hard to get testers interested in your experience. This option does however come with the drawback that your testing team will not be permanent and tester-turnover will be high, generally this option is difficult for larger games as bug-report quality cannot be controlled and secrecy cannot be guarenteed.

• A dedicated testing team:
  This can be very difficult to do if you do not have a strong-standing in the development community as it may be difficult convincing testers to join your testing team without a strong-unique game-idea, casual-atmosphere and decent-compensation. This method comes with the benefit that you will build a strong connection with your testing team and as a result may be better at communicating issues. You will also have more control over quality and secrecy of your testers. The next section is dedicated specifically to getting testers on your QA team if you choose this method of testing…

Structure of your testing team:

This section only applies if you opted to setting up a dedicated testing team, if not, skip to running a QA test

Hold up! Before you start bringing on testers onto your team, you need to decide on how your testing team will be structured, mainly the following points:

• Who will testers communicate with? Will they directly communicate with the dev team, or will you hire someone specifically for collecting bugs / feedback, filtering it for notable issues and sending it to you.
• Will you host live testing sessions or will you open your game to testers and collect feedback over multiple days?
• How will testers gain access to your game, will they have to friend you, join a group, or will they simply be whitelisted through code?
• How will you compensate testers? This is the big-one, testers will want to be compensated, choose what method is best for what testers you wish to target. For example, experienced-testers may expect Robux or USD while some fans may be perfectly fine with a chat tag in-game, this will be discussed in more detail later in this topic.
• Will you require testers to sign an NDA? Some developers may ask testers to sign a contract to prevent leaking info, please note that this will limit the number of people willing to test and may require expensive consultations with a lawyer to make sure your contract is legal in all countries and at all testers’ ages (including minors!)

Compensation (in more detail):
Testers will likely expect some form of compensation, whether it be through Robux, USD, or simply an in-game item. You should judge this based on four major factors:

• Tester experience (please note that paying more experienced testers more money within the same QA program is generally frowned upon and should not be practiced!)
• Your current revenue - testers will expect more pay if you are a front-page game compared to if you are just a small passion project making less than 100 Robux, in the latter they may not expect more than an in-game reward
• Amount of bugs reported
• The “audience” of your testing team - if the testers on your testing team are all fans of your experience, they may settle for an in-game item rather than some professional testers that expect Robux for their work

Selecting testers:
There is no one “best tester”, each tester may suit different games and the following are fairly generalised and do not home-into a specific game-genre:

• Previous experience: Former-experience is important however don’t gate-keep those without QA experience out! Obviously it is hard to judge those who may not have QA tested before, however many developers combat this be letting them “practice test” on a public version of the game, etc. Remember that previous experience doesn’t necessarily have to be former games the tester has tested for, testing unofficially (for example, game-criticism videos) is also experience you should look into.
• Important qualities: Every tester can find bugs (well at least I hope they can). However, communication is key to QA testing. If a tester is too vague when communicating bugs or doesn’t inform you of them, they may as well not be a tester at all. Make sure to take this skill into account when selecting testers.
• Make people aware that you are testing! Don’t hide the opportunity to those who ask, let everyone who may be interested know that you are accepting testers. Barely any testers will directly reach-out to you about a testing opportunity unless it’s advertised to them - although don’t directly-target specific testers with advertisements for QA opportunities.
• Be aware of your legal restraints! Make sure you aren’t bringing on testers who are under 13 if you are hosting tests via text-messaging platforms like Discord or if you have an NDA. In the latter, make sure you also follow any other legal actions you may need, consult with a lawyer if you are bringing contracts into the equation.

Running a QA test:

When hosting a QA test it is important to adjust to a certain style of communication. Will you adopt a professional stance towards all reports or be more casual? It is also important to be active in the session and respond to possible game-breaking issues appropriately. A QA session should not be your break but rather a time for you to take notes of testers’ behaviour in-game. Is a tester drifting majorly off-track in your story-game, are they doing that intentionally to find bugs or are they genuinely confused on where to go? Remember that if a tester is going into your game blind they will likely act as how a normal-player would when playing your game, if your testers are getting confused on what to do and need your intervention to progress, you may need to re-think your game-design!

Make sure to set-up some way of receiving feedback, it’s absolutely useless testing your Roblox experience if all the feedback / bug-reports can’t be recorded somewhere. It may be tempting to act as a proxy for testers and write the issues / feedback down yourself, however, this can get over-whelming quickly and could end-up with lost issues. Many experiences opt for a simple solution of Discord Threads, Google Forms and more-professional teams may decide to use a Slack channel. Make sure that the option you select is available for all your testers to use and matches your budget!

Make sure you get all required info from reports! Let testers self-assess a bug’s impact on a player, you can always correct this later if you discover a bug to be marked too critical in severity. Make sure you figure out which testers are reporting which issues and which devices a tester is on as-well as any required screenshots / dev-console logs. This is stuff you can’t get later, get it now!

Make sure you make it easy for testers to report issues / bugs, don’t make them scared to report. I know this may seem obvious at first-glance, but it does happen! Don’t scare testers by saying “no duplicates” hundreds of times during your test, generally the extra effort it will take for you to filter reports is worth it rather than the possible bug-exclusions you may have if testers are scared to file their bug-reports.

After the test, it is important to clear-up any possible confusion there could be. “Are the testers allowed to mention that they tested?” (generally the answer to this should be yes unless you have testers under an NDA and are not releasing yet), “Will testers be receiving in-game rewards?”, etc. Then it comes down to sorting feedback, remove duplicates or non-issues (that are also not feedback), send on the most critical issues / features to your dev-team (or patch it yourself) and get ready for your next-test. It is worth noting that your “fixes” to some reported issues may actually cause more issues! As a result, you should not release your game until you have passed through a full-test without any critical issues.

Scalability Tests (Stress Tests)

So, you just hypothetically-released and all of a sudden your entire game breaks! Did your testing team lie to you, did making the game public break everything? Not necessarily, Roblox imposes rate-limits (MemoryStores, etc) and physical-limits (RAM, etc) on servers/experiences, while your game may run perfectly-fine with a small amount of players, an increased load could cause errors or slow-downs! Generally, most game-developers will run a “stress test” closer to launch with a much larger amount of people invited, many developers may even open it up to the public! At this point the game’s secrecy is generally considered to no longer exist so many developers prefer to do stress tests after launch - however it is worth noting that during-launch is when you’ll need your game to be functioning. That would likely be when your highest-engagement and highest player-load will be in game, so take that approach with caution. During stress tests, feedback generally isn’t as heavily weighed compared to bug-reports, you are looking for any bugs that may occur due to the high player-load!

Conclusion:

And that’s it, I hope I have helped at least some developers on their path to testing their Roblox experience. Did I make a mistake? I’m sure I probably did somewhere as I come specifically from the tester-side of the equation. If so, make sure to leave a reply to this topic so other developers can learn from you! Is there any questions you have for me? Make sure to ask them!

[JuanGamerPlayz_RBLX]

Heya! My name is Juan (@JuanGamerPlayz_RBLX) and I am also associated with experiences that @Abcreator QA tests as well such as “Rainbow Friends”, a popular experience since 2021 with the variety of more than 10-20K DAU (Daily Active Users).

Reviewing the topic and every chapter, led me to also contribute by providing further information about the importance of applying QA (abbreviation of Quality Assurance).

Summary:

• Bugs

  • Methods to Locate Bugs
  • Creating a Bug Report

• Feedback

  • Structuring your Feedback
  • Feedback and Recommendations

> Bugs

Methods to Locate Bugs:

While having an active testing session hosted by developers and/or contributors of the following experience you are QA testing, sometimes, we do not know where to begin with… “What do I report?”, “Do I follow the others near me?”. Many questions can come up to our minds and interrupt us from thinking. It is recommended to stay calm and begin by producing a simple analysis of the experience information, the title, description, etc.

Let’s see using the experience of “Natural Disaster Survival” as a short example:

It is always good to report the information in cases of having multiple sentences and/or paragraphs censored, which can make up players unable to comprehend what is the objective in-game as a summary:

Badges and Gamepasses are as well good to always be sure that they are capable to see overall.

The next step is to join the Roblox experience and test how the visual interfaces work and if you can communicate with them one by one. We can also communicate with in-game events and look forward to their respective results as expected. However, it is primary to note that you can choose which of these you would like to start. Every tester has their perspectives and cases to identify… Therefore, do not copy their ways, be creative!

Creating a Bug Report:

Whenever encountering a bug you have found, it is always good to note them, state how are their impact from low (does not damage too much the experience) → critical (does damage the experience highly) and reproduce the steps for the developers understand where and how to showcase the issue. It does not need to be long and filled with basic details, you can make it short as long as it is following the context and is easy to get it right away.

» Here is a demonstration and let’s apply the experience of “Natural Disaster Survival” as the sample for this area:

Juan’s Bug Report:

1. In the Happy Home map, the house does not fall apart during a round of disaster called “Earthquake”;
2. In the lobby, the billboard advertising the green balloon does not provide any information to buy the game pass.

Images and videos are also good guidance for them to see in real-time how the player saw. If the place allows you to submit files, then it is your chance to provide them right in time!

> Feedback

Structuring your Feedback:

Quality Assurance not only tests the quality of the product but can also deliver feedback! Feedback helps to indicate how the product can be better in the context of the overall perspective of regular users while experimenting, UX (abbreviation of User Experience).

Structuring your feedback needs to first apply the simple basis, which we can begin by asking ourselves: “Why the experience was difficult for me?”, “Could it be complicated for the rest?”… It is good to remember the steps you have passed along the way and reflect on how complex it was to get to think how other regular and new players experience the same action. Always good to think of others that are newbies and tell them what can be good for them such as tutorials at the beginning, advice in-game, and so on.

» Let’s think of NDS, its objective, and how players can comprehend great joining and walking by:

Feedback Report:

1. Provide a tutorial that explains what are the perks of buying the green balloon in Natural Disaster Survival;
2. Every disaster should have an explanation about them and how to survive shortly.

Again, every tester has their ways to do so and you should always be unique while still following the content.

Feedback and Recommendations:

The very last topic to mention within this chapter is regarding feedback and recommendations.

Sometimes, when we come up with these, we usually confuse them as considering both of them the same but as synonyms… This is fine, yet it can be bad on the other side as the developers may see it differently and not take it into further consideration.

Feedback → information available by a person once reacting to a product and suggesting improvements.

Recommendation/Advice → A recommendation from something or someone to use because it is good for the best practices.

Difference → Feedback is either a reaction/response to a particular process or activity while for Advice, opinion, or recommendations offered as a guide to action, conduct, etc.

Having the knowledge about both of them and getting that all aren’t similar or equal, you may apply them separately as the developers analyze, review everything, and pick by part those that are primary and can genuinely contribute to a better experience for players!

---

And that is about it! Hopefully, my reply was also helpful and could assist with the topic of this tutorial.

I wish you all amazing luck and lets QA!

[Abcreator]

Added a new paragraph dedicated to “Stress Tests”, following on from a suggestion I received in messages. Thank you for your suggestion.

[GamingLamaWasTaken (GamingLama)]

Thank you! this is actually an great resource for “Developer” & evolving “QA-Testers” who wanna get into the professional side of OA-Testing.

[Abcreator]

Thank you for the kind words, I’ve been in the QA-industry for the past 3 years and unfortunately have seen many developers handle QA in the ‘wrong-way’ so I made this topic to help future developers not fall into those same pitfalls.

[Clxzed]

Thank you for this amazing post! I hate that I will be hiring testers later on due to money right now, but I will be sure to compensate all my testers with what they are worth (equally!)

EDIT: I’m a bit of a tester myself so this is especially helpful.

[Abcreator]

Thank you! Funnily enough, we in the Roblox QA-industry have seen a massive increase in developers testing their games since this topic which is certainly nice to see whether or not it was actually prompted by this post. It’s really nice to see developers taking more notice of the QA sector on Roblox, it was majorly unused in the past and some developers even brushed it off as entirely unnecessary, when in-reality it can make-or-break your game launch in some extreme cases!

@Clxzed

EDIT: I’m a bit of a tester myself so this is especially helpful.

Wish you good luck in the industry! It can be very tough as a tester nowadays, it relies a lot on luck and prediction. I will likely make a tutorial more dedicated to the QA-side of things in the future, just have to be more-careful when doing stuff related to that since people are more likely to take my word as the “truth” and complain if I’m incorrect and end up losing someone a really good opportunity. As a result, I’d likely just go over the malicious practices some-people do to QA testers (yes, some developers to maliciously attempt to take advantage of testers ) and how to avoid them.

[Clxzed]

@Abcreator

malicious practices some-people do to QA testers

I have some personal experience with that. I have changed my methods of testing a lot. Though sometimes I will still accept a lousy job (maybe for a friend), I am now looking for organization and well compensation for my worth. I wish you the best endeavors and I can’t wait for that possible tutorial

[justice_romqxn (Roman)]

I just came across this post. I think this is one most well written resources I’ve seen on the forum.

A great resource not only for developers that are looking to start hiring testers for their project, but also for people that are starting out in QA (or have been in the industry for a while and want some more tips).

I’ve done QA for a few projects in the past and the developers had no understanding of the work. They were bad at communicating and not open to feedback. Anytime feedback was given, we were immediately shutdown and told that’s how they want it and it’ll stay as such.

QA is definitely a vital part of the development process and I personally have neglected it on my project however, it is somewhat hard to procure a team. Many people claim they’ve done QA before but haven’t. You mentioned you don’t need to hire experienced but it is difficult to get good feedback with the few bad actors.

Thank you so much for creating this, definitely a great resource.

[Abcreator]

@romqxn

I just came across this post. I think this is one most well written resources I’ve seen on the forum.

Thank you!

@romqxn

Anytime feedback was given, we were immediately shutdown and told that’s how they want it and it’ll stay as such.

From what I’ve seen, developers tend to either to be incredibly open to feedback, or incredibly closed to feedback. Obviously the first approach is much more supportive and lets your game actually benefit from QA other than bug-patches (many large developers have this approach). From my experience, the second approach is mainly taken by developers who believe QA is only for bug-reporting, or more maliciously, are possibly testing so they can say “everything was tested” to users and pass blame for bugs / bad game-design onto testers, it is surprisingly very rare though.

@romqxn

Many people claim they’ve done QA before but haven’t.

Yeah, QA is one of the easiest Roblox-jobs to fake, there’s no fool-proof way to evaluate if someone is lying or not except checking group ranks. Many developers do create ranks on their group for testers, mainly to authorise access. However, de-ranking testers after the testing-period ends is something that does happen sometimes which is something you should be aware of.

@romqxn

You mentioned you don’t need to hire experienced but it is difficult to get good feedback with the few bad actors.

Generally, you’ll want a mix of experience-levels within your QA team. Experienced testers are more likely to find bugs but may be less likely to play your game as intended meaning you may not receive accurate feedback that regular players may have. Players with little QA experience tend to be the exact opposite to that.

[Abcreator]

Just re-visting this topic to fix a few formatting issues which I found and just thought that I’d reply while I’m at it.

---

So many amazing Roblox developers and studios have contacted me showing their extreme appreciation of this post and I am honoured. I made this post in my free-time after having a really frustrating moment with a successful developer who simply just didn’t understand how the QA industry worked. To make it clear and to dispel any rumours, I no longer test for this developer.

I honestly expected this topic to surface to the depths of the forum, never to seen or read by anyone; however I am really glad that some developers have been able to benefit from this topic and I hope to provide much more resources into the future! Thank you, everyone!