Hello there, I’m @Abcreator,
You may or may not know me but I’m a QA tester for some of largest experiences on the Roblox platform, for example, DOORS and Rainbow Friends. I’m making this guide to inform developers on some of the best-practices when deciding to bring people together to test your Roblox experience.
First, let’s dispel some common misconceptions when it comes to QA testing on the Roblox platform:
- QA testing is not necessarily just “bug testing”, it may also include game-design feedback as-well as suggestions for possible new-features.
- Following on from the last-point, QA testing shouldn’t be the last step in your development process, when everything is completed and ready for release. A tester may inform you that UX (user-experience) in your game is confusing and you may have to re-design the entire thing!
- QA testing is not just for “big-experiences” with thousands of players, many small Roblox experiences get tested and it can be incredibly important for their future-success.
- QA testing is a job, make sure to treat QA testing that way, don’t down-play testers as regular players, treat them as you would someone who you are commissioning.
Why is testing important?
- Decreases critical release-day issues.
- Makes your game more user-friendly and decreases the chance of players getting confused.
- Helps you discover new features for your game that you may have never thought about.
If you’ve decided that testing is important for your Roblox experience, that’s great! But how are you going to get testers, is there a grading-system you should be working on, how important is past-experience for QA testers? I’ll try to answer as much as possible in the remainder of this topic.
What options are there for testing?
You have two options when it comes to testing your Roblox experience, both with their own drawbacks:
-
“Free QA testing” groups:
These groups provide free testers to QA test your Roblox experience, you can find many of these amazing groups with posts in #resources:community-resources, these groups are great for smaller Roblox experiences that may find it hard to get testers interested in your experience. This option does however come with the drawback that your testing team will not be permanent and tester-turnover will be high, generally this option is difficult for larger games as bug-report quality cannot be controlled and secrecy cannot be guarenteed. -
A dedicated testing team:
This can be very difficult to do if you do not have a strong-standing in the development community as it may be difficult convincing testers to join your testing team without a strong-unique game-idea, casual-atmosphere and decent-compensation. This method comes with the benefit that you will build a strong connection with your testing team and as a result may be better at communicating issues. You will also have more control over quality and secrecy of your testers. The next section is dedicated specifically to getting testers on your QA team if you choose this method of testing…
Structure of your testing team:
This section only applies if you opted to setting up a dedicated testing team, if not, skip to running a QA test
Hold up! Before you start bringing on testers onto your team, you need to decide on how your testing team will be structured, mainly the following points:
- Who will testers communicate with? Will they directly communicate with the dev team, or will you hire someone specifically for collecting bugs / feedback, filtering it for notable issues and sending it to you.
- Will you host live testing sessions or will you open your game to testers and collect feedback over multiple days?
- How will testers gain access to your game, will they have to friend you, join a group, or will they simply be whitelisted through code?
- How will you compensate testers? This is the big-one, testers will want to be compensated, choose what method is best for what testers you wish to target. For example, experienced-testers may expect Robux or USD while some fans may be perfectly fine with a chat tag in-game, this will be discussed in more detail later in this topic.
- Will you require testers to sign an NDA? Some developers may ask testers to sign a contract to prevent leaking info, please note that this will limit the number of people willing to test and may require expensive consultations with a lawyer to make sure your contract is legal in all countries and at all testers’ ages (including minors!)
Compensation (in more detail):
Testers will likely expect some form of compensation, whether it be through Robux, USD, or simply an in-game item. You should judge this based on four major factors:
- Tester experience (please note that paying more experienced testers more money within the same QA program is generally frowned upon and should not be practiced!)
- Your current revenue - testers will expect more pay if you are a front-page game compared to if you are just a small passion project making less than 100 Robux, in the latter they may not expect more than an in-game reward
- Amount of bugs reported
- The “audience” of your testing team - if the testers on your testing team are all fans of your experience, they may settle for an in-game item rather than some professional testers that expect Robux for their work
Selecting testers:
There is no one “best tester”, each tester may suit different games and the following are fairly generalised and do not home-into a specific game-genre:
- Previous experience: Former-experience is important however don’t gate-keep those without QA experience out! Obviously it is hard to judge those who may not have QA tested before, however many developers combat this be letting them “practice test” on a public version of the game, etc. Remember that previous experience doesn’t necessarily have to be former games the tester has tested for, testing unofficially (for example, game-criticism videos) is also experience you should look into.
- Important qualities: Every tester can find bugs (well at least I hope they can). However, communication is key to QA testing. If a tester is too vague when communicating bugs or doesn’t inform you of them, they may as well not be a tester at all. Make sure to take this skill into account when selecting testers.
- Make people aware that you are testing! Don’t hide the opportunity to those who ask, let everyone who may be interested know that you are accepting testers. Barely any testers will directly reach-out to you about a testing opportunity unless it’s advertised to them - although don’t directly-target specific testers with advertisements for QA opportunities.
- Be aware of your legal restraints! Make sure you aren’t bringing on testers who are under 13 if you are hosting tests via text-messaging platforms like Discord or if you have an NDA. In the latter, make sure you also follow any other legal actions you may need, consult with a lawyer if you are bringing contracts into the equation.
Running a QA test:
When hosting a QA test it is important to adjust to a certain style of communication. Will you adopt a professional stance towards all reports or be more casual? It is also important to be active in the session and respond to possible game-breaking issues appropriately. A QA session should not be your break but rather a time for you to take notes of testers’ behaviour in-game. Is a tester drifting majorly off-track in your story-game, are they doing that intentionally to find bugs or are they genuinely confused on where to go? Remember that if a tester is going into your game blind they will likely act as how a normal-player would when playing your game, if your testers are getting confused on what to do and need your intervention to progress, you may need to re-think your game-design!
Make sure to set-up some way of receiving feedback, it’s absolutely useless testing your Roblox experience if all the feedback / bug-reports can’t be recorded somewhere. It may be tempting to act as a proxy for testers and write the issues / feedback down yourself, however, this can get over-whelming quickly and could end-up with lost issues. Many experiences opt for a simple solution of Discord Threads, Google Forms and more-professional teams may decide to use a Slack channel. Make sure that the option you select is available for all your testers to use and matches your budget!
Make sure you get all required info from reports! Let testers self-assess a bug’s impact on a player, you can always correct this later if you discover a bug to be marked too critical in severity. Make sure you figure out which testers are reporting which issues and which devices a tester is on as-well as any required screenshots / dev-console logs. This is stuff you can’t get later, get it now!
Make sure you make it easy for testers to report issues / bugs, don’t make them scared to report. I know this may seem obvious at first-glance, but it does happen! Don’t scare testers by saying “no duplicates” hundreds of times during your test, generally the extra effort it will take for you to filter reports is worth it rather than the possible bug-exclusions you may have if testers are scared to file their bug-reports.
After the test, it is important to clear-up any possible confusion there could be. “Are the testers allowed to mention that they tested?” (generally the answer to this should be yes unless you have testers under an NDA and are not releasing yet), “Will testers be receiving in-game rewards?”, etc. Then it comes down to sorting feedback, remove duplicates or non-issues (that are also not feedback), send on the most critical issues / features to your dev-team (or patch it yourself) and get ready for your next-test. It is worth noting that your “fixes” to some reported issues may actually cause more issues! As a result, you should not release your game until you have passed through a full-test without any critical issues.
Scalability Tests (Stress Tests)
So, you just hypothetically-released and all of a sudden your entire game breaks! Did your testing team lie to you, did making the game public break everything? Not necessarily, Roblox imposes rate-limits (MemoryStores, etc) and physical-limits (RAM, etc) on servers/experiences, while your game may run perfectly-fine with a small amount of players, an increased load could cause errors or slow-downs! Generally, most game-developers will run a “stress test” closer to launch with a much larger amount of people invited, many developers may even open it up to the public! At this point the game’s secrecy is generally considered to no longer exist so many developers prefer to do stress tests after launch - however it is worth noting that during-launch is when you’ll need your game to be functioning. That would likely be when your highest-engagement and highest player-load will be in game, so take that approach with caution. During stress tests, feedback generally isn’t as heavily weighed compared to bug-reports, you are looking for any bugs that may occur due to the high player-load!
Conclusion:
And that’s it, I hope I have helped at least some developers on their path to testing their Roblox experience. Did I make a mistake? I’m sure I probably did somewhere as I come specifically from the tester-side of the equation. If so, make sure to leave a reply to this topic so other developers can learn from you! Is there any questions you have for me? Make sure to ask them!
) and how to avoid them.
