Nexus Unit Testing - Automation Testing User Interface for Roblox Studio

Resources Community Resources
#1

About a year ago, I teased a user interface for a Unit Testing plugin. For many reasons, that version of the plugin was never released. After releasing Nexus Plugin Framework, I started work on a new Unit Testing plugin to release a fully polished Unit Testing plugin for Roblox development.

image
image858Ă—416 51.3 KB

Why Unit Test
Most developers on the Roblox platform use manual testing for developing systems. While this works for most games, manual testing is a long and fault-prone activity that still leads to major bugs reaching players. Unit testing provides a fast and consistent way to test the logic of games, plugins, and subsystems for cases that could come up in production, both as common cases and edge cases.

How to Write Unit Tests
Writing unit tests for Nexus Unit Testing and TestEZ is described in the docs for both projects:

For the broader topic of “how to write unit tests”, this thread does not cover practices for creating unit tests. At a super high level, you should focus on:

  • “Edge coverage”: Test cases that hit specific logic (ex: the if and else conditions are ran).
  • “Branch coverage”: Test cases that reach logic (ex: all cases that determines how an if statement is evaluated is ran)
  • “Path coverage”: Test cases that cover all possible branches through all possible edges.

Major Features of the Plugin

Pricing
V.1.2.0 and newer as of September 24th, 2022 has 3 options:

  1. Itch.io - Free with the option to donate.
  2. Rbolox Plugin marketplace - Fixed price of 50,000 Robux. This will change if variable pricing is added by Roblox.
  3. GitHub - Free.

You are free to start using it and maybe decide to donate back if it proves to be a major aid in developing games. I have been using it for years and it has helped enable tens of thousands of dollars in revenue with relatively small-scale games.

Plugin Limitations
The current design does have some limitations that should be known going in. For bugs, see the README in the GitHub repository.

  • When rerunning test cases, the ModuleScript is rerun with all tests. This may change in the future since it doesn’t require that much in terms of changes.
  • Some tests require a running session. Running tests does not automatically do it and you should skip the test if a running game is not detected.
82 Likes
Is it(), expect(), and describe() functions real functions? And if they are how and when do I use them?
Benchmarker Plugin - Compare function speeds with graphs, percentiles, and more!
Introducing Nexus game framework!
Unit Testing Workflows are Impossible
Creator Store: Buy and Sell Plug-Ins with Real-World Currency
#3

This looks great, purchased instantly! I will definitely be using this for my project. Great plugin with amazing documentation.

#4

1 month has passed since the release and no significant changes have been made, so the plugin is now free. I did have a few people say that running all tests with a large game takes too much time since it creates a list frame for each test, but it will be months before I address that.

3 Likes
#6

I think the plugin is broken? I keep getting these errors (like 5000 of them actually) and it causes Studio to freeze up for a good while. I’m not even sure where it’s getting IsNotOccluded because it doesn’t appear anywhere in any script.

image
image1695Ă—296 35.2 KB

#7

The plugin itself is still functional. This popped up a few weeks ago and is due to a design flaw in the plugin framework. I currently am allocating my time to other projects so I can’t deliver a fix soon.

#8

2 years after release, I have finally released a new version of the Nexus Unit Testing plugin that fixes the major performance issues and more!

Distribution Change: itch.io

Unlike the previous versions of the plugin, the version on Roblox has not been updated yet. The latest version can be obtained from itch.io instead, which you can directly support. Proper support for TestEZ is something I want in the plugin so the output is easier to follow, but I can’t justify the effort. If you have been using this plugin for your games and you feel it has helped enable a significant amount of revenue or reduction in costs to testing or reactive support to issues, consider donating back.

If you do get the plugin from the toolbox, you will be stuck on V.1.1.3 of the plugin with a new banner about the version being deprecated. If it breaks, that version will not be fixed.
image

Breaking Change

Be aware though - Nexus Unit Testing V.2.2.0 introduces the following break change: “Removed require("NexusUnitTesting") test detection support. ModuleScripts that contain tests now must end with .spec (TestEZ) or .nexusspec (Nexus Unit Testing).”

This means if you had ModuleScripts without .spec or .nexusspec in the name and relied on require("NexusUnitTesting"), they will no longer be detected. At scale, this makes detecting tests significantly faster.

Improvements

Test Output (Nexus Unit Testing only, not TestEZ)

Due to how Lua handles function global environments, if a function was defined outside of a test, it would output to the parent test instead of the test it was called. This has been addressed, but only for Nexus Unit Testing since TestEZ still doesn’t support nested tests properly.

For the following example:

local NexusUnitTesting = require("NexusUnitTesting")

local function TestWarn()
	warn("Test")
end

NexusUnitTesting:RegisterUnitTest(NexusUnitTesting.UnitTest.new("Test 1"):SetRun(function()
	TestWarn()
end))

NexusUnitTesting:RegisterUnitTest(NexusUnitTesting.UnitTest.new("Test 2"):SetRun(function()
	TestWarn()
end))

return true

V.1.1.3 and older would show the following:

image
image834Ă—228 22.2 KB

While V.1.2.0 and newer will show the following:

image
image828Ă—228 20.4 KB

Improved Performance

Performance for displaying tests is dramatically better.

Persistent Selections

Due to a limitation in the previous plugin framework, if you were to rerun tests with tests selected, they would not remain selected. The limitation was addressed and this is fixed. Rerunning tests also show a lot faster now.

No Tests Helper Info

If you have no tests found when attempting to run, the plugin will now provide a bit of helper text.
image

9 Likes
#9

Could I just double-check, for tables :AssertEquals() checks equality of tables (by comparing recursively checking equality of elements), whereas :AssertSame() checks if tables are the same object in memory?

#10

That is correct. Equals checks for the contents of something like a table to be the same (i.e. it can be replaced with this equal thing) while Same requires it to be exactly the same as an in-memory reference (i.e. it can only be replaced with the same exact thing, not a copy).

1 Like
#11

Is there any support for running tests in the debugger? Not even sure this is possible. I know the Roblox team added support for debugging plugins but I haven’t seen any documentation on how to invoke it.

#12

Debugging isn’t supported but probably could with the debugger API. I don’t plan to implement that though.

#13

what’s the difference between this and assert()? :face_with_raised_eyebrow:

#14

Reasons why apply to most testing frameworks, not just Nexus Unit Testing or TestEZ. Many others have explained the reasons why in better detail than I could have.

Besides that, the plugin gives you a graphical user interface for visualizing test cases.

#15

Work on the next major update for Nexus Unit Testing is in progress and I now have a beta version to try out for a few weeks before it releases. The next release will be fairly big.

I will be deprecating Nexus Unit Testing’s way of tests in favor of TestEZ. In order to do this, certain functions needed to be added to TestEZ to reach feature parity. TestEZ seems abandoned by Roblox, so I am planning to introduce opt-in extensions.

image
image1427Ă—838 164 KB

If you add --$NexusUnitTestExtensions to a unit test script, it will unlock a couple of functions to use when running expects:

  • near - Replaces the stock version that only works on numbers with a custom version that will work on all current number-based data types. This is like AssertClose and AssertNotClose in Nexus Unit Testing’s asserts.
  • deepEqual - Instead of doing a simple == on the expected and actual values, deepEqual will compare the contents of 2 tables to see if the contents are equal. It also handles cyclic tables. This is like AssertEquals and AssertNotEquals in Nexus Unit Testing’s asserts.
  • contain - For strings, it checks if a substring or pattern is in the main string. For tables, it checks if a value is contained in the table. Nexus Unit Testing’s asserts has no equivalent.

In the event the opt-in comment is not included in the script, a warning will be shown with near before it fails for non-number inputs and the error for deepEqual and contain will contain information about the opt-in. Opting in does mean the tests written are no longer compatible with the master version of TestEZ.

Tests written using the older Nexus Unit Testing format will continue to work. There are no plans to add new features and you won’t have typing or autocomplete. TestEZ requires less typing for the same tests with the extensions and complete faster than Nexus Unit Testing’s older format.

2 Likes
#16

Hey… Why is it 50K ROBUX???

Thats messed up…

#17

The answer is simple: realize the price is insane and look elsewhere for the price. There is a reason why this post lists Itch first.

#18

Sorry didn’t realize, I found the link and here where to download for others who are confused like me:

image
image1825Ă—735 118 KB

Edited:

This plugin is so useful, Tysm for it, this would help me so much in the future!

#19

is there any way to specify a root folder(s) to run tests from? Atm every .spec file in my wally directory is getting tested

#20

That use case isn’t supported with the current plugin. I’ve always practiced not shipping tests with libraries, but I know this is not common practice.

#21

I would highly suggest making that a feature, as of right now anyone using any open-source modules with .spec files is left yielding until their desired test is reached. The tests in the promise library alone add 14 seconds to completion time

#22

All I can really suggest right now is submitting a pull request for this feature.

1 Like