Nexus Unit Testing - Automation Testing User Interface for Roblox Studio

About a year ago, I teased a user interface for a Unit Testing plugin. For many reasons, that version of the plugin was never released. After releasing Nexus Plugin Framework, I started work on a new Unit Testing plugin to release a fully polished Unit Testing plugin for Roblox development.

Why Unit Test
Most developers on the Roblox platform use manual testing for developing systems. While this works for most games, manual testing is a long and fault-prone activity that still leads to major bugs reaching players. Unit testing provides a fast and consistent way to test the logic of games, plugins, and subsystems for cases that could come up in production, both as common cases and edge cases.

How to Write Unit Tests
Writing unit tests for Nexus Unit Testing and TestEZ is described in the docs for both projects:

For the broader topic of “how to write unit tests”, this thread does not cover practices for creating unit tests. At a super high level, you should focus on:

  • “Edge coverage”: Test cases that hit specific logic (ex: the if and else conditions are ran).
  • “Branch coverage”: Test cases that reach logic (ex: all cases that determines how an if statement is evaluated is ran)
  • “Path coverage”: Test cases that cover all possible branches through all possible edges.

Major Features of the Plugin

Pricing
V.1.2.0 and newer as of September 24th, 2022 has 3 options:

  1. Itch.io - Free with the option to donate.
  2. Rbolox Plugin marketplace - Fixed price of 50,000 Robux. This will change if variable pricing is added by Roblox.
  3. GitHub - Free.

You are free to start using it and maybe decide to donate back if it proves to be a major aid in developing games. I have been using it for years and it has helped enable tens of thousands of dollars in revenue with relatively small-scale games.

Plugin Limitations
The current design does have some limitations that should be known going in. For bugs, see the README in the GitHub repository.

  • When rerunning test cases, the ModuleScript is rerun with all tests. This may change in the future since it doesn’t require that much in terms of changes.
  • Some tests require a running session. Running tests does not automatically do it and you should skip the test if a running game is not detected.
61 Likes

This looks great, purchased instantly! I will definitely be using this for my project. Great plugin with amazing documentation.

1 month has passed since the release and no significant changes have been made, so the plugin is now free. I did have a few people say that running all tests with a large game takes too much time since it creates a list frame for each test, but it will be months before I address that.

3 Likes

I think the plugin is broken? I keep getting these errors (like 5000 of them actually) and it causes Studio to freeze up for a good while. I’m not even sure where it’s getting IsNotOccluded because it doesn’t appear anywhere in any script.

The plugin itself is still functional. This popped up a few weeks ago and is due to a design flaw in the plugin framework. I currently am allocating my time to other projects so I can’t deliver a fix soon.

2 years after release, I have finally released a new version of the Nexus Unit Testing plugin that fixes the major performance issues and more!

Distribution Change: itch.io

Unlike the previous versions of the plugin, the version on Roblox has not been updated yet. The latest version can be obtained from itch.io instead, which you can directly support. Proper support for TestEZ is something I want in the plugin so the output is easier to follow, but I can’t justify the effort. If you have been using this plugin for your games and you feel it has helped enable a significant amount of revenue or reduction in costs to testing or reactive support to issues, consider donating back.

If you do get the plugin from the toolbox, you will be stuck on V.1.1.3 of the plugin with a new banner about the version being deprecated. If it breaks, that version will not be fixed.
image

Breaking Change

Be aware though - Nexus Unit Testing V.2.2.0 introduces the following break change: “Removed require("NexusUnitTesting") test detection support. ModuleScripts that contain tests now must end with .spec (TestEZ) or .nexusspec (Nexus Unit Testing).”

This means if you had ModuleScripts without .spec or .nexusspec in the name and relied on require("NexusUnitTesting"), they will no longer be detected. At scale, this makes detecting tests significantly faster.


Improvements

Test Output (Nexus Unit Testing only, not TestEZ)

Due to how Lua handles function global environments, if a function was defined outside of a test, it would output to the parent test instead of the test it was called. This has been addressed, but only for Nexus Unit Testing since TestEZ still doesn’t support nested tests properly.

For the following example:

local NexusUnitTesting = require("NexusUnitTesting")

local function TestWarn()
	warn("Test")
end

NexusUnitTesting:RegisterUnitTest(NexusUnitTesting.UnitTest.new("Test 1"):SetRun(function()
	TestWarn()
end))

NexusUnitTesting:RegisterUnitTest(NexusUnitTesting.UnitTest.new("Test 2"):SetRun(function()
	TestWarn()
end))

return true

V.1.1.3 and older would show the following:

While V.1.2.0 and newer will show the following:

Improved Performance

Performance for displaying tests is dramatically better.

Persistent Selections

Due to a limitation in the previous plugin framework, if you were to rerun tests with tests selected, they would not remain selected. The limitation was addressed and this is fixed. Rerunning tests also show a lot faster now.

No Tests Helper Info

If you have no tests found when attempting to run, the plugin will now provide a bit of helper text.
image

9 Likes

Could I just double-check, for tables :AssertEquals() checks equality of tables (by comparing recursively checking equality of elements), whereas :AssertSame() checks if tables are the same object in memory?

That is correct. Equals checks for the contents of something like a table to be the same (i.e. it can be replaced with this equal thing) while Same requires it to be exactly the same as an in-memory reference (i.e. it can only be replaced with the same exact thing, not a copy).

1 Like