YOU, yes YOU need more tests!

DISCLAIMER: This is not a guide on HOW to test, as in how to use things like TestEZ. The documentation is fairly accessible and easy to get started: TestEZ Documentation. Rather, it’s a discussion as to WHY exactly you want to start automated testing in your games.

The Importance of Testing

Coming from an enterprise software world (ah, boring I know, sorry), it sometimes astounds me as to how little of an emphasis developers on Roblox place on testing. The most common complaint I hear when discussing this is, “but I’ve coded it in, I see that it works! Why would I ever want to make a test for something I KNOW works?” and to that I will say, it’s because you’re your own worst enemy.

There’s a common joke that goes around that I’m sure you’ve felt that you write seemingly great code, take a week break, and come back to utter chaos. But wait, nothing’s changed! In truth, your eyes just needed a bit of time to adjust to the amount of refactoring that your code needed - game development code is rarely ever an unchanging monolith. In order to develop a full-fledged game, no matter how small it is, you will inevitably increase your original scope of the code you wrote, or break things out into more modular functions, or rewrite portions that simply don’t make sense. And once you do that? You go back to manually test it to make sure the units spawn as expected, they drop the right coins as expected, do damage how you want it. Annoying, right?

In fact, I find that a huge portion of developers quit developing their game not because of the level of difficulty of their coding aspirations or scope, but rather because their codebase became so tedious to maintain because once one thing changes, something, somewhere down the line breaks, and you don’t find out until 10 changes later when you finally run into the broken code but you have no idea what broke it in the first place or when or where!

Sound familiar?

Automated testing is one of the most powerful tools a Roblox developer can adopt — not just for preventing bugs, but for scaling creativity with confidence.

As games grow in complexity — with mechanics, systems, and UI tightly coupled — the cost of manually testing every interaction rises fast. A small code change can break something critical, and you might not catch it until it’s too late — or worse, until a player does.

That’s where automated testing comes in. Tools like TestEZ let you write fast, repeatable, automated tests. You can validate logic, simulate edge cases, and spot regressions before they hit production. Furthermore, you catch interactions and bugs that you wouldn’t have even thought of in the first place.

Automated Test Output Example

Say I’m writing a small module that converts letters into the number 0. If the input isn’t a string, then we want our output to be the number -1. There are also special cases that give us special numbers. Why would you want this? No idea, but it gets the point across!

local module = {}

local function transform(input)
    local specialCases = {
        a = 1,
        b = 2,
        c = 3,
        d = 4,
    }
    if specialCases[input] then
        return specialCases[input]
    end
    return 0
end

function module.convert(input)
    if type(input) == "string" then
        return transform(input)
    end
end

return module

When testing this, we would need to go in the game and test however you can. If a sword is calling this (for some reason) or a chat command calls this, you need to go through every single case. And if this module changes at all, for the logic, or edge cases? You need to go through every single consume and test it again.

As it turns out, with automated testing, we immediately found that it’s wrong.


Convenient! And now these tests are a safeguard in times you need to make adjustments to your module.

function module.convert(input)
    if type(input) == "string" then
        return transform(input)
    else
        return -1
    end
end

Our fixed module

Rojo and CI/CD

Enter Rojo
WARNING: The following discussion block probably isn’t applicable for the vast majority of hobby developers. It’s a bit more dense of a topic and unless you’re running a larger project where reliability, teamwork, and visibility are key, it probably doesn’t apply to you.

Just a quick rant on Rojo. Now honestly, Rojo ISN’T required for a huge portion of the developing userbase. I find that many people misunderstand exactly the purpose of Rojo. They feel like without it, they aren’t “real” developers. Many great games have been made without the use of Rojo! However, what EXACTLY does it give us?

Rojo allows us to separate files from the Roblox Studio itself. That’s it. If you don’t care at all about that, then don’t use Rojo. However, for a great many people, this separation of files means the world. It means we can use package managers like wally to maintain our dependencies in a simple way. It means we can use external source control like git (which I highly recommend). And because we’re using git, it means we can push to an external pipeline for Continuous Integration and Continuous Delivery (CI/CD). All this means is that as we make changes to our code base, we can monitor and control how it integrates into our total existing codebase and make derived operations on it. What exactly do I mean by derived operations?

Well, let’s see our typical workflow without continuous integration.

Make changes to our code locally → Run tests (if existing) → Publish to Studio

Notice how many holes there are in this flow? If tests are run, all we have is the output that exists in the moment. If we have failing tests, then our publish still merges with all of our actual code!

A very, very common workflow for CI/CD is to make derived conclusions from the results of our code. For instance:

Make changes to our code locally → Run tests locally → (We’re bad developers so we don’t care if it fails) → Push to our remote testing branch → Here’s where our pipeline kicks in! → Runs the place in the remote environment and runs tests against it → Packages the tests into a file → Publishes the tests results in an easily digestible and storable format → Allows a merge if the tests are all passing

And this is all automated! All we have to do is commit our code and push, and we get all the results. Our CI pipeline, courtesy of Rojo (and friends) let us move fast without fear — catching issues before they ever reach our players.

Automated Testing in Action

For me, understanding a topic is seeing it in action, so I’ll give an example where automated testing has saved many more hours than it took writing my tests.

Consider the following:
I have a game that contains units that all have a StatusManager class which manages statuses like Bleeding, Weakened, etc. Originally, my StatusManager had an addStatus method with the following header:

function StatusManager:add(effectData, context)
  --create status using the effectData structure
...code

where a general effectData object was coming and giving the data required for the Status. However, I needed to change this to something more general like so

function StatusManager:add(statusType, value, context)
  --create status using the statusType and value arguments
...code

When performing this change, I didn’t actually know the full ramifications of this code. Although the flow to add a status is fairly linear (GameInstance->UnitManager->Units->StatusManager), so many things want to add statuses. If I had a button that adds a status to my unit, or if I had an item, or a card to apply a status, wouldn’t we want peace of mind that these all work? Well, I had tests for many of these cases!


Behold, our instant feedback

Do note that while this test result is an output of our pipeline, even if you don’t have one set up, your tests would still identify the defects

Not only do we know what is broken, but we also know how it’s broken. In doing so, we can immediately go back and fix it, saving us hours of manually searching, trying to think up every test case, and praying it works. In essence, we offload the thinking to our testing!

Conclusion
While I know it went a little bit deeper and off the rails than most people would need, I wanted everyone to understand the importance of testing. There’s a reason every serious software team relies on automated testing. Not because they enjoy writing tests, but because they hate wasting time fixing bugs that could’ve been prevented. Even with a simple test framework and no CI pipeline, testing pays for itself the moment your game grows beyond a prototype.

22 Likes

What you’ve made looks really interesting, but could you elaborate on it for a novice who’s willing to inform themselves more about this project?

4 Likes

Sure, which part in particular were you curious about?

1 Like

Considering how both TestEZ and Lemur are deprecated (both repositories are archived & no longer worked on), what would you recommend today in the context of unit testing?

I don’t feel comfortable building the foundation of a new project using code that was deprecated 2+ years ago – that’s just a disaster waiting to happen.

Roblox dropped TestEZ in favor of their luau port of Jest, but as far as I could find, integration is almost entirely undocumented and the same would go for rotriever, including what it’s for and how it’s supposed to be used.

A while back they introduced their open-cloud testing API, but this makes you reliant on… an API. For unit testing. Not exactly what you’d want.
I did discover what looks to be a service intended for general test automation, but I’ve yet to experiment with this

Overall though, from all the extensive research I’ve done on this topic in the context of Roblox, it really looks like automated tests running inside of a CLI is in a state of limbo for the community right now.

2 Likes

This is a great response, and in the time from when I’ve posted to now, I’ve also come to the same conclusion. Currently, my project is still using TestEZ with too much overhead to switch over, but if I were to do this again, I would definitely use jest. In regards to integration, I haven’t done much of any deep research but from the documentation here, it appears that the formatted output is very similar to TestEZ, and the way that Roblox themselves run their integration tests is very similar to the process outlined where the build the place in the pipeline and run the tests headlessly and retrieve the output that way. For unit testing, I’ve experimented with the tests being in the place and being excluded via project.json config in rojo, and deploying the correct project.json file in the pipeline to the public place, so that they aren’t bundled in the public build and you can easily run unit tests locally just by running. I’ve seen some unit test implementation ideas using lune, but haven’t experimented with it myself at all personally.

1 Like

Woah, it seems like I missed this on the github page it seems.
This is exactly what I was missing :smiling_face_with_tear:

I’ve been wondering; how do you actually achieve this?
Made myself a handful of project commands, but from what I’ve found there doesn’t seem to be a way to configure rojo (through command args, for example) so that it picks a specific project.json to build from. Rojo internally seems to look for a .project.json file with only a parent directory you can configure it to look for?

I made my own testing “library” last week to work around the current limbo, which unfortunately only works in studio as a plugin. Don’t know of a way to exclude them from a “prod” build, but they’re never required, so there’s little to no impact they have on the game regardless…

Basic test syntax based on PHPUnit


I think I’ll give Jest another try now that I can actually read some docs.

I now also see why I first couldn’t find the docs for Jest.

The first (and only) search result regarding lua Jest you get points to an internal github repo (GitHub - Roblox/jest-roblox: Delightful testing for Luau. This is a read-only mirror.) which contains docs meant for internal use only (Getting Started | Jest Roblox)

The ones you linked and actually want to use are the repo (GitHub - jsdotlua/jest-lua: Delightful testing for Lua.) and the docs (Getting Started | Jest Lua)

Incredibly confusing, but a good thing to keep in mind.
…It also looks like the latter is no longer being worked on :upside_down_face:
Back to square 1?

I recently made a CLI tool to run TestEZ tests on the Roblox Open Cloud. It is useful for testing data-driven systems and jecs.

It currently only runs in a server context.

GitHub: https://github.com/gado7h/aether"