Preface
Automated testing is a fairly underutilized concept in Roblox development. Generally only used by large scale games with the resources to dedicate to implementing it. However, it brings numerous benefits that can help out smaller teams as well. Automated testing helps to simplify your workflows and allows you as a developer more confidence that your changes won’t have unintended consequences.
Going one step further, implementing Continuous Integration (CI) can be an incredibly powerful tool for quickly checking if code on a given branch or a PR meets a minimum standard for functionality. Continuous Integration is the process of running your suite of unit tests on your code in a remote environment such as Github Actions or TravisCI.
This post is written with the assumption that you are already familiar with Rojo and you already manage your codebase with Rojo. It also assumes familiarity with Git and assumes that you are already using Github for your source control. Additionally, I’m going to skip over some of the more basic examples that can be found in the documentation for the tools that are used and focus more on the theory and implementation of those tools. If at any point you find yourself confused with a concept relating to one of the tools I discuss, make sure to check out the docs for those tools as they can explain their own functions much better than I can. (Feel free to ask questions though, I’ve undoubtedly left out some important detail).
Unit Testing
Unit Testing is the simplest form of testing and the basis for everything else in this article. A unit test is designed to test a single piece of functionality (e.g. a distance function in a math library or a color mixing function in a color library). Roblox provides an excellent library called TestEZ for writing unit testing.
TestEZ can be installed a few different ways, with Git Submodules, including a compiled .rbxmx file, or using a tool like Kayak to manage Lua dependencies. Chose whichever fits your workflow the best.
Unit tests are generally written on a file by file basis, with each file you want to test having a corresponding .spec.lua file in the same directory. The Roact repository has a great example of this workflow.
Check out TestEZ’s documentation for more information on writing and running your tests.
Automatically running tests in Roblox Studio
Now that you have written some unit tests, you might want to be able to click one button that launches Roblox Studio, runs your tests, and exits with the results. Luckily, a few community tools make this easy to accomplish.
First, you’ll need to pick up run-in-roblox, a simple tool for opening a place file in Roblox Studio and executing a given script. This tool will become even more valuable when moving into a CI environment.
Next, we’ll need to compile your code into a place file for run-in-roblox to open. Working under the assumption that all your code is managed with Rojo, all you need to do is run rojo build to compile all your code into a .rbxlx file. Keep in mind that this only includes files managed with Rojo, so any tests that rely on items outside of what Rojo manages will fail. Though, it is a good practice to make sure your tests are completely isolated from any external environment and mock any APIs that may prove unreliable. Tests should be consistent and repeatable.
Finally, before you can shove all your work into run-in-roblox, you need a bootstrap script to execute your tests. By default, run-in-roblox doesn’t execute any scripts, it’s no different than you opening a place file in studio.
This section is probably where some code examples can come in handy, so here’s the bootstrap script I use for one of my projects.
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local TestEZ = require(ReplicatedStorage.Packages.TestEZ)
local testFolders = {
ReplicatedStorage.Libraries
}
local results = TestEZ.TestBootstrap:run(testFolders, TestEZ.Reporters.TextReporter)
if #results.errors > 0 or results.failureCount > 0 then
error("Tests failed")
end
It simply pulls in TestEZ from where I stored it (I use Kayak to manage Lua dependencies in this case), executes unit tests, and then errors if the unit tests failed. This final part is important since the exit code of run-in-roblox is determined based on whether or not the bootstrap script errored. TestEZ accepts an array of locations to search for tests in, which makes it easy to quickly run tests on server, client, and shared code.
Continuous Integration
Taking this even further, now that we can execute a script to run all of our tests and give the results back in the terminal, we can do this all in a CI environment such as Github Actions.
Starting though, we need to manage our tools more efficiently, for this, we’ll use foreman, which allows quick installation of tooling, specifically geared towards Roblox development.
In our case, we’ll need both Rojo and run-in-roblox. If you use other tools in your workflows, such as Selene, this can be a great way to manage them and make sure that all team members have the tools installed.
[tools]
rojo = { source = "roblox/rojo", version = "0.5.0" } # If using the Rojo 6 RC, update version to "6.0.0-rc.1"
run-in-roblox = { source = "rojo-rbx/run-in-roblox", version = "0.3.0" }
The rest of this article will specifically focus on Github Actions, but the techniques should translate to other CI tools if your team prefers those.
Github Actions run a given set of workflows, each defined by .yml files. I suggest familiarizing yourself with how Github Actions work with the official docs. Your workflow will need to handle every part of the process from installing studio to bootstrapping tests.
First, a few prerequisites required for installing Roblox Studio in a CI environment. Grab install.py and GlobalSettings_13.xml from OrbitalOwen/roblox-win-installer. We’ll need these to install Roblox Studio. I suggest creating a tests/ directory if you haven’t already to all the files needs for running tests (this would be a good place to keep your bootstrap script).
I’d also suggest creating a throwaway Roblox Account that will be used to download Roblox Studio. You’ll need its ROBLOSECURITY token, but make sure not to sign out of the account, just close the browser window (it’s best to do this in an incognito window).
Let’s break down a basic workflow for running your tests in Roblox Studio.
unit-tests:
name: Unit Testing
# Roblox Studio needs to run on Windows
runs-on: windows-latest
steps:
# checkout + npm install
- uses: actions/checkout@v1
with:
submodules: recursive
# Dependencies for OrbitalOwen/roblox-win-installer
- run: pip install wget psutil
# Install Roblox Studio
#
# If your repo is private, you can get away with just the second line of this script
# since you can just store your ROBLOSECURITY token in Github Actions secrets.
#
# However, if this project is public, you may want to use a backup token so
# that Github Actions won't fail on Pull Requests.
- run: |
if (![string]::IsNullOrEmpty("${{ secrets.ROBLOSECURITY }}")) {
python tests/scripts/install.py "${{ secrets.ROBLOSECURITY }}"
} else {
python tests/scripts/install.py "backup token here"
}
# This uses the foreman.toml you already wrote to install the needed tools.
# foreman.toml should be in your repositories root directory, not in the tests/ directory
- uses: rojo-rbx/setup-foreman@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
- run: foreman install
# Build your project into a .rbxlx file
- run: rojo build -o ./tests/test.rbxlx
# Run your tests with run-in-roblox
- run: run-in-roblox --place ./tests/test.rbxlx --script ./tests/spec.lua
This workflow runs through every step needed to set up the environment and execute your tests.
Addendum
I wanted to get this basic article out in order to get feedback, however, I plan to expand more on concepts like mocking tests, automatic alerts if a build fails, or even turning CI into CD.