Add way to get lcov code coverage in studio, engine, and in open-cloud luau-execution-session-task.
As a Roblox developer, it is currently too hard to get code-coverage stats from our unit testing runs. This prevents us from tracking coverage and code quality through code-coverage. Code coverage is a metric that measures the extent to which a program’s source code has been executed during testing, helping identify areas that might need more attention. While luau and luau-cli has the ability to output lcov files - a standard code coverage format, the Roblox engine itself has no such mechanism.
While unit testing games is typically a bad idea, we have many core libraries which we depend upon. We want unit tests to ensure we don’t break these during refactors. And while code coverage is often a bad statistic to force in terms of velocity, just reporting code coverage itself can be a powerful mechanism to increase test quality.
Proposed workflows
Write unit test
Roblox studio reports lcov coverage and a plugin can report the actual coverage percentage
Unit test runs in open cloud
A script extracts the lcov files and sends them back to github action
A standard lcov github action bot can parse the files and report coverage
This would make a big difference, and is a core requirement for higher code quality and more professional development on Roblox. This is also a feature we can’t do without Roblox’s help.
This is a great feature request, thanks! We agree we need a story for code coverage analysis.
We don’t provide access to the full debug library in the engine due to security concerns. This means the approach taken by LuaCov & similar to do code coverage won’t work in the Roblox runtime.
Bytecode instrumentation is probably the right approach here (and is what we do for internal Luau development). However - that means instrumentation needs to be done when the scripts are compiled. As such, there’s a path to us supporting in Open Cloud Luau Execution (through some sort of flag) but supporting in plugins / Studio would be much more complicated.
From your feature request - it sounds like you were expecting this already? Let me know though!
I will talk to the engineering team and get a sense of what this would take to support. Will update here in due course
Is there an update on this? I run unit tests with Jest through Open Cloud for my open-source and internal libraries. My use case only requires code coverage reporting to work in Open Cloud, so if that can be supported it would be awesome.
I’d also like to +1 this as I would like support for this. Jest should have code coverage but it’s currently not implemented, likely due to the fact that at the Roblox native level it does not exist.
