Reading the Report¶

After every action has run, Wisp prints a report to standard output. It is plain text — no colours, no spinner, no JSON — so it reads the same in a terminal, in CI logs, and in Roblox Studio's output window.

Anatomy¶

Calculator
 ✓ adds two numbers
 ✗ multiplies numbers
   ./demo.luau:8: Assertion failed: expected 5, got 4
Strings
 ✓ uppercases

Suites: 1 passed out of 2 total
Tests:  2 passed out of 3 total

Took:   0.11ms

Part	Meaning
`Calculator`	The suite name, printed once.
`✓`	A passing action. `✗` marks a failing one.
Action name	The string you passed to `Suite:action`.
`./demo.luau:8: ...`	Location and message of a failure. Only printed for failures.
`Suites: n passed out of n total`	A suite counts as passed only if it has zero failures.
`Tests: n passed out of n total`	Per-action tally across every suite.
`Took: n.nnms`	Wall-clock time for the whole run, via `os.clock()`.

Both counts are always printed, even when everything passes. An empty run still produces a report.

Passed and failed are tallied separately¶

A suite is counted as passed when #stats.failed == 0. An action is counted as passed when its function returned without raising.

Because the two tallies are independent, a suite with two failures and one pass contributes 0 passed out of 1 total to the Suites line and 1 passed out of 3 total to the Tests line. Compare the two lines to tell "most things broke" from "one thing broke hard".

Failure messages¶

A failing action prints its error message on the following line, indented by three spaces. Whatever your action raised is what you see, because Suite:run records tostring(err) verbatim:

Raised by	Example message
An assertion	`Assertion failed: expected 5, got 4`
`error("boom")`	`boom`
Indexing `nil`	`attempt to index a nil value`
A type error	`Cannot call a nil value`
Another suite's `require`	the module-not-found text from the engine

Wisp does not reformat, truncate, or decorate these. If the message is unhelpful, the fix is in the test — assert more precisely, or check the value before using it.

Timing¶

Took measures only the run: the clock starts after all suites are registered and stops when the last action finishes, so require time and assertion-definition time are not counted. It is formatted to two decimal places in milliseconds.

Exit codes¶

A failing run still exits 0

Wisp reports failures on standard output but never calls os.exit itself. If your runner or CI step keys off the process exit status, a red report will still look like a green build.

The fix is in your run file, not in Wisp. Wisp() returns the tallies, so you can branch on them and set the status yourself:

tests/run.luau

local Wisp = require("../src")

require("./Calculator.spec")
require("./Cart.spec")

local passed, total = Wisp()

-- [1] - suites
-- [2] - tests

if passed[2] ~= total[2] then
  os.exit(1)
end

What comes back¶

Wisp() — and Runner:run underneath it — returns two tables, in the same order as the Suites and Tests lines in the report:

Return	Index	Contents
`passed`	`[1]`	Suites that had zero failures.
`passed`	`[2]`	Actions that returned without raising.
`total`	`[1]`	Registered suites.
`total`	`[2]`	Actions that ran.

So a run of one suite with one pass and one failure returns passed = { 0, 1 } and total = { 1, 2 }.

Choosing what fails the build¶

The two indices are independent, so pick the one that matches your gate. A suite is passed only when it has zero failures, which means one broken action marks the whole suite red:

Check	Fails the build when
`passed[2] ~= total[2]`	Any action failed. The usual choice.
`passed[1] ~= total[1]`	Any suite contains a failure — equivalent here, since a suite only fails if one of its actions did.
`passed[1] ~= total[1] or passed[2] ~= total[2]`	Either, for a belt-and-braces check.

A run that failed on both lines is a red build either way, so for most projects the per-action check is the honest one: it is the number in the Tests line.

Fail on failures, not on emptiness

An empty run returns passed = { 0, 0 } and total = { 0, 0 }, so it exits 0. If a run that collected no suites at all should be a failure, check for that separately:

if total[1] == 0 then
  error("no suites were registered")
end

Under Roblox¶

os.exit terminates the whole process, so it is only appropriate from a standalone run under Lune or the Luau CLI. In Roblox Studio there is no process to exit — read the same tallies and raise a warn instead:

local passed, total = Wisp()

if passed[2] ~= total[2] then
  warn(`{total[2] - passed[2]} of {total[2]} actions failed`)
end

Reading results from your own runner¶

The tallies are the printed summary. For the per-action detail — which actions failed, and with what messages — read runner._stats instead. See Reading results.

Interpreting a missing suite¶

If a suite you wrote does not appear in the report at all, the cause is almost always one of these:

Duplicate name. The first suite with a given name wins; later ones are ignored.
The spec file was never required. A spec only registers when something requires it — a require call in your top-level run file, or an autotest that discovers it.
Wisp() ran too early. Suites registered after the call are never run.

Troubleshooting¶

Symptom	Likely cause
`attempt to index a nil value` on `Wisp.assume`	The require returned `nil`; check the require path.
`require path must start with a valid prefix`	A relative require used at the top level — prefix with `./`, `../`, or `@`.
Suite never printed	Duplicate name, or the file was never required.
Only some actions ran	Something in an action called `os.exit`, or an error escaped outside the action.
A passing action is reported failed	A non-assertion `error` in the body — remember every error is a failure.
`Suite X is not registered`	You called `Runner:pass`/`fail` for a suite the runner does not know about.

Next¶

Advanced Usage — driving the runner yourself.
Runner API reference.