Skip to content

Runner

The Runner collects suites, runs them, records results, and prints the report. Most projects never touch it directly: Wisp.suite registers into a lazily created default runner, and Wisp() runs it.

At a glance

Symbol Kind Signature Description
Runner.new function () -> Runner Constructs an empty runner.
Runner:registerSuite method (suite: Suite) -> () Registers a suite. Ignored if the name is taken.
Runner:pass method (suiteName: string, actionName: string) -> () Records a passing action.
Runner:fail method (suiteName: string, actionName: string, message: string) -> () Records a failure.
Runner:run method () -> ({number}, {number}) Runs all registered suites, prints, returns the tallies.
Runner:_report method (elapsedSeconds: number) -> ({number}, {number}) Prints the report, returns the tallies. Internal.

Fields

Field Type Contents
_suites { Suite } Registered suites, in registration order.
_stats { [string]: SuiteStats } One entry per registered suite name.

Both are underscore-prefixed and internal by convention. _stats is what the guides read directly — it is stable enough to use, but there is no guarantee across versions.

Runner.new

Runner.new(): Runner

Constructs a runner with _suites and _stats both empty. No arguments, no side effects, and no registration.

Methods live on a shared metatable table, so they are reached with : rather than ..

Runner:registerSuite(suite)

Runner:registerSuite(suite: Suite): ()

Appends suite to _suites and creates a fresh, empty stats entry for it. Returns nothing.

Parameter Type Description
suite Suite The suite to register. Only _name is read from it.

Duplicate names are silently ignored. The guard is a lookup in _stats, so if a suite name is already registered, this returns immediately — the suite is not added to _suites and no stats entry is created, meaning its actions never run and it never appears in the report. There is no warning.

The guard runs before the insertion, so a name is claimed by the first registration and stays claimed for the runner's lifetime.

Runner:pass(suiteName, actionName)

Runner:pass(suiteName: string, actionName: string): ()

Appends actionName to the suite's stats.passed list. Returns nothing.

Parameter Type Description
suiteName string Must already be registered.
actionName string Stored verbatim and later printed.

Names accumulate in call order and are never deduplicated — calling pass twice for the same action records it twice.

Unregistered suite names raise

If no stats entry exists for suiteName, this calls error("Suite {suiteName} is not registered", 3). Unlike an action failure, this is not caught by Suite:run — it propagates out of Runner:run and aborts the remaining suites.

Runner:fail(suiteName, actionName, message)

Runner:fail(suiteName: string, actionName: string, message: string): ()

Appends a failure record to the suite's stats.failed list. Returns nothing.

Parameter Type Description
suiteName string Must already be registered.
actionName string Stored on the record as name.
message string Stored verbatim and later printed.

The message is stored exactly as given — no reformatting, truncation, or decoration. It is a positional argument after the names, so a message containing any text is safe.

The unregistered-suite check is identical to pass, including the error level.

Runner:run

Runner:run(): ({number}, {number})

Runs every registered suite, then prints the report. Returns the two tallies — see Return values.

Suites are iterated from _suites in registration order, and each is passed the runner itself as its reporter, which is why a runner satisfies the reporter shape Suite:run expects.

Timing starts immediately before the first suite and stops once the last action finishes, so require time and action-definition time are excluded. The elapsed seconds are handed to _report.

A runner is not reset between runs

_suites and _stats are never cleared. Calling run twice executes every action again, appends a second set of results, and reports doubled tallies with the action names repeated. Construct a fresh runner, or call run once.

Runner:_report(elapsedSeconds)

Runner:_report(elapsedSeconds: number): ({number}, {number})

Prints the report to standard output. Internal — called by run, not meant to be called directly. Returns the two tallies.

It walks _stats, printing one block per suite: the suite name, then each passing action with a check mark, then each failing action with a cross and its message indented by three spaces. It finishes with the Suites, Tests, and Took lines.

Two counting rules explain most of the output:

  • A suite counts as passed only when #stats.failed == 0. Two failures and one pass contributes 0 passed out of 1 total to the Suites line.
  • A test is one action. Passing actions increment both the passed and total counts; failing actions increment only the total.

Took is elapsedSeconds * 1000 formatted to two decimal places, so it always shows milliseconds. The glyphs are utf8.char(0x2713) and utf8.char(0x2717), and there are no colours. Both summary lines are always printed, even for an empty run.

Section order is not guaranteed

Actions are iterated from each suite's ordered passed and failed lists, so actions within a suite always print in run order. But _report iterates _stats — a hash map — so the order of the suite blocks is not guaranteed to match registration order. Execution order is still deterministic; only the printed grouping is unordered. The tallies are unaffected.

Return values

run and _report both return two tables, in the same order as the Suites and Tests lines in the report.

Return Index Contents
passed [1] Suites with zero failures.
passed [2] Actions that returned without raising.
total [1] Registered suites.
total [2] Actions that ran.

A run of one suite containing one pass and one failure returns passed = { 0, 1 } and total = { 1, 2 }. The declared type is ({number}, {number}) — two number arrays.

The counters are the same values the summary prints, and they follow the same two rules: a suite passes only when #stats.failed == 0, and a passing action increments both the passed and total counts.

An empty run returns passed = { 0, 0 } and total = { 0, 0 }, so a run that collected no suites is indistinguishable from a clean one by comparison alone.

Wisp does not control the exit code

The tallies are returned, but nothing calls os.exit — a failing run still exits 0. Set the status yourself; see Exit codes.

Reading results

A runner accumulates everything in _stats, keyed by suite name, so per-action detail — which actions failed, and with what messages — is available programmatically after a run.

Field Type Contents
stats.passed { string } Passing action names, in run order.
stats.failed { AssertionFailure } Failures, in run order.
runner._suites { Suite } Registered suites, in registration order.
runner._stats { [string]: SuiteStats } One entry per registered suite name.

A failure record is AssertionFailure in Types: a name and a message, where the message is the tostring of the raised error and usually carries the position prefix the engine adds.

To count failures, sum #stats.failed across every entry in _stats.

Wisp does not control the exit code

run only prints. It never calls os.exit, so a failing run still exits 0. Raise a non-zero status yourself if your CI keys off it — see Exit codes.

Reporter protocol

Suite:run accepts anything with pass and fail, so a Runner is not required. This is the supported extension point for other output formats.

Method Signature Required by
registerSuite (self, suite: Suite) -> () Suite.new only.
pass (self, suiteName: string, actionName: string) Suite:run.
fail (self, suiteName: string, actionName: string, message: string) Suite:run.

Both are called as methods, so self is the reporter table — store state on it if you need to. A registry only needs registerSuite; a reporter only needs pass and fail, and the two roles can be separate tables.

Reporter errors are not caught

Wisp wraps each action in pcall, but a reporter that raises propagates straight out of Suite:run and aborts the remaining actions. Keep reporter code total.

Errors

Situation Result
pass / fail for an unregistered suite error("Suite {name} is not registered", 3), uncaught.
registerSuite with a taken name Silently ignored.
A suite argument without _name Raises on the field read.
message is not a string Stored as given; raises at print time in _report.

Next