Page MenuHomePhabricator

Updated 773 Days AgoPublic

This is for Goldilocks 2.0. If you're using Goldilocks 1.0 (part of PawLIB), see the official documentation at

The most important element in Goldilocks is the Test, which is the smallest executable unit, and is usually made up of Expect statements.

Every Goldilocks test is derived from the Test abstract class, which has multiple functions that may be overloaded:


REQUIRED. The Constructor must call the Test constructor and pass the name and docstring, as follows:

: Test("test_name", "Some test documentation string here.")

bool pre()

OPTIONAL. Sets up the test to be run. In cases where a test is run multiple consecutive times, it is only called once. Thus, it must be possible to call pre() once, and then successfully call run() any number of times.

The function must return true if setup was successful, and false otherwise.

void prefail()

OPTIONAL. Tears down the test after a failed call to pre(). It is the only function to be called in that situation, and it will not be called under any other circumstances. If not defined, calls post()

It has no fail handler itself, so prefail() must succeed in any reasonable circumstance.

bool janitor()

This is designed to perform cleanup in between run() calls, but not to perform first time setup (pre()) or end of testing (post()) cleanup. Must return a boolean indicating success.

Janitor is always called before each call to run() or run_optimized(), including before the first run()` call.

bool run()

REQUIRED. This function contains all the code for the test run itself. After pre() is called once (optionally), run() must be able to handle any number of consecutive calls to itself.

There must always be a version of run() that accepts no arguments. However, you may overload run() to accept a scenario string (part of the LIT Standard) for generating a particular scenario, or prompting a random one to be generated.

The function must return true if the test succeeded, and false if it failed.

IMPORTANT: run() (with no arguments) must be consistent in its success. Assuming pre() was successful, if the first consecutive call to run() is successful, all subsequent calls to run() must also be successful. This is vital to the benchmarker functions, as they can call a single test many thousands of times. One consideration, then, is that run() should only use one scenario in a single lifetime, unless explicitly instructed by its function arguments to do otherwise.

bool run_optimized()

OPTIONAL. Called instead of run() by the benchmarker. Roughly identical in behavior to run(), except it is written to better accommodate taking benchmark measurements. If undefined, calls run().

The function must return true if the test succeeded, and false if it failed.

void post()

OPTIONAL. Handles teardown at the end of a test's normal lifetime. It is generally responsible for cleaning up whatever was created in pre(), janitor(), and run().

It has no fail handler itself, so post() must succeed in all reasonable circumstances.

void postmortem()

OPTIONAL. Handles teardown if a test fails (run() or run_optimized() returns false). It is responsible for cleaning up whatever was created in pre() and run(), much like post() is, but only for those scenarios where run() returns false. If not defined, calls post().

It has no fail handler itself, so postmortem() must succeed in all reasonable circumstances.


REQUIRED. However, does nothing.

Last Author
Last Edited
Jul 2 2020, 12:24 PM

Event Timeline

jcmcdonald created this object.
jcmcdonald created this object with edit policy "All Users".