Page MenuHomePhabricator

Updated 389 Days AgoPublic

This is for Goldilocks 2.0. If you're using Goldilocks 1.0 (part of PawLIB), see the official documentation at

Classifiers are ways of classifying Tests, based on the Live-In Testing Standard.


Every Test must be of a particular type. The type is passed to the test as a template parameter, usually when inheriting as we create a new Test:

class TestSomeFeature : Test<Type::Behavior>

The LIT standard defines several types, which are formalized in Goldilocks with Type:

Type::Behavior (B)

A Behavior Test tests a single behavior or feature. This is the most common type of test, and is usually the best type for comparative benchmarks.

Type::Consistency (C)

A Consistency Test repeats one scenario a large number of times within known limits, and ensures the output is valid and consistent, usually either matching a target value or being in a range of values.

Type::Developer (D)

A Developer Test is usually similar to a unit test, but which does not fit into one of the other categories.

Type::Edge (E)

An Edge Test tests for edge cases, usually with specific scenarios.

Type::Fuzz (F)

A Fuzz Test creates random or non-deterministic scenarios, usually to simulate user input.

Type::Integration (I)

Integration Tests ensure that connected classes or systems are communicating with one another properly.

Type::Stress (S)

A Stress Test is designed to break more fragile code, usually by inputting large data. Should never be run automatically.

Type::Fatality (X)

A Fatality Test is intended to deliberately crash the program or system while logging, to determine limits. Should never be run automatically. Unlike a Stress Test, these tests are supposed to crash, and are usually written as infinite loops.

Should be used with caution. Goldilocks must display a warning and a complex-input prompt (e.g. requiring the user to type "I KNOW WHAT I AM DOING." before it runs. Should resist being run in a CI/CD. (Perhaps these should also have to be run via a special bombard command, instead of run.
Last Author
Last Edited
Jul 16 2021, 5:32 PM