Page MenuHomePhabricator

Expects
Updated 381 Days AgoPublic

This is for Goldilocks 2.0. If you're using Goldilocks 1.0 (part of PawLIB), see the official documentation at https://docs.mousepawmedia.com/pawlib

Expects are the essential building blocks for Goldilocks testing. They compare the expected and actual outputs of an operation and automate the related reporting and returns.

Macros

Within a test, you would invoke Expects using one of three specialized macros, passing an Expect object (expect in the following examples):

REQUIRE(expect)

This requires the expectation to succeed, otherwise the test will fail (return false).

CHECK(expect)

This checks if the expectation succeeds, and reports the outcome accordingly, but does *not* fail the test if it fails. This is primarily intended for warning-level tests that are not considered important enough to abort an automatic CI/CD build, but which should still be known.

UNLESS(expect)

This requires the expectation to fail, otherwise the test will fail (return false). Seldom needed; use the Should::Fail template parameter instead (covered later.)

UNLESS() is intended for situations where the Expect should still be considered as having failed, and logged as such, but the test should pass because of the failure. A theoretical usage of this might be in writing a specialized test that confirms a point of failure is definitely scenario A, and not scenario B.

No, I'm not certain this is useful, but it's cheap to include, so it'll stay put for now.

Structure of an Expect

An Expect object is defined as follows:

Expect<That, Should>(args...)

That is the type of expectation. Should is the (optional) expected outcome. The arguments passed to the Expect depend on the type.

Expect Types (That)

There are a number of Expect types, each with a different behavior.

Code blocks in this section represent equivalent evaluation logic; actual implementation may differ.

Expect<That::IsTrue>(op)

Expects op to implicitly evaluate to true.

return (op == true);

If op is a non-null pointer, dereferences op and evaluates value. If op is nullptr, returns false.

return (op != nullptr && *op == true);

Expect<That::IsFalse>(op)

Expects op to implicitly evaluate to false.

return (op == false);

If op is a non-null pointer, dereferences op and evaluates value. If op is nullptr, returns false.

return (op != nullptr && *op == false);

Expect<That::IsEqual>(left, right)

Expects left and right to be equal.

return (left == right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left == *right);

Expect<That::IsNotEqual>(left, right)

Expects left and right to be not equal.

return (left != right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left != *right);

Expect<That::IsLess>(left, right)

Expects left to be less than right.

return (left < right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left < *right);

Expect<That::IsLessEqual>(left, right)

Expects left to be less than or equal to right.

return (left <= right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left <= *right);

Expect<That::IsGreater>(left, right)

Expects left to be greater than right.

return (left > right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left > *right);

Expect<That::IsGreaterEqual>(left, right)

Expects left to be greater than or equal to right.

return (left >= right);

If left and right are both non-null pointers, dereferences both and evaluates. If either is nullptr, returns false.

return (left != nullptr && right != nullptr && *left >= *right);

Expect<That::IsApproxEqual>(value, target, margin)

Expects value to be approximately equal to target, within the margin.

return ((value - target) < 0 ? ((value - target) * (-1.0)) < margin : (value - target) < margin));

If value is a non-null pointer, dereferences value and evaluates. If value is nullptr, returns false.

return (value != nullptr && (*value - target) < 0 ? ((*value - target) * (-1.0)) < margin : (*value - target) < margin);

Expect<That::IsApproxNotEqual>(value, target, margin)

Expects value to not be approximately equal to target, within the margin.

return ((value - target) < 0 ? ((value - target) * (-1.0)) > margin : (value - target) > margin);

If value is a non-null pointer, dereferences value and evaluates. If value is nullptr, returns false.

return (value != nullptr && (*value - target) < 0 ? ((*value - target) * (-1.0)) > margin : (*value - target) > margin));

Expect<That::IsInRange>(value, lower, upper)

Expects value to be in the inclusive range defined by lower and upper.

return (value >= lower && value <= upper);

If value is a non-null pointer, dereferences value and evaluates. If op is nullptr, returns false.

return (value != nullptr && *value >= lower && *value <= upper);

Expect<That::IsNotInRange>(value, lower, upper)

Expects value to be outside the inclusive range defined by lower and upper.

return (value < lower || value > upper);

If value is a non-null pointer, dereferences value and evaluates. If op is nullptr, returns false.

return (value != nullptr && (*value < lower || *value > upper));

Expect<That:PtrIsNull>(ptr*)

Expects ptr to be equal to nullptr.

return (ptr == nullptr);

Expect<That:PtrIsNotNull>(ptr*)

Expects ptr to be not equal to nullptr.

return (ptr != nullptr);

Expect<That:PtrIsEqual>(left*, right*)

Expects the pointers left and right to point to the same address.

return (left == right);

Expect<That:PtrIsNotEqual>(left*, right*)

Expects the pointers left and right do not point to the same address.

return (left != right);

Expect<That::FuncReturns>(target, name, func, args...)

Expects that the function call func(args...) returns the value target. name is used to inform the output of the function name.

return (func(args...) == target);

Expect<That::FuncThrows>(target, name, func, args...)

Expects that the function call func(args...) throws target. name is used to inform the output of the function name.

try {
    func(args...);
} catch (target) {
    return true;
} catch (...) {
    return false;
}
return false;

Expect Outcome (Should)

When defining an Expect, you may optionally provide a Should value as the second template parameter. The Should defines the expected outcome of the test.

  • Should::Pass is the default. The Expect will succeed if its expression evaluates to true, and will fail if its expression evaluates to false.
  • Should::Fail will cause the Expect to fail if its expression evaluates to true, and succeed if its expression evaluates to true.

Why Use Should::Fail?

This allows testing for expected failures separately from expected successes, which is critical in guarding against false positives. Consider the following fragment of a test function:

my_object = SomeObject();
REQUIRE(Expect<That::IsEqual>(my_object, target));
REQUIRE(Expect<That::IsNotEqual>(my_object, badTarget);

Internally, this is checking my_object == target and my_object != badTarget, but would still succeed even if there were a bug in SomeObject::operator==(), such as one causing it to always return true. Expect<That::IsNotEqual> uses the != operator, rather than ==, so it cannot catch that false positive.

Instead, you could use Should::Fail alongside these tests to safeguard against false positives:

my_object = SomeObject();

REQUIRE(Expect<That::IsEqual>(my_object, target));
REQUIRE(Expect<That::IsEqual, Should::Fail>(my_object, badTarget));

REQUIRE(Expect<That::IsNotEqual>(my_object, badTarget);
REQUIRE(Expect<That::IsNotEqual, Should::Fail>(my_object, target);

You'll notice that I'm not employing UNLESS() in this scenario Should::Fail is preferred for most inverse testing.

Last Author
jcmcdonald
Last Edited
Jul 29 2021, 6:57 AM