Page MenuHomePhabricator

Goldilocks 1.0 run all
Closed, CompletedPublic

Description

What is the exact feature or behavior required?

A command line argument to run all test suites registered to a project

One or more example use cases for the feature, including sample expected input and output.

Running all suites at once instead of one at a time from command line.

An explanation of why the feature is needed. This is important to prioritization. “It would be nice if...” takes a backseat to “This common scenario doesn’t work without...”.

It would be much more convenient to run three test suites with one command instead of three ,for example :)

Details

Task Type
Feature
Proposed Urgency
Unknown
Gravity
g3: Major
Friction
f2: Street
Relativity
r1: Trivial
Volatility (Caught At)
Not a Bug
Origin
Not a Bug/Unknown

Revisions and Commits

Event Timeline

jcmcdonald changed the task status from Proposed to Open.Jan 27 2021, 12:30 PM
jcmcdonald claimed this task.
jcmcdonald triaged this task as p5: Emergency priority.
jcmcdonald changed Gravity from Triage Gravity to g3: Major.
jcmcdonald changed Relativity from Triage Relativity to r1: Trivial.
jcmcdonald changed Friction from Triage Friction to f2: Street.

@ardunster It seems I've added this. I'm pushing it directly, because it's a small change to a code base that won't live long. ;)

On your machine, you'll need to pull the latest devel for PawLIB. In the PawLIB directory on your machine, run:

git pull origin devel
make ready

Then recompile SIMPLEXpress. You should be able to run all the tests now:

make tester_debug
./tester_debug --runall

At the moment, I've only added it to the command line arguments. If one test fails, the whole thing stops.

Cool, it works! (One test failing stopping everything is how suites already ran, so basically what I expected.)

At the moment, I've only added it to the command line arguments. If one test fails, the whole thing stops.

By the way: this is not the actual behavior. If one test fails, that suite stops, but it still proceeds to run the remaining suites. Might lead to missed failures. I was expecting the test to fail, so scrolled up to look for it when I ran all and all the suites still ran.