Overview

Critical Test

Critical Test is a CLI tool designed to provide an enhanced CLI experience when running Dart unit tests.

Fixing broken unit tests is an ongoing job in any active project. Whilst it is usually better to run your unit tests from within your IDE, in some circumstances, this isn't possible or convenient.

Critical Test runs your unit tests from the CLI and makes it easy to identify broken tests and re-run those tests.

By default Critical Test suppresses the output of any tests that succeed so you can focus on those critical failed tests.

Critical Test then lets you re-run individual failed tests or re-run all failed tests.

Critical Test also provides an enhanced view of failed unit tests making it easier to review those tests.

Help support Critical Test by supporting OnePub, the private Dart repository.

OnePub allows you to privately share Dart packages across your Team and with your customers.

Try it for free and publish your first private package in seconds.

Publish a private package in five commands:

dart pub global activate onepub

onepub login

cd <my package>

onepub pub private

dart pub publish

To run all tests simply run critical_test from the root of your project.

dart pub global activate critical_test
cd <your project root>
critcal_test

Run a single test

If a test fails, Critical Test outputs instructions on how to re-run that single test.

You will see a blue line just above the 'END ERROR' line with the instructions.

package:test_api           fail
test/test2_test.dart 13:7  main.<fn>.<fn>

Rerun test via: critical_test --single=test/test2_test.dart
******************************** END ERROR (2) *********************************
6:2:0:0 test2_test.dart: Tests completed with some failures

To re-run the failed test:

critical_test --single=test/test2_test.dart

Re-run all failed tests

Each time you do a test run (except when --single is used) Critical Test tracks each of the failed tests.

You can re-run just the failed tests by running:

critical_test --runfailed

exit codes

You can check how the test ran via the critical test exit code.

  • 0 - all tests passed

  • 1 - some tests failed

  • 5 - no tests were run

progress

The --[no]-progress flag allows you to control whether progress messages are output.

By default progress messages are displayed.

When using Critical Test in a CI pipeline we recommend running with --no-progress as this reduces the amount of clutter in your CI's logs.

In this mode, you will only get a small intro message and a completion message. If any errors are generated they are still logged to the console in full.

show

When Critical Test runs, it normally suppresses the output of any tests that succeed.

You can use the --show command-line switch to run the test showing output from both failed and successful tests.

critical_test --show

Selecting tests to run

Critical Test takes two command-line arguments that allow you to control which tests are run:

  • --tags=<tag expression>

  • --exclude-tags=<tag expression>

Both of these flags are passed directly to the unit test package and so must conform to the documented tag expression syntax.

e.g. --tags="(chrome || firefox) && !slow" --exclude-tags=slow

logPath

By default critical_tests logs both successful and failed tests to <system temp dir>/critical_test/unit_tests.log.

You can modify the file the unit tests are logged to via:

critical_test --logPath=<somepath>

Monitoring progress

Critical Test provides a single updating line that shows the progress of the unit tests.

    2:1:0 test_test.dart: Loading

The three numbers at the start of the line, in order are:

  • Successes - shown in green

  • Errors - shown in red

  • Skipped - shown in blue

You can also monitor the full output of the unit tests (including successful unit tests) by tailing the log file:

tail -f /<system temp dir>/critical_test/unit_tests.log

Settings.yaml

Critical Test allows you to provide default values for any of the command line arguments by creating a settings.yaml file in the tool/critical_test directory.

To set a default for a command line argument use the same name in the settings file:

<project root>/tool/critical_test/settings.yaml

tags: ['backend','frontend']
log-path: /tmp/log.tmp

DCli

Critical Test was written in Dart using DCli to support the Conduit project.

NextPre/Post test hooks.

Last updated

Was this helpful?