On this page:
13.1 Test Configuration by Submodule
13.2 Test Configuration by "info.rkt"
13.3 Responsible-Party and Varying-Output Logging
13.4 Logging Test Results
test-log!
test-report
test-log-enabled?
current-test-invocation-directory

13 raco test: Run tests🔗ℹ

The raco test command requires and runs the (by default) test submodule associated with each path given on the command line. Command-line flags can control which submodule is run, whether to run the main module if no submodule is found, and whether to run tests directly, in separate processes (the default), or in separate places. The current directory is set to a test file’s directory before running the file.

When an argument path refers to a directory, raco test recursively discovers and runs all files within the directory that end in a module suffix (see get-module-suffixes, but the suffixes always include ".rkt", ".scrbl", ".ss", and ".scm") or have a (possibly empty) list of command-line arguments provided by test-command-line-arguments in an "info.rkt" file, or as directed by test-include-paths in an "info.rkt" file. At the same time, raco test omits files and directories within a directory as directed by test-omit-paths in an "info.rkt" file.

A test is counted as failing if it logs a failing test code via test-log!, causes Racket to exit with a non-zero exit code, or (when -e or --check-stderr is specified) if it produces output on the error port.

The raco test command accepts several flags:

Changed in version 1.1 of package compiler-lib: Added --heartbeat.
Changed in version 1.4: Changed recognition of module suffixes to use get-module-suffixes, which implies recognizing ".ss" and ".rkt".
Changed in version 1.5: Added ++ignore-stderr.
Changed in version 1.6: Added ++arg and ++args.
Changed in version 1.8: Added --output and -o.
Changed in version 1.11: Added --make/-y.
Changed in version 1.12: Added --errortrace.

13.1 Test Configuration by Submodule🔗ℹ

When raco test runs a test in a submodule, a config sub-submodule can provide additional configuration for running the test. The config sub-submodule should use the info module language to define the following identifiers:

In order to prevent evaluation of a file for testing purposes, it suffices to create a submodule that does not perform any tests and does not trigger the evaluation of the enclosing module. So, for instance, a file might look like this:

#lang racket
 
(/ 1 0)
 
; don't run this file for testing:
(module test racket/base)

Changed in version 1.5 of package compiler-lib: Added ignore-stderr support.

13.2 Test Configuration by "info.rkt"🔗ℹ

Submodule-based test configuration is preferred (see Test Configuration by Submodule). In particular, to prevent raco test from running a particular file, normally the file should contain a submodule that takes no action.

In some cases, however, adding a submodule is inconvenient or impossible (e.g., because the file will not always compile). Thus, raco test also consults any "info.rkt" file in the candidate test file’s directory. In the case of a file within a collection, "info.rkt" files from any enclosing collection directories are also consulted for test-omit-paths and test-include-paths. Finally, for a file within a package, the package’s "info.rkt" is consulted for pkg-authors to set the default responsible parties (see Responsible-Party and Varying-Output Logging) for all files in the package.

The following "info.rkt" fields are recognized:

Changed in version 1.5 of package compiler-lib: Added test-ignore-stderrs support.

13.3 Responsible-Party and Varying-Output Logging🔗ℹ

When a test has a declared responsible party, then the test’s output is prefixed with a

  raco test:which @(test-responsible 'responsible)

line, where which is a space followed by an exact non-negative number indicating a parallel task when parallelism is enabled (or empty otherwise), and responsible is a string, symbol, or list datum.

When a test’s output (as written to stdout) is expected to vary across runs—aside from varying output that has the same form as produced by timethen it should be declared as varying. In that case, the test’s output is prefixed with a

  raco test:which @(test-random #t)

line.

13.4 Logging Test Results🔗ℹ

 (require raco/testing) package: compiler-lib

This module provides a general purpose library for tracking test results and displaying a summary message. The command raco test uses this library to display test results. Therefore, any testing framework that wants to integrate with raco test should also use this library to log test results.

Added in version 1.13 of package compiler-lib.

procedure

(test-log! result)  void?

  result : any/c
Adds a test result to the running log. If result is false, then the test is considered a failure.

procedure

(test-report [#:display? display? 
  #:exit? exit?]) 
  
(cons/c exact-nonnegative-integer?
        exact-nonnegative-integer?)
  display? : any/c = #f
  exit? : any/c = #f
Processes the running test log. The first integer is the failed tests, the second is the total tests. If display? is true, then a message is displayed. If there were failures, the message is printed on (current-error-port). If exit? is true, then if there were failures, calls (exit 1).

parameter

(test-log-enabled?)  boolean?

(test-log-enabled? enabled?)  void?
  enabled? : any/c
 = #t
When set to #f, test-log! is a no-op. This is useful to dynamically disable certain tests whose failures are expected and shouldn’t be counted in the test log, such as when testing a custom check’s failure behavior.

parameter

(current-test-invocation-directory)  (or/c #f path?)

(current-test-invocation-directory path)  void?
  path : (or/c #f path-string?)
 = #f
Contains the directory from which tests were invoked by, e.g., raco test. This may differ from current-directory when the test runner changes directory before invoking a specific test file and should be set by test runners to reflect the directory from which they were originally invoked.

This should be used by test reports to display appropriate path names.

Added in version 1.14 of package compiler-lib.