33 lines
1.5 KiB
Markdown
33 lines
1.5 KiB
Markdown
|
|
The `run-tests.sh` script is called by various testers to do the work of
|
|
testing. Each test is actually fairly simple: it is a comparison of standard
|
|
output and standard error, as per the program specification.
|
|
|
|
In any given program specification directory, there exists a specific `tests/`
|
|
directory which holds the expected return code, standard output, and standard
|
|
error in files called `n.rc`, `n.out`, and `n.err` (respectively) for each
|
|
test `n`. The testing framework just starts at `1` and keeps incrementing
|
|
tests until it can't find any more or encounters a failure. Thus, adding new
|
|
tests is easy; just add the relevant files to the tests directory at the
|
|
lowest available number.
|
|
|
|
The files needed to describe a test number `n` are:
|
|
- `n.rc`: The return code the program should return (usually 0 or 1)
|
|
- `n.out`: The standard output expected from the test
|
|
- `n.err`: The standard error expected from the test
|
|
- `n.run`: How to run the test (which arguments it needs, etc.)
|
|
- `n.desc`: A short text description of the test
|
|
- `n.pre` (optional): Code to run before the test, to set something up
|
|
- `n.post` (optional): Code to run after the test, to clean something up
|
|
|
|
In most cases, a wrapper script is used to call `run-tests.sh` to do the
|
|
necessary work.
|
|
|
|
The options for `run-tests.sh` include:
|
|
* `-h` (the help message)
|
|
* `-v` (verbose: print what each test is doing)
|
|
* `-t n` (run only test `n`)
|
|
* `-c` (continue even after a test fails)
|
|
* `-d` (run tests not from `tests/` directory but from this directory instead)
|
|
|