From 0aa4c6ce3177ca5e2111fa7ba2237ffe6d83b516 Mon Sep 17 00:00:00 2001 From: Remzi Arpaci-Dusseau Date: Fri, 1 Mar 2019 05:35:46 -0600 Subject: [PATCH] more description of testing framework --- tester/README.md | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/tester/README.md b/tester/README.md index 6761303..71f29a2 100644 --- a/tester/README.md +++ b/tester/README.md @@ -4,15 +4,21 @@ testing. Each test is actually fairly simple: it is a comparison of standard output and standard error, as per the program specification. In any given program specification directory, there exists a specific `tests/` -directory which holds the expected standard output and error in files called -`n.out` and `n.err` for each test `n`. The testing framework just starts at -`1` and keeps incrementing tests until it can't find any more or encounters a -failure. Thus, adding new tests is easy; just add the relevant files to the -tests directory at the lowest available number. +directory which holds the expected return code, standard output, and standard +error in files called `n.rc`, `n.out`, and `n.err` (respectively) for each +test `n`. The testing framework just starts at `1` and keeps incrementing +tests until it can't find any more or encounters a failure. Thus, adding new +tests is easy; just add the relevant files to the tests directory at the +lowest available number. -A couple of other files are needed to fully describe a test. The first is a -run file (e.g., `n.run`) which shows exactly how to run the test; the second -is a description file (e.g., `n.desc`) which just describes the test. +The files needed to describe a test number `n` are: +- `n.rc`: The return code the program should return (usually 0 or 1) +- `n.out`: The standard output expected from the test +- `n.err`: The standard error expected from the test +- `n.run`: How to run the test (which arguments it needs, etc.) +- `n.desc`: A short text description of the test +- `n.pre` (optional): Code to run before the test, to set something up +- `n.post` (optional): Code to run after the test, to clean something up In most cases, a wrapper script is used to call `run-tests.sh` to do the necessary work. @@ -22,5 +28,5 @@ The options for `run-tests.sh` include: * `-v` (verbose: print what each test is doing) * `-t n` (run only test `n`) * `-c` (continue even after a test fails) - +* `-d` (run tests not from `tests/` directory but from this directory instead)