From 65c8e18a4ca69dd8b30d7e79cab8123cb829b6fe Mon Sep 17 00:00:00 2001 From: Remzi Arpaci-Dusseau Date: Sat, 16 Feb 2019 19:16:35 -0500 Subject: [PATCH] description of tester --- tester/README.md | 26 ++++++++++++++++++++++++++ tester/run-tests.sh | 2 +- 2 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 tester/README.md diff --git a/tester/README.md b/tester/README.md new file mode 100644 index 0000000..6c9aa2d --- /dev/null +++ b/tester/README.md @@ -0,0 +1,26 @@ + +The 'run-tests.sh' script is called by various testers to do the work of +testing. Each test is actually fairly simple: it is a comparison of standard +output and standard error, as per the program specification. + +In any given program specification directory, there exists a specific `tests/` +directory which holds the expected standard output and error in files called +`n.out` and `n.err` for each test `n`. The testing framework just starts at +`1` and keeps incrementing tests until it can't find any more or encounters a +failure. Thus, adding new tests is easy; just add the relevant files to the +tests directory at the lowest available number. + +A couple of other files are needed to fully describe a test. The first is a +run file (e.g., `n.run`) which shows exactly how to run the test; the second +is a description file (e.g., `n.desc`) which just describes the test. + +In most cases, a wrapper script is used to call `run-tests.sh` to do the +necessary work. + +The options for `run-tests.sh` include: +* `-h` (the help message) +* `-v` (verbose: print what each test is doing) +* `-t n` (run only test `n`) +* `-c` (continue even after a test fails) + + diff --git a/tester/run-tests.sh b/tester/run-tests.sh index 70cbefc..de650d6 100755 --- a/tester/run-tests.sh +++ b/tester/run-tests.sh @@ -72,7 +72,7 @@ run_and_check () { # usage: call when args not parsed, or when help needed usage () { - echo "usage: run-tests.sh [-h] [-v] [-t test]" + echo "usage: run-tests.sh [-h] [-v] [-t test] [-c]" echo " -h help message" echo " -v verbose" echo " -t n run only test n"