From 497bf226e2c331c9985566f9fb3c5ffa0b958f77 Mon Sep 17 00:00:00 2001
From: Nick Treleaven <n@trelsoft.com>
Date: Mon, 7 Sep 2020 12:48:15 +0100
Subject: [PATCH] add TESTS.md file to explain different tests (#6318)

---
 .github/PULL_REQUEST_TEMPLATE |  1 +
 CONTRIBUTING.md               |  6 +--
 TESTS.md                      | 69 +++++++++++++++++++++++++++++++++++
 cmd/v/help/other.txt          |  2 +-
 vlib/v/tests/inout/os.out     |  1 +
 5 files changed, 75 insertions(+), 4 deletions(-)
 create mode 100644 TESTS.md

diff --git a/.github/PULL_REQUEST_TEMPLATE b/.github/PULL_REQUEST_TEMPLATE
index ab6a178dfa..d6cbadfb00 100644
--- a/.github/PULL_REQUEST_TEMPLATE
+++ b/.github/PULL_REQUEST_TEMPLATE
@@ -30,6 +30,7 @@ Before submitting a PR, please:
 ./v -o v cmd/v
 ./v -o v cmd/v
 ```
+See also `TESTS.md`.
 
 I try to process PRs as soon as possible. They should be handled within 24 hours.
 
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index aeed3f5ee4..07d75c51e0 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -16,8 +16,8 @@ Be careful, if you introduce a breaking change and rebuild V, you will no longer
 be able to use V to build itself. So it's a good idea to make a backup copy of a
 working compiler executable.
 
-But don't worry, you can always simply run `make`, it will download the C
-version of the compiler and rebuild it from scratch.
+But don't worry, you can always simply run `make` (or `make.bat`), it will
+download the C version of the compiler and rebuild it from scratch.
 
 
 The main files are:
@@ -138,7 +138,7 @@ feature/bugfix* that you make.
 
 ### Testing your commits locally:
 You can test locally whether your changes have not broken something by
-running: `v test-compiler`
+running: `v test-compiler`. See `TESTS.md` for more details.
 
 ### Publishing your commits to GitHub:
 
diff --git a/TESTS.md b/TESTS.md
new file mode 100644
index 0000000000..5cb1e35ba8
--- /dev/null
+++ b/TESTS.md
@@ -0,0 +1,69 @@
+# Automated tests
+
+In the `v` repo there are several different tests. The main types are:
+
+* `_test.v` tests - check that `test_` functions succeed. These can be 
+run per directory or individually.
+* `.out` tests - run a `.vv` file and check the output matches the 
+contents of the `.out` file with the same base name. This is 
+particularly useful for checking that errors are printed.
+
+Tip: use `v -cc tcc` when compiling tests for speed.
+
+## `v test-compiler`
+
+This builds and tests:
+
+* `vc` repo
+* `_test.v` files
+* examples
+* `v install` a module
+
+Some of these can be invoked separately.
+
+## `vlib/v/compiler_errors_test.v`
+
+This runs tests for:
+* `checker/tests/*.vv`
+* `parser/tests/*.vv`
+
+## `vlib/v/tests`
+
+General runnable program tests.
+
+## Actual code
+
+* `v build-examples`
+* `v build-tools`
+
+## `v test-fixed`
+
+Test `vlib`.
+
+## vfmt tests
+
+In `vlib/v/fmt/` there's:
+
+* `fmt_test.v`
+
+This checks `.out` tests.
+
+* `fmt_keep_test.v`
+
+This verifies that `_keep.v` files would be unchanged by `vfmt -w`.
+
+* `fmt_vlib_test.v`
+
+This checks all source files are formatted and prints a summary. This 
+is not required.
+
+* `v test-fmt`
+
+Test all files in the current directory are formatted.
+
+## `.github/workflows/ci.yml`
+
+This runs various CI tests, e.g.:
+
+* `v vet vlib/v` - style checker
+* `v fmt -verify` on certain source files
diff --git a/cmd/v/help/other.txt b/cmd/v/help/other.txt
index 9d5703a36f..d7dcbc3a7a 100644
--- a/cmd/v/help/other.txt
+++ b/cmd/v/help/other.txt
@@ -7,7 +7,7 @@ but which are used less frequently by users:
    build-examples    Test if all examples can be built.
    build-tools       Test if all tools can be built.
    build-vbinaries   Test if V can be built with different configuration.
-   test-fmt          Test if all files in the current directory is formatted properly.
+   test-fmt          Test if all files in the current directory are formatted properly.
    test-compiler     Test if V is working properly by running all tests, including the compiler ones.
                      NB: this can take a minute or two to run
 
diff --git a/vlib/v/tests/inout/os.out b/vlib/v/tests/inout/os.out
index bd656fcf15..de7224eb57 100644
--- a/vlib/v/tests/inout/os.out
+++ b/vlib/v/tests/inout/os.out
@@ -2,3 +2,4 @@ CHANGELOG.md
 CODE_OF_CONDUCT.md
 CONTRIBUTING.md
 README.md
+TESTS.md