From e94bc87ebe8104d24d23b60487e4c5a7e480ea29 Mon Sep 17 00:00:00 2001
From: Gilles Peskine <Gilles.Peskine@arm.com>
Date: Fri, 27 Sep 2019 20:22:41 +0200
Subject: [PATCH] Document test case descriptions

---
 docs/architecture/testing/test-framework.md | 30 +++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/docs/architecture/testing/test-framework.md b/docs/architecture/testing/test-framework.md
index 270a3f9be..dd2968e0f 100644
--- a/docs/architecture/testing/test-framework.md
+++ b/docs/architecture/testing/test-framework.md
@@ -4,6 +4,36 @@ This document is an overview of the Mbed TLS test framework and test tools.
 
 This document is incomplete. You can help by expanding it.
 
+## Unit tests
+
+See https://tls.mbed.org/kb/development/test_suites
+
+### Unit test descriptions
+
+Each test case has a description which succinctly describes for a human audience what the test does. The first non-comment line of each paragraph in a `.data` file is the test description. The following rules and guidelines apply:
+
+* Test descriptions may not contain semicolons, line breaks and other control characters, or non-ASCII characters. <br>
+  Rationale: keep the tools that process test descriptions (`generate_test_code.py`, [outcome file](#outcome-file) tools) simple.
+* Test descriptions must be unique within a `.data` file. If you can't think of a better description, the convention is to append `#1`, `#2`, etc. <br>
+  Rationale: make it easy to relate a failure log to the test data. Avoid confusion between cases in the [outcome file](#outcome-file).
+* Test descriptions should be a maximum of **66 characters**. <br>
+  Rationale: 66 characters is what our various tools assume (leaving room for 14 more characters on an 80-column line). Longer descriptions may be truncated or may break a visual alignment. <br>
+  We have a lot of test cases with longer descriptions, but they should be avoided. At least please make sure that the first 66 characters describe the test uniquely.
+* Make the description descriptive. “foo: x=2, y=4” is more descriptive than “foo #2”. “foo: 0<x<y, both even” is even better if these inequalities and parities are why this particular test data was chosen.
+* Avoid changing the description of an existing test case without a good reason. This breaks the tracking of failures across CI runs, since this tracking is based on the descriptions.
+
+`tests/scripts/check-test-cases.py` enforces some rules and warns if some guidelines are violated.
+
+## TLS tests
+
+### SSL extension tests
+
+#### SSL test case descriptions
+
+Each test case in `ssl-opt.sh` has a description which succinctly describes for a human audience what the test does. The test description is the first parameter to `run_tests`.
+
+The same rules and guidelines apply as for [unit test descriptions](#unit-test-descriptions). In addition, the description must be written on the same line as `run_test`, in double quotes, for the sake of `check-test-cases.py`.
+
 ## Running tests
 
 ### Outcome file