From ca29765e977d6b7b9306e9fe8af034a45ac9088f Mon Sep 17 00:00:00 2001 From: Angus Gratton Date: Wed, 17 Feb 2021 18:06:21 +1100 Subject: [PATCH] test apps: Add some documentation about test app configs --- tools/test_apps/README.md | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/tools/test_apps/README.md b/tools/test_apps/README.md index 0d21877701..961aab02a0 100644 --- a/tools/test_apps/README.md +++ b/tools/test_apps/README.md @@ -10,13 +10,42 @@ The test apps are not intended to demonstrate the ESP-IDF functionality in any w Test applications are treated the same way as ESP-IDF examples, so each project directory shall contain * Build recipe in cmake and the main component with app sources -* Configuration files - - `sdkconfig.ci` - Default configuration for the project - - `sdkconfig.ci.` - Other configurations, where `` indicates name of the configuration +* Configuration files, `sdkconfig.ci` and similar (see below) * Test executor in `ttfw_idf` format if the project is intended to also run tests (otherwise the example is build only) - test file in the project dir must end with `_test.py`, by should be named `app_test.py` - test cases shall be decorated with `@ttfw_idf.idf_custom_test(env_tag="...")` + +# CI Behavior + +## Configuration Files + +For each project in test_apps (and also examples): + +* If a file `sdkconfig.ci` exists then it's built as the `default` CI config (same as if this file was named `sdkconfig.ci.default`) +* If any files `sdkconfig.ci.` exist then these are built as alternative configs, with the specified `` name. + +* By default, every CI configurations is built for every target SoC (an `m * n` configuration matrix). However if any `sdkconfig.ci` file contains a line of the form `CONFIG_IDF_TARGET="targetname"` then that CI config is only built for that one target. +* Each configuration is also built with the contents of any `sdkconfig.defaults` file or a file named `sdkconfig.defaults.` appended. (Same as a normal ESP-IDF project build.) + +## Test Execution + +If an example test or test app test supports more targets than just `ESP32`, then the `app_test.py` file needs to specify the list of supported targets in the test decorator. For example: + +``` +@ttfw_idf.idf_example_test(env_tag='Example_GENERIC', target=['esp32', 'esp32s2']) +def test_app_xyz(env, extra_data): +``` + +If the app test supports multiple targets but you only want some of these targets to be run automatically in CI, the list can be further filtered down by adding the `ci_target` list: + +``` +@ttfw_idf.idf_example_test(env_tag='Example_GENERIC', target=['esp32', 'esp32s2'], ci_target=['esp32']) +def test_app_xyz(env, extra_data): +``` + +(If no ci_target list is specified, all supported targets will be tested in CI.) + # Test Apps layout The test apps should be grouped into subdirectories by category. Categories are: