Wykres commitów

44 Commity (47be69329bb6f24b2100f98515a93243e1d553b7)

Autor SHA1 Wiadomość Data
Sachin Parekh 6582f7070e Update tests for assert and abort
- Place panic_abort in IRAM
- Added abort, assert test case in case of cache disabled
- Expect assert instead of abort in a freertos_mutex test
2021-08-05 11:09:22 +05:30
Fu Hanxi 85d4bca81a feat(tiny_test_fw): unify all junit report test case name
new format: <target>.<config>.<case_name>, the default value of
"config" is "default"
2021-01-28 12:25:14 +08:00
Fu Hanxi 0146f258d7 style: format python files with isort and double-quote-string-fixer 2021-01-26 10:49:01 +08:00
Angus Gratton 6d3e06a9a3 ttfw: Move TestCaseFailed exception and handle differently to other exceptions
But also ensure the string form of this exception is never empty, if it ends up
somewhere else.
2021-01-12 11:42:03 +11:00
Angus Gratton 441d1306be ci ttfw: Consistently handle unexpected exceptions in test cases 2021-01-12 11:42:03 +11:00
He Yin Ling a4fab9c67f test: fix some unit test results not uploaded to Jira:
`UnitTestParser.py` will strip unit case config name when generate test
ID. But in `unit_test.py` we didn't do this. It makes the test ID in 2
stages are different. Now we will add this step to `unit_test.py`.
2020-12-15 21:36:27 +00:00
Fu Hanxi 377d3eaaa5 Move local target detection to `ttfw_idf`
rename upper_list to upper_list_or_str
minor fix for `unit_test.py` `is 'name'` -> `== 'name`
2020-06-01 12:08:57 +08:00
Ivan Grokhotkov f5d37c453a unit-test-app: make compatible with ESP32-S2 reset reasons
ESP32-S2 has slightly different names of reset reasons, printed by
the ROM. Allow using ESP32 reset reason names in test cases.
2020-01-29 19:05:33 +01:00
He Yin Ling 269af21ce6 test: fix multiple devices cases fail on python3 2019-12-30 20:21:00 +08:00
Angus Gratton 9b8fffef53 unit_test.py: Add some additional console logging for context 2019-12-12 11:05:05 +11:00
Angus Gratton 9fa87f04da unit_test.py: Allow identifying a unit test which has : in the name 2019-12-12 11:05:05 +11:00
Angus Gratton 0978fc1b84 unit_test: Log each test case to console as we run it
Include a dump of the test tags
2019-12-12 11:05:05 +11:00
Angus Gratton d35debcd99 unit_test: Print the timeout value when an 'expect' times out 2019-12-12 11:05:04 +11:00
Angus Gratton 563227d9f8 Merge branch 'bugfix/gitlab_api_error_with_python3' into 'master'
ci: fix fetch submodule error on python3

See merge request espressif/esp-idf!6961
2019-12-11 07:23:53 +08:00
He Yin Ling 486f1a05b3 test: fix incorrect error type in xunit result 2019-12-10 13:50:32 +08:00
Jakob Hasse 78e4b0ff96 CXX: enable exceptions ESP32S2-beta
Closes IDF-1039

* Enabled test cases for -fno-exceptions to S2-beta again
* Added different reset tag for S2-beta
2019-12-10 10:23:00 +08:00
He Yin Ling c906e2afee test: update example and unit tests with new import roles:
tiny_test_fw is a python package now. import it using normal way.
2019-12-07 10:34:54 +08:00
He Yin Ling 857d10a926 unit-test-app: print config name in test script 2019-11-23 20:40:23 +08:00
He Yin Ling 9665c6ef66 unit-test-app: fix test script error when handling multiple unit test configs in one config file:
If we have multiple configs, we need to flash DUT with different binaries. But if we don't close DUT before apply new config, the old DUT will be reused, so new config name will not be applied.
2019-11-23 20:40:23 +08:00
Ivan Grokhotkov 0e6e7f49be tiny-test-fw: add build config and target options 2019-11-21 15:36:06 +01:00
He Yin Ling 304e7119ce ci: fix random unit test CI failure:
we use `-` command to check if DUT reset pass. If we input `-` command
during DUT bootup, DUT could only receive `\n` and print test cases.
Print test cases could take long time and lead to reset check timeout.
Now we will add delay after reset, and enlarge reset check timeout to
solve this problem.
2019-06-18 12:22:00 +08:00
He Yin Ling b75400294a test: collect unit-test performance items and save to report 2019-04-19 14:39:44 +08:00
He Yin Ling 0a3975e80d test: handle exception by unit test script:
unit test script will detect exception, we don't need to let test fail
when exception happened.
2019-03-19 11:24:08 +08:00
He Yin Ling 323a790f01 test: fix CI UT job pass even ut app bootup fail 2019-03-10 06:21:37 +00:00
He Yin Ling 431653bba8 unit-test: fix 2 issues in unit test script:
1. Gitlab will only regard failure (not error) as failed case. Use log
failure instead of log error, to display all error msg in test result
2. fix failed case not recognized by test script when failed during
reset DUT
2019-01-10 09:51:17 +00:00
He Yin Ling 783cb692e8 unit-test-app: support passing parameter with signals 2019-01-10 09:51:17 +00:00
Roland Dobai bfa9610f58 tools: Fix the Python coding style 2018-12-19 11:56:24 +01:00
Angus Gratton 0a27cfa850 unit_test.py: Allow importing tiny-test-fw from IDF_PATH as well as TEST_FW_PATH 2018-12-18 14:09:05 +11:00
He Yin Ling b0154cbf58 unit-test-app: fix the RegEx to detect test case:
tags are not correctly parsed if eol do not contain `\r`
2018-11-29 23:06:16 +08:00
He Yin Ling 5b3d09d5c8 test: fix unit test script code style warnings 2018-11-26 10:07:41 +08:00
He Yin Ling 644571f78b test: generate junit test report according to executed cases 2018-11-26 10:07:41 +08:00
He Yin Ling 2b4e34171b test: fix UT script can't process cmd after reset:
There's still a small chance that can't process command issue still
happen on some runners. Now we change the way to detect if the runner
reset finished: input `-` to check test history for a few times until it
succeeded. If DUT can process check history command, then it should be
able to process other test commands.
2018-10-26 08:42:22 +00:00
He Yin Ling 44e8ec584f test: ut can't process test command:
check ut reset done condition is not correct. If bootup pattern already
exist in cache, we will return without waiting for DUT bootup.
2018-09-19 09:40:59 +08:00
houchenyao 73a763fecb fix the bug that multi device's dut can't detect case 2018-09-12 07:18:00 +00:00
houchenyao df83323ee3 bugfix:fix the multi device stop thread bug 2018-08-23 08:51:16 +00:00
Ivan Grokhotkov b600a908c4 unit test: fix regex for detecting abort 2018-08-20 16:49:20 +08:00
Roland Dobai 40486c8723 tools: Allow running unit tests from command line
Add argument parsing to unit_test.py to allow running unit tests from
command line.
2018-07-30 13:05:41 +02:00
He Yin Ling bffcea09df CI: erase nvs partition before test:
Latest NVS partition bin can't be parsed by old IDF revision. Need to erase before test.
2018-07-27 17:22:24 +08:00
houchenyao 9223639010 stop main thread when one failed 2018-06-26 11:02:23 +08:00
He Yin Ling 3b24bc42d2 CI: assign unit test cases according to sdkconfig:
We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).

Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.

In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
2018-05-24 03:40:03 +00:00
Angus Gratton e8525396dd test: Allow per-unit-test timeouts
Work around for "(WL) write/read speed test" taking >30s in some configs
2018-05-21 12:27:10 +00:00
Ivan Grokhotkov 48379b8bbe unit-test.py: increase timeout for test app start up 2018-04-26 18:52:45 +08:00
He Yin Ling fee844d22c unit-test-app: support multi stages test 2018-03-30 11:38:49 +08:00
houchenyao 5b8a9478a3 CI: new CI ut framework, and can run it in local PC 2018-02-05 06:58:22 +00:00