2dd12ae5f8
spi: fix speed test issues on esp32s2beta
...
Also support performance value for different targets.
2019-12-23 10:23:01 +08:00
c63684cf6c
hw crypto: activated hardware acceleration for esp32s2beta
...
Activated AES, RSA and SHA hardware acceleration for esp32s2 and enabled related unit tests.
Updated with changes made for ESP32 from 0a04034961f59fcaea288
2019-12-12 12:37:29 +08:00
f119cc58cf
components/freertos: tuned performance value for ISR exit cycles to pass in test in -Og
2019-12-04 10:39:22 -03:00
bcdc35be59
components/freertos: refactor of isr_latency tests to perform full measurement
2019-12-04 10:39:22 -03:00
346b12e29a
freertos/test: added spill register timer measurement test
2019-12-04 10:39:22 -03:00
d158fa3f5c
sdio: add unit test between host and slave
2019-12-03 22:58:42 +08:00
5b33d6cf94
Merge branch 'feature/mbedtls_add_faster_modexp' into 'master'
...
mbedtls: Add a new (X^Y) mod M implementation (HAC 14.94)
Closes IDF-965
See merge request espressif/esp-idf!6418
2019-11-06 15:51:28 +08:00
e8d3b80e4b
mbedtls: Add an UT for performance RSA key operations
...
(New) - Montgomery exponentiation: Z = X ^ Y mod M (HAC 14.94)
keysize = 2048 bits
RSA key operation (performance): public [21894 us], private [199119 us]
RSA key operation (performance): public [18768 us], private [189051 us]
RSA key operation (performance): public [16242 us], private [190821 us]
keysize = 3072 bits
RSA key operation (performance): public [39762 us], private [437480 us]
RSA key operation (performance): public [36550 us], private [449422 us]
RSA key operation (performance): public [40536 us], private [443451 us]
keysize = 4096 bits
RSA key operation (performance): public [65671 us], private [885215 us]
RSA key operation (performance): public [60770 us], private [880936 us]
RSA key operation (performance): public [68951 us], private [872027 us]
(Old) - Sliding-window exponentiation: Z = X ^ Y mod M (HAC 14.85)
keysize = 2048 bits
RSA key operation (performance): public [93206 us], private [280189 us]
RSA key operation (performance): public [93060 us], private [278893 us]
RSA key operation (performance): public [97520 us], private [283252 us]
keysize = 3072 bits
RSA key operation (performance): public [293614 us], private [858157 us]
RSA key operation (performance): public [289902 us], private [843701 us]
RSA key operation (performance): public [291495 us], private [845232 us]
keysize = 4096 bits
RSA key operation (performance): public [653192 us], private [1912126 us]
RSA key operation (performance): public [656661 us], private [1901792 us]
RSA key operation (performance): public [641390 us], private [1938911 us]
2019-11-05 16:33:11 +08:00
589a1f216f
mbedtls: add SHA performance test
...
Results with this revision:
SHA256 rate 2.599MB/sec Debug 240MHz SW
SHA256 rate 1.147MB/sec Release 80MHz SW
SHA256 rate 3.469MB/sec Release 240MHz SW
SHA256 rate 2.687MB/sec Release 240MHz SW + PSRAM workaround
SHA256 rate 9.433MB/sec Debug 240MHz HW rev1
SHA256 rate 3.727MB/sec Release 80MHz HW rev1
SHA256 rate 10.961MB/sec Release 240MHz HW rev1
SHA256 rate 9.966MB/sec Release 240MHz HW rev1 + PRAM workaround
SHA256 rate 10.974MB/sec Debug 240MHz HW rev3
SHA256 rate 4.362MB/sec Release 80MHz HW rev3
SHA256 rate 13.207MB/sec Release 240MHz HW rev3
Debug = Og, assertions enabled
Release = O2, assertions disabled
2019-11-04 10:48:08 +01:00
d10107a50e
idf_test: Disable tests for Wi-Fi auto reconnect
...
Disable one, remove IDF tag from the other
2019-09-17 18:41:09 +10:00
6fa20a4d33
Merge branch 'test/use_correct_config_for_nvs_compatible_case' into 'master'
...
test: fix nvs compatible case error:
See merge request espressif/esp-idf!5890
2019-09-02 12:40:38 +08:00
954a23be37
test: fix nvs compatible case error:
...
nvs compatible test case uses new test env and app. update test config
file.
2019-08-25 20:35:49 +08:00
2116968d78
test: fix test case MESH_EST_2403
2019-08-16 09:15:31 +00:00
caea2889c8
aes: Add fault injection checks when writing key to hardware
...
Vulnerability reported by LimitedResults under Espressif Bug Bounty Program.
2019-08-11 11:16:33 +10:00
02e5266778
modify WIFI_CONN_0101 case for wrong authmode of AP
2019-08-02 10:22:46 +08:00
fa02598b5c
lwip_2.1.2 for idf_4.0
2019-07-07 01:51:45 +00:00
81dd9d4e27
CI: use parallel attribute in CI config file
2019-07-03 16:53:34 +08:00
f14563a47d
test: modify test cases for bluedroid
2019-06-25 23:14:32 +08:00
9eccd7c082
components: use new component registration api
2019-06-21 19:53:29 +08:00
2deb267be5
test: fix some wifi case issues:
...
* remove heap size check cases as we have bg tasks allocate memory
* fix wifi connect to open ap issue
2019-06-13 11:14:39 +08:00
0b70dfc27f
Add floating point performance test
2019-05-30 10:09:24 +10:00
c9edb7c8a9
mbedtls: Add AES-CBC performance test
...
Hardware AES engine gets 11.0MB/sec on Release config
Software AES is around 2.3MB/sec on Release config
2019-05-21 12:41:40 +10:00
ffec9d4947
components: update with build system changes
2019-05-13 19:59:17 +08:00
9549ae4c12
test: sync integration test cases
2019-04-19 16:44:00 +08:00
801e2cdaa3
test: update mesh test cases for ci
...
add mesh cases in files:
TC_IT_MESH_EST.yml, TC_IT_MESH_COMM.yml and TC_IT_MESH_MISC.yml
2019-04-17 17:05:33 +08:00
12bdf8e45b
esp32: Chunk input blocks for esp_sha() function performance, add perf test
2019-03-14 05:56:06 +00:00
932f2707ee
modify test case to support ipv6
2019-01-26 13:02:04 +00:00
e1ac7ed922
test: modify the reason code from 204 to 15 in WIFI_CONN_0904
...
1. remove the step of testing station connect using a wrong password.
2019-01-15 22:37:49 +08:00
7c6211fe45
Component/bt: fix save keys when auth mode without bond
2018-12-13 15:43:31 +08:00
738c56e84a
Fix VFS_OPEN_WRITE_CLOSE_TIME unit test issue
...
Closes idf/esp-idf#44
2018-11-28 20:36:08 +01:00
86fedc3cbb
utest: increase IDF_PERFORMANCE_MAX_VFS_OPEN_WRITE_CLOSE_TIME to 50000
2018-11-21 19:14:07 +08:00
6091021e83
unity: separate common and IDF specific functionality
...
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
2018-11-19 12:36:31 +08:00
242724bbe9
Merge branch 'test/fix_bug_of_scan_interval_and_windows' into 'master'
...
test: fix bug of scan interval and window
See merge request idf/esp-idf!3606
2018-11-05 11:00:58 +08:00
d28c99d36c
esp_event: lower test performance threshold for PSRAM config
2018-11-01 13:50:08 +08:00
d336f1513f
test: fix bug of scan interval and window
2018-11-01 11:54:52 +08:00
5a83347bec
event: Implement event loop library
2018-10-26 13:14:19 +08:00
5d71940178
component/bt: add sequential check when add include service
2018-09-30 14:19:54 +08:00
4af51833f3
spi_master: add new polling mode to decrease time cost each transaction
2018-09-20 19:46:46 +08:00
57573d0ed7
Merge branch 'feature/btdm_add_clear_rand_addr_API' into 'master'
...
component/bt: add clear rand address API
See merge request idf/esp-idf!3137
2018-09-20 10:47:37 +08:00
60f0e9b77b
test: fix case use depreacated sta connect command
2018-09-19 13:31:56 +08:00
da6479e6b7
Correct Python 3 support for some test related scripts
2018-09-18 09:55:51 +00:00
fa00e6d4f7
component/bt: add clear rand address API
2018-09-17 20:09:03 +08:00
4b41bd0cdd
test: fix incorrect tx power value in adv
2018-09-13 09:47:23 +08:00
cff2ef695a
Merge branch 'master' into feature/cmake
2018-09-06 20:23:04 +08:00
3578fe39e0
Feature/sync lwip as submodule
2018-09-06 19:43:08 +08:00
b9f7e0a1a2
test: split CI test environment:
...
Currently CI test environment is complicated, usually includes NIC, AP.
These devices are not used for a lot test cases. Therefore we split the
test environment, remove unnecessary devices. Then it will be easier to
create CI runners.
2018-09-05 01:45:48 +00:00
a9c4ed7139
Merge branch 'master' into feature/cmake
2018-08-30 18:51:01 +08:00
d04c101c96
test: fix bug of BTSTK_GATT_25007
2018-08-28 12:00:15 +08:00
ef0287de7d
fix bug BTSTK_GAP_14007,BTSTK_GAP_14008,BTSTK_GAP_14009 duplicate summary
2018-08-24 14:25:24 +08:00
32171da525
test: disable mesh recreate network test:
...
We have fixed a bug in test script. Before that, we didn't do strict check when generate the mesh tree, which makes the test pass.
Temp disable these cases, to let CI pass before bug fixed.
2018-08-23 17:49:59 +08:00
ff2404a272
Merge branch 'master' into feature/cmake
2018-08-16 17:14:17 +10:00
28ab4acb7e
test: update Wi-Fi test cases:
...
1. The test step for error code REASON_ASSOC_EXPIRE is not reliable. SoftAP
could also send error code WIFI_REASON_NOT_AUTHED
2. STA will be disconnected now if softAP restart DHCP server
2018-08-07 10:50:05 +00:00
964087b7c8
freertos: bump limit for spinlock performance test to 300 cycles
2018-07-24 09:56:40 +03:00
baa0b8547c
test: add mesh basic function test cases
2018-06-20 05:59:54 +00:00
0b447f0003
test/fix bug refresh cache after connect
2018-06-06 02:17:06 +00:00
0796099c29
test: filter duplicated ble adv events in test
2018-06-04 14:14:40 +08:00
6b9784cc45
Merge branch 'master' into feature/cmake_update
2018-05-31 14:46:23 +10:00
cb649e452f
remove executable permission from source files
2018-05-29 20:07:45 +08:00
340ad5c430
Component/bt: fix scan duplicate
2018-05-21 15:23:15 +08:00
a2241fb4bc
tests: Increase performance threshold for VFS running with PSRAM
...
Cache effects on microbenchmarks in cached flash can be significant, depending on linker layout in flash.
2018-05-17 21:02:47 +08:00
e8dd203e47
example test: add test case for iperf example
2018-05-15 13:42:26 +08:00
5129bca67c
VFS: Use smaller numbers as file descriptors
2018-05-07 09:01:56 +02:00
1cb5712463
cmake: Add component dependency support
...
Components should set the COMPONENT_REQUIRES & COMPONENT_PRIVATE_REQUIRES variables to define their
requirements.
2018-04-30 09:59:20 +10:00
4f1a856dbf
cmake: Remove defaults for COMPONENT_SRCDIRS, COMPONENT_SRCS, COMPONENT_ADD_INCLUDEDIRS
...
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
2018-04-30 09:59:20 +10:00
c671a0c3eb
build system: Initial cmake support, work in progress
2018-04-30 09:59:20 +10:00
7cfd08deca
test: fix join AP fail during AP power on
2018-04-16 09:27:46 +08:00
f409a33653
test: update ci test case
2018-04-10 14:08:10 +00:00
595ddfd825
Merge branch 'feature/build_ssc_in_multiple_ci_jobs' into 'master'
...
CI: support build SSC in multiple jobs
See merge request idf/esp-idf!1622
2018-04-03 18:20:38 +08:00
3b3ad875c4
test: use SSC_BLE_WIFI to test NVS cases
2018-04-02 18:13:52 +08:00
48e3e3efca
test: change SSC binary path
2018-03-29 19:40:07 +08:00
15be0829fa
test(spi_master): test spi master speed performance by median value.
...
also increase the boundary of SPI limit by a little.
2018-03-16 16:37:19 +08:00
62f924544d
Merge branch 'test/spi_performance' into 'master'
...
test(spi_master): add performance display for spi master.
See merge request idf/esp-idf!1923
2018-02-08 22:59:40 +08:00
a151767426
test(spi_master): add performance display for spi master.
2018-02-07 17:50:13 +08:00
70ab924dbb
Especially when internal memory fills up, some FreeRTOS structures (queues etc) get allocated in psram. These structures also contain a spinlock, which needs an atomic-compare-swap operation to work. The psram hardware, however, does not support this operation. As a workaround, this patch detects these spinlocks and will, instead of S32C1I, use equivalent C-code to simulate the behaviour, with an (internal) mux for atomicity.
2018-02-02 17:11:06 +08:00
63a401057c
test: enlarge https bin size threshold:
...
use performance matrix to tracking bin size is not convenient. bin size
do change for a lot reasons.
we'll implement a new method to track it. before that, set a large
enough threshold, so we can still keep tracking it.
2018-02-01 09:57:00 +08:00
c9607e9753
test: fix BLE update conn param test case errors
2018-01-16 13:39:12 +08:00
6e1453e864
Merge branch 'feature/bot' into 'master'
...
CI: support customize test with @bot
See merge request !1502
2017-11-21 18:43:56 +08:00
81f806e676
test: change initial condition of unit test:
...
now UT App will not print test menu by default. it will print a prompt
to indicate it's ready. we can check this prompt instead of delay.
2017-11-20 17:51:27 +08:00
9395a702ef
test: check and collect performance via CI
2017-11-16 17:48:30 +08:00
f80824a2d8
test: support test for UT cases expect to reset
2017-11-07 12:19:39 +08:00
363a20abec
test: udpate new BLE known issues:
...
1. some cases caused by GATTC can't get correct attribute handle
2. some cases with increased fail ratio
2017-09-22 09:49:18 +08:00
09a3e4fd58
squesh the commit.
...
component/bt: Change the idf test KnownIssues file.
component/bt: Fixed the bug of gattc mutilple read.
2017-09-14 09:35:36 -04:00
ac5e1f65b2
test:update BLE GATT MTU cases
2017-09-07 13:46:13 +08:00
cf64c7e9a6
unit test: add sd env and spi env for sdmmc
2017-08-22 21:01:34 +08:00
31f7e8d7e5
CI: add BLE test cases to CI
2017-06-27 10:35:33 +08:00
2e7f6a2247
test: add NVS compatible test case and CI job
2017-06-04 10:29:15 +08:00
659cd2af43
CI: remove readme content:
...
README will be updated in internal wiki
2017-05-08 11:44:45 +08:00
24862b1a6c
CI: remove CI configs and update test case files:
...
we'll auto assign cases to test jobs so we don't need those config
files. cases with different wifi modes will now be generated when
creating test case database, we need to remove them from YAML files.
2017-05-08 11:44:38 +08:00
e9913a11c1
Merge branch 'test/reboot_between_unit_test_cases' into 'release/v2.0'
...
CI: add reset between running each unit test cases
Unit test is designed to detect bug **within** each test case. Therefore we'll reset between each case to provide a clean context. We will later add stress cases to run unit test cases together to detect potential bugs.
See merge request !636
2017-04-10 12:59:49 +08:00
0501507d6b
CI: add script for parsing unit test cases for CI from test files
...
Add python script that parses list of unit test cases for CI from component test folder
Modify .gitlab-ci.yml to run this script as part of build unit tests stage
2017-01-18 17:08:20 +08:00
6c41080fd9
CI: update known issues
2017-01-04 19:27:58 +08:00
b9a6d276a2
CI: update known issues
2017-01-03 14:06:50 +08:00
c47bc43afd
CI: fix bug that restore initial condition fail:
...
in STA+AP mode, when STA is reconnecting to external AP, its AP
interface is not connectable. do disconnect on STA interface first
2017-01-03 14:06:50 +08:00
8f3c74353a
newlib: fix unit test name in test config
2016-12-13 13:24:01 +08:00
a283042258
switch socket recv print for stress test
2016-12-09 10:45:10 +08:00
5e1cdd3422
fix bug in test case TCPIP_TCP_5101
...
test time is not set in test case and not used to control test time
2016-12-08 16:15:49 +08:00
ec40053720
add supported stress test cases to CI
2016-12-06 15:41:02 +08:00
13e7e69239
remove unexisting tests in unit test CIConfig
2016-11-23 16:22:51 +08:00
47910466ce
Set result "Succeed" for passed cases
2016-11-02 20:54:22 +08:00
e56b745527
incorrect "/", should be "\"
2016-11-02 20:21:46 +08:00
c12aeb1127
add case select by name mark " before case name
2016-11-02 20:12:43 +08:00
2e319705ec
forget to break when test succeed
2016-11-02 19:41:33 +08:00
fdff6b2b01
run unit test case by case name
2016-11-02 19:08:55 +08:00
5784586041
integrate unit test to CI
2016-11-01 19:30:42 +08:00
3edcf5b096
fix bug for case WIFI_CONN_0102:
...
after set AP, STA will disconnected and do reconnect. scan could fail as reconnect also use scan
2016-10-24 18:59:56 +08:00
abd4dc7d43
add know issue ^WIFI_CONN_0601:
...
in STA+AP mode, when STA reconencting to enternal AP, external STA can't connect to AP
2016-10-24 18:42:40 +08:00
d3d8c04535
add known issue exception when setting mac address:
...
WIFI_ADDR_0101,^WIFI_ADDR_0101
2016-10-20 16:43:04 +08:00
5e8849d032
modify TCPIP_DHCP_0211:
...
delay 30s to make sure STA got enough time to disconnect and reconnect.
2016-10-20 11:28:06 +08:00
c350972eac
update following know issues:
...
1. ^WIFI_CONN_0801: auth change event not correct
2. WIFI_SCAN_0303: scan interact with JAP not correct
3. ^WIFI_SCAN_0103,^WIFI_SCAN_0105: no scan done event
4. ^TCPIP_UDP_0304,TCPIP_UDP_0104: UDP poor performance
2016-10-19 20:22:43 +08:00
18ebc6411e
fix bug for WIFI_SCAN_0304: need to set AP config first
2016-10-19 18:03:43 +08:00
6eaf595a0e
change test WAN server URL:
...
form iot.espressif.cn to factory.espressif.cn
2016-10-19 18:00:27 +08:00
0324373cd1
update auto generated CI job config
2016-10-19 15:23:07 +08:00
c2b63a614e
sync test env from auto_test_script (add SSC_T5_IOT1)
2016-10-19 15:22:13 +08:00
de7b2f5a09
fix following test case bugs:
...
1. (^)TCPIP_DHCP_0101, (^)TCPIP_DHCP_0301: IDF set IP 0.0.0.0 may return error, don't check the result of setting IP to 0.0.0.0;
2. rename (^)TCPIP_DHCP_0102 to (^)TCPIP_DHCP_0212 as it's a DHCP server test case;
3. (^)TCPIP_TCP_0204,(^)TCPIP_TCP_0210,(^)TCPIP_UDP_0201,(^)TCPIP_UDP_0202: recv thread can't be deleted, change case to not create recv thread when create socket ;
4. (^)TCPIP_TCP_0206,(^)TCPIP_TCP_0212: query TCP server status format changed.
5. WIFI_SCAN_0301: check command error and test environment not correct
6. WIFI_SCAN_0302, WIFI_SCAN_0303, WIFI_SCAN_0304: test environment not correct
2016-10-19 15:16:47 +08:00
3f521c4afa
update KnowIssues:
...
1. unsupported cases by "phy" command
2. DHCP server behavior different from ESP8266
3. TCP failed cases causes by unstable TCP behavior
4. UDP failed cases by poor UDP performance
2016-10-19 15:11:38 +08:00
b72d22041c
rename components/test to idf_test:
...
1. test will be categorized by test level
2. add test level as attribute to test cases
3. will select TestCaseScript by the test cases added to CI (currently no test case uses test case script)
4. adding test level to test jobs
5. update .gitlab-ci.yml, each job need to set its test case file path
6. update .gitlab-ci.yml, test case path for test report is changed to idf_test
2016-10-12 11:17:56 +08:00
michael