d0a79c9cc9 | ||
---|---|---|
.. | ||
dependencies | ||
README.md | ||
assign-test.yml | ||
build.yml | ||
deploy.yml | ||
docs.yml | ||
host-test.yml | ||
pre_check.yml | ||
rules.yml | ||
static-code-analysis.yml | ||
target-test.yml |
README.md
IDF CI
- IDF CI
General Workflow
- Push to a remote branch
- Create an MR, choose related labels (not required)
- A
detached
pipeline will be created. - if you push a new commit, a new pipeline will be created automatically.
What if Expected Jobs ARE NOT Created?
-
check the file patterns
If you found a job that is not running as expected with some file changes, a git commit to improve the
pattern
will be appreciated. -
please add MR labels to run additional tests, currently we have to do this only for
target-test
jobs, please use it as few as possible. Our final goal is to remove all the labels and let the file changes decide everything!
MR labels for additional jobs
Supported MR Labels
build
build_docs
component_ut[_esp32/esp32s2/...]
custom_test[_esp32/esp32s2/...]
docker
docs
example_test[_esp32/esp32s2/...]
fuzzer_test
host_test
integration_test[_wifi/ble]
iperf_stress_test
macos
macos_test
nvs_coverage
submodule
unit_test[_esp32/esp32s2/...]
weekend_test
windows
There are two general labels (not recommended since these two labels will trigger a lot of jobs)
target_test
: includes all target forexample_test
,custom_test
,component_ut
,unit_test
,integration_test
all_test
: includes all test labels
How to trigger a detached
pipeline without pushing new commits?
Go to MR web page -> Pipelines
tab -> click Run pipeline
button.
In very rare case, this tab will not show up because no merge_request pipeline is created before. Please use web API then.
curl -X POST --header "PRIVATE-TOKEN: [YOUR PERSONAL ACCESS TOKEN]" [GITLAB_SERVER]/api/v4/projects/103/merge_requests/[MERGE_REQUEST_IID]/pipelines
How to Develop With rules.yml
?
General Concepts
-
pattern
: Defined in an array. A GitLab job will be created if the changed files in this MR matched one of the patterns. For example:.patterns-python-files: &patterns-python-files - "**/*.py"
-
label
: Defined in an if clause, similar as the previous bot command. A GitLab job will be created if the pipeline variables contains variables inBOT_LABEL_xxx
format (DEPRECATED) or included in the MR labels. For example:.if-label-build_docs: &if-label-build_docs if: '$BOT_LABEL_BUILD_DOCS || $CI_MERGE_REQUEST_LABELS =~ /^(?:[^,\n\r]+,)*build_docs(?:,[^,\n\r]+)*$/i'
-
rule
: A combination of various patterns, and labels. It will be used by GitLab YAMLextends
keyword to tell GitLab in what conditions will this job be created. For example:.rules:build:docs: rules: - <<: *if-protected - <<: *if-label-build_docs - <<: *if-label-docs - <<: *if-dev-push changes: *patterns-docs
An example for GitLab job on how to use extends:
check_docs_lang_sync: extends: - .pre_check_job_template - .rules:build:docs script: - cd docs - ./check_lang_folder_sync.sh
How to Add a New Job
?
check if there's a suitable .rules:<rules-you-need>
template
- if there is, put this in the job
extends
. All done, now you can close this window. (extends
could be array or string) - if there isn't
- check How to Add a New
Rules
Template?, create a suitable one - follow step 1
- check How to Add a New
How to Add a New Rules
Template?
check if this rule is related to labels
, patterns
- if it is, please refer to dependencies/README.md and add new rules by auto-generating
- if it isn't, please continue reading
check if there's a suitable .if-<if-anchor-you-need>
anchor
-
if there is, create a rule following
rules
Template Naming Rules.For detail information, please refer to GitLab Documentationrules-if
. Here's an example..rules:dev: rules: - <<: *if-trigger - <<: *if-dev-push
-
if there isn't
- check How to Add a New
if
Anchor?, create a suitable one - follow step 1
- check How to Add a New
How to Add a New if
Anchor?
Create an if
anchor following if
Anchors Naming Rules. For detailed information about how to write the condition clause, please refer to GitLab Documentation `only/except (advanced). Here's an example.
.if-schedule: &if-schedule:
if: '$CI_PIPELINE_SOURCE == "schedule"'
Naming Rules
Common Naming Rules
if a phrase has multi words, use _
to concatenate them.
e.g.
regular_test
if a name has multi phrases, use -
to concatenate them.
e.g.
regular_test-example_test
if
Anchors Naming Rules
-
if it's a label:
.if-label-<label_name>
-
if it's a ref:
.if-ref-<ref_name>
-
if it's a branch:
.if-branch-<branch_name>
-
if it's a tag:
.if-tag-<tag_name>
-
if it's multi-type combination:
.if-ref-<release_name>-branch-<branch_name>
Common Phrases/Abbreviations
-
no_label
$BOT_TRIGGER_WITH_LABEL == null
-
protected
($CI_COMMIT_REF_NAME == "master" || $CI_COMMIT_BRANCH =~ /^release\/v/ || $CI_COMMIT_TAG =~ /^v\d+\.\d+(\.\d+)?($|-)/)
-
target_test
a combination of
example_test
,custom_test
,unit_test
,component_ut
,integration_test
and all targets
-
rules
Template Naming Rules
- if it's tag related:
.rules:tag:<tag_1>-<tag_2>
- if it's label related:
.rules:labels:<label_1>-<label_2>
- if it's test related:
.rules:test:<test_type>
- if it's build related:
.rules:build:<build_type>
- if it's pattern related:
.rules:patterns:<patterns>
Reusable Shell Script tools/ci/utils.sh
It is used to put all the reusable shell scripts as small functions. If you want to set before_script: []
for you job, now you can set extends: .before_script_slim
instead. it will only run source tools/ci/utils.sh
If you're developing CI shell scripts, you can use these functions without source
them. They're already included in all before_script
To run these commands in shell script locally, place source tools/ci/utils.sh
at the very beginning.
Functions
CI Job Related
add_gitlab_ssh_keys
add_github_ssh_keys
add_doc_server_ssh_keys
fetch_submodules
get_all_submodules
Shell Script Related
error
: log in red colorwarning
: log in orange colorinfo
: log in green colorrun_cmd
: run the command with duration seconds inforetry_failed
: run the command with duration seconds info, retry when failed