The gitlab-ci support in test-pkg allows to parallelize the test-pkg
work into several gitlab jobs. It's much faster than local serialized
testing.
To trigger this, a developer will have to add, in the latest commit of
their branch, a token on its own line, followed by a configuration
fragment, e.g.:
test-pkg config:
SOME_OPTION=y
# OTHER_OPTION is not set
SOME_VARIABLE="some value"
This configuration fragment is used as input to test-pkg.
To be able to generate one job per test to run, we need the list of
tests in the parent pipeline, and the individual .config files (one per
test) in the child pipeline. We use the newly-introduced --prepare-only
mode to test-pkg, and collect all the generated .config files as
artefacts; those are inherited in the child pipeline via the
"needs::pipeline" and "needs::job" directives. This is a bit tricky,
and is best described by the Gitlab-CI documentation [0].
We also list those .config files to generate the actual list of jobs to
run in the child pipeline.
Notes:
- if the user provides an empty fragment, this is considered an error:
indeed, without a fragment (and the package name), there is no way
to know what to test;
- if that fragment yields an empty list of tests, then there is
nothing to test either, so that is also considered an error.
[0] https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
- split the change to test-pkg to its own patch
- generate the actual yml snippet in support/scripts/generate-gitlab-ci-yml,
listing the .config files created by test-pkg
- some code-style-candies...
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In a followup commit, the make command used to log and display the last
lines on error will be used in another job.
Factorize it by introducing .run_make template.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Note that those tests were so far ignored only when requesting a single
defconfig build, or a single runtime test build; everything else
was trigerring thoses tests.
However, it feels more natural that they are also ignored when all
defconfigs build. or all runtime tests, are explictly requested.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Note that we do not propagate the existing comment, because it is
partially wrong; instead we just keep the per-condition comments.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Note that we do not propagate the existing comment, because it is
partially wrong; instead we just keep the per-condition comments.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, the image name and version are duplicated in the main
pipeline and the generated, child pipeline.
This is a condition for a future gaffe, so let's use the image from the
main pipeline when generating the child one.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Move the code to run check-flake8 into the Makefile, like we have for
check-package, so that it is easy to run locally (and not wait for
someone to report a failure from their Gitlab pipelines).
Compared to the existing check from gitlab-ci.yml, the Makefile check
differs in this respect:
- don't explicitly find *.py files: they are supposed to also be found
as a result of running 'file' on them;
- use git ls-tree instead of find: this is supopsedly faster as it
uses the index rather than readdir();
- don't output the count of warnings or errors: the output is a single
integer, which is confusing when there are errors, and even more so
when there are no, when it is simply '0';
- don't sort: the output is already stable and independent from the
locale;
- don't report the number of processed files: this information is
rather useless, and getting a hold of it would be more challenging
in this new code.
Note: ideally, we would want to use --null, --zero, or similar options,
with utilities that generates or parses a files listing. While git
ls-tree and xargs do support it, it becomes a little bit tricky to use
the --print0 option of file, and then grep in that output (it is not
undoable, but would requires replacing grep+cut with some sed trickery).
Since we do not expect our scripts names to contain funky chars (like
\n or a colon), we just hand-wave away that issue (and the old code was
doing the same assumption too).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The pkg-stats script now uses Python3 only constructs (the "async"
keyword) and therefore fails to pass the Python2 flake8 test.
Let's use the Python3 flake8 instead.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/681711009
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The defconfig check has been introduced by the previous
patch before the building each defconfig but those builds
are done every week or more.
Checking if a defconfig is valid can be done on every
push in the repository since it take few seconds.
This would allow to detect as soon as possible a problem
in a defconfig and eventually avoid breaking the build
while build testing all defconfig.
Introduce a new job template ".defconfig_check" in
gitlab-ci.yml.in and modify the generate-gitlab-ci-yml
to create a job for each defconfig to run the test.
Although, we could have used only one job to do all
tests, using one job per defconfig allow to identify
easily in gitlab which defconfig is falling.
Tested:
https://gitlab.com/kubu93/buildroot/pipelines/138331069https://gitlab.com/kubu93/buildroot/pipelines/171223758
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Use the script added by the previous patch to check
generated config files.
Tested on gitlab:
https://gitlab.com/kubu93/buildroot/pipelines/137597966
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since Gitlab 12.9, Gitlab allow to trigger child pipeline with generated configuration file.
See: https://gitlab.com/gitlab-org/gitlab/-/issues/35632
This allow us to stop updating the .gitlab-ci.yml file when a
new defconfig is added to Buildroot.
Remove check-gitlab-ci.yml job since it is now uneeded.
Remove .gitlab-ci.yml make target.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[ann.morin.1998@free.fr: manual: no longer needed to update at all]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
