Triggering a single defconfig or runtime test job can be handy:
- when adding or changing a defconfig;
- when adding or changing a runtime test case;
- when fixing some bug on a use case tested by a runtime test case.
Currently there are 3 subsets of jobs that can easily be triggered by
pushing a temporary branch with specific suffix:
- to trigger only the check-* jobs:
$ git push gitlab HEAD:<name> # currently 4 jobs
- to trigger all defconfigs and all check-* jobs:
$ git push gitlab HEAD:<name>-defconfigs # currently 197 jobs
- to trigger all runtime tests and all check-* jobs:
$ git push gitlab HEAD:<name>-runtime-tests # currently 118 jobs
When the user wants to trigger a single defconfig or runtime test job,
hand-editing the .gitlab-ci.yml and creating a temporary commit are
currently needed.
Add 2 more subsets that can be triggered based on the name of the
branch pushed.
- to trigger one defconfig job:
$ git push gitlab HEAD:<name>-<defconfig name> # currently 1 jobs
- to trigger one runtime job:
$ git push gitlab HEAD:<name>-<test case name> # currently 1 jobs
The check-* jobs are fast, so there is no need to add a per job trigger
for them. Also, they are anyway triggered with every push already.
While adding those new triggers, use the full name of the job as suffix.
This leads to large branch names:
$ git push gitlab HEAD:test1-tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc
$ git push gitlab HEAD:test2-olimex_a20_olinuxino_lime_legacy_defconfig
But those branches are temporary, and this way the user doesn't need to
think much, just copy and paste the job name as suffix.
The hidden keys that now hold the commonalities between jobs does not
hold only a script anymore, so rename then from *_script to *_base.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout: squash two patches]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In order to make the file easier to maintain, reorder the keys in a more
logical way:
Keep the docker image at the top.
Then all check-* jobs in the case-insensitive alphabetical order they
appear on Gitlab-CI pipeline results.
Then all keys related to defconfigs.
Finally all keys related to runtime tests.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We recommend wrapping at 80 columns but we accept 132 columns when it
makes more readable.
When running flake8 locally, use maximum line length 80.
But when running in GitLab CI, keep the check-flake8 job failing only
for lines longer than 132.
Reported-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Replace all YAML anchors with the new "extends" keyword because it is
more readable and more flexible (it works across configuration files
combined with the new "include" keyword).
Readability is more meaningful in .gitlab-ci.yml.in.
In the part of .gitlab-ci.yml that is auto-generated by 'make
.gitlab-ci.yml' keep the keyword in the same line of the job name.
So instead of this:
zynqmp_zcu106_defconfig:
extends: .defconfig
tests.boot.test_atf.TestATFAllwinner:
extends: .runtime_test
Use this:
zynqmp_zcu106_defconfig: { extends: .defconfig }
tests.boot.test_atf.TestATFAllwinner: { extends: .runtime_test }
Do this to to keep .gitlab-ci.yml easier to be post-processed by a
script.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add Buildroot's own .config file, as well as any package's .config file
(uclibc, linux, and busybox), for later inspection should a build fails,
notably due to changes in the kconfig-package infrastructure.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently the user has 2 sets of jobs that can be triggered on a GitLab
pipeline.
- to trigger all defconfigs, all runtime tests and all check-* jobs:
$ git tag <name>
$ git push gitlab <name> # currently 260 jobs
- to trigger only the check-* jobs:
$ git push gitlab HEAD:<name> # currently 4 jobs
This is not much versatile, so the user ends up hand-editing the
.gitlab-ci.yml in order to trigger some subsets, even the common ones,
for instance all runtime tests.
Add 2 more subsets that can be triggered based on the name of the
branch pushed.
- to trigger all defconfigs and all check-* jobs:
$ git push gitlab HEAD:<name>-defconfigs # currently 192 jobs
- to trigger all runtime tests and all check-* jobs:
$ git push gitlab HEAD:<name>-runtime-tests # currently 72 jobs
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When the runtime tests were first introduced, they still ran pretty
quickly. Nowadays, however, there are a lot of runtime tests, and some
of them take a really long time. So running them on every push is
really too much.
Just like we do for the defconfigs, run them on explicit trigger only.
The explicit trigger is now done every week, but it can be increased
to e.g. twice or three times per week.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The snippet of code that runs a check-package on all
.mk/.hash/Config.in files is currently only available within
.gitlab-ci.yml, and isn't immediately and easily usable by Buildroot
users. In order to simplify this, this commit introduces a top-level
"check-package" make target that implements the same logic. The
.gitlab-ci.yml file is changed to use "make check-package".
Since this target is oriented towards Buildroot developers, we
intentionally do not clutter the already noisy "make help" text with
this additional make target.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Now that all issues in Config.in files have been fixed, let's try to
make sure we don't introduce new ones by checking regularly these files
issues in Gitlab CI.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a test to check Python code style in the whole buildroot tree.
Search files by type in order to help flake8 to find the Python scripts
without .py extension. But don't rely only in the output of 'file' as it
uses heuristics and sometimes it is wrong (specially identifying Python
files as C++ source for the 'file' version currently in the Docker
image).
Include in the output:
- the list of Python files processed;
- statistics for each kind of warning;
- the total number of warnings;
- the number of Python files processed.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Bump the docker image to the latest tag to have flake8 pre-installed.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The defconfig is dumped at the very beginning of the build log, but the
resulting .config is still interesting to get, in case some settings did
not end up in there (e.g. because of the host gcc version, or because of
conflicting settings, or because the Config.in changed without the tests
being updated, or...).
So, add the .config to the list of artefacts to save.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that all issues in hash files have been fixed, let's try to make
sure we don't introduce new ones by checking regularly the hash files
issues in Gitlab CI.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Multiply the timeouts for emulator in the gitlab runners by 10 to avoid
sporadic failures in elastic runners.
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Sometimes when a test fails in a gitlab pipeline the reason of the
failure cannot be determined using only the logfile.
Add the modified rootfs as an artifact of the job to improve
troubleshooting. To accomplish this, always use -k option from the test
infra, unconditionally add the resulting images to the artifacts, and
let the runner do the cleanup for us.
These artifacts can also be useful when a test fails locally but pass at
gitlab runners.
When the test does not generate a image, this message is displayed in
the runner log:
WARNING: test-output/*/images/*: no matching files
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit improves our .gitlab-ci.yml logic to execute our runtime
tests located in support/testing/. To do so, this commit:
- Adds more Debian packages to be installed, namely the nose2 and
pexpect packages needed by the runtime testing infrastructure, as
well as the necessary Qemu emulators
- The description of how to run the runtime tests. Each test is
executed as a separate Gitlab CI job, so that the status of each
test is easily visible in the Gitlab CI web interface.
- The Makefile is improved to auto-generate .gitlab-ci.yml from
.gitlab-ci.yml.in, like we're doing for defconfigs. Since the
dependencies of .gitlab-ci.yml are no longer correct, we abandon
them and instead make it a PHONY target.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Arnout: simplify .gitlab-ci.in a little, removing redundant stuff;
make .gitlab-ci.yml a PHONY target]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It seems the ! is removed by the yaml parser, so it needs to be quoted.
While we're at it, add some explanatory comment.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check-DEVELOPERS test would always fail, because && will return
the exit code of the left-hand side if not 0, so it *always* returns
1.
But in fact, the test can be simplified quite a lot: we can run
getdeveloper directly, without arguments. That should just print
'No action specified', but it will also print any syntax errors it
encountered. So we can just 'grep -v' the expected output away; if
there is anything more, grep will return 0, but if there is nothing
more, it will return 1. So we just need to invert the result.
Incidentally, this also fixes the incorrect remaining reference to
support/scripts.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This verifies that the DEVELOPERS file doesn't contain errors. It
just runs parse_developers from getdeveloperlib and errors out if
that produces any output.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is needed for defconfigs that use syslinux.
Fixes https://gitlab.com/buildroot.org/buildroot/builds/11701092
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The buildroot repository is now mirrored on
https://gitlab.com/buildroot.org/buildroot so we can use Gitlab-CI to
test Buildroot. Gitlab-CI is controlled by a .gitlab-ci.yml file
that exists in the repository.
For now, the only test is building all defconfigs (inspired on
https://travis-ci.org/buildroot/buildroot-defconfig-testing/). Since
all the defconfigs have to be specified in the .gitlab-ci.yml file,
we generate the file based on .gitlab-ci.yml.in. The generated
.gitlab-ci.yml file has to be committed into the repository, though,
otherwise Gitlab-CI doesn't see it. So there is also a test to verify
that .gitlab-ci.yml is up-to-date.
Building all the defconfigs takes a long time. Gitlab-CI will do that
every time it pulls from git.buildroot.org, which is once per hour.
That is way too often. Therefore, the defconfigs are not built on pull,
but only on explicit trigger through the API or when a tag is added.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- fix typo not -> no
- add LC_ALL=C when calling 'ls -1' to get a predictable order of the
defconfigs
- regenerate .gitlab-ci.yml.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
