Sometimes when a test fails in a gitlab pipeline the reason of the
failure cannot be determined using only the logfile.
Add the modified rootfs as an artifact of the job to improve
troubleshooting. To accomplish this, always use -k option from the test
infra, unconditionally add the resulting images to the artifacts, and
let the runner do the cleanup for us.
These artifacts can also be useful when a test fails locally but pass at
gitlab runners.
When the test does not generate a image, this message is displayed in
the runner log:
WARNING: test-output/*/images/*: no matching files
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit improves our .gitlab-ci.yml logic to execute our runtime
tests located in support/testing/. To do so, this commit:
- Adds more Debian packages to be installed, namely the nose2 and
pexpect packages needed by the runtime testing infrastructure, as
well as the necessary Qemu emulators
- The description of how to run the runtime tests. Each test is
executed as a separate Gitlab CI job, so that the status of each
test is easily visible in the Gitlab CI web interface.
- The Makefile is improved to auto-generate .gitlab-ci.yml from
.gitlab-ci.yml.in, like we're doing for defconfigs. Since the
dependencies of .gitlab-ci.yml are no longer correct, we abandon
them and instead make it a PHONY target.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Arnout: simplify .gitlab-ci.in a little, removing redundant stuff;
make .gitlab-ci.yml a PHONY target]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It seems the ! is removed by the yaml parser, so it needs to be quoted.
While we're at it, add some explanatory comment.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check-DEVELOPERS test would always fail, because && will return
the exit code of the left-hand side if not 0, so it *always* returns
1.
But in fact, the test can be simplified quite a lot: we can run
getdeveloper directly, without arguments. That should just print
'No action specified', but it will also print any syntax errors it
encountered. So we can just 'grep -v' the expected output away; if
there is anything more, grep will return 0, but if there is nothing
more, it will return 1. So we just need to invert the result.
Incidentally, this also fixes the incorrect remaining reference to
support/scripts.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This verifies that the DEVELOPERS file doesn't contain errors. It
just runs parse_developers from getdeveloperlib and errors out if
that produces any output.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is needed for defconfigs that use syslinux.
Fixes https://gitlab.com/buildroot.org/buildroot/builds/11701092
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The buildroot repository is now mirrored on
https://gitlab.com/buildroot.org/buildroot so we can use Gitlab-CI to
test Buildroot. Gitlab-CI is controlled by a .gitlab-ci.yml file
that exists in the repository.
For now, the only test is building all defconfigs (inspired on
https://travis-ci.org/buildroot/buildroot-defconfig-testing/). Since
all the defconfigs have to be specified in the .gitlab-ci.yml file,
we generate the file based on .gitlab-ci.yml.in. The generated
.gitlab-ci.yml file has to be committed into the repository, though,
otherwise Gitlab-CI doesn't see it. So there is also a test to verify
that .gitlab-ci.yml is up-to-date.
Building all the defconfigs takes a long time. Gitlab-CI will do that
every time it pulls from git.buildroot.org, which is once per hour.
That is way too often. Therefore, the defconfigs are not built on pull,
but only on explicit trigger through the API or when a tag is added.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- fix typo not -> no
- add LC_ALL=C when calling 'ls -1' to get a predictable order of the
defconfigs
- regenerate .gitlab-ci.yml.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
