This script is intended to be used by gitlab CI to test at runtime Qemu
images generated by Buildroot's Qemu defconfigs.
This allows to troubleshoot different issues that may be associated with
defective builds by lanching a qemu machine, sending root password,
waiting for login shell and then perform a shutdown.
This script is inspired by toolchain builder [1] and the Buildroot
testing infrastructure.
The gitlab CI will call this script for each defconfig build but only
Qemu defconfig will be runtime tested, all others defconfig are ignored.
Some Qemu defconfig must be used with a specific Qemu version (fork)
that is not always available, so the script doesn't error out when it
can't spawn a missing command. That condition is anyway printed in the
log.
Finally, the script start Qemu like it's done for the Buildroot
testing infrastructure (using pexpect).
Note:
We noticed some timeout issues with pexpect when the Qemu machine is
powered off. That's because Qemu process doesn't stop even if the
system is halted (after "System halted"). So the script doesn't error
out when such timeout occure. The behaviour depends on the architecture
emulated by Qemu.
[1] https://github.com/bootlin/toolchains-builder/blob/master/build.sh
Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When the 'nvd-path', 'json' and 'html' are used like this:
--html ~/foo
then the tilde expansion is properly done by the shell. However, when
they are used like this:
--html=~/foo
The shell doesn't do the tilde expansion, and pkg-stats doesn't do
it. This commit modifies pkg-stats to ensure that tilde expansion is
done when parsing the 'nvd-path', 'json' and 'html' arguments.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[Thomas: improve commit log]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As of upstream commit 735eb608637e7bbab4082a541ac802cc919fec22,
available since version v20.1.1, support for Python 2.x has been
dropped, and Python >= 3.5 is required. So we make python-txaio depend
on python3, and remove the python2 test of python-txaio.
We also remove the python-six dependency which is no longer used by
txaio, since upstream commit 62b0e7eaa22769687df1de8f57374cb0a42bdc4d.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Upstream commit
c0223223f8,
which has been part of the releases since v20.1.2 has dropped support
for Python 2.x. So python-autobahn is now only available for Python
3.x.
We therefore remove the Python 2.x autobahn tests, and while at it,
drop the python-six dependency, which is no longer needed since
upstream commit
79bd2ba41b,
part of the releases since v20.1.3.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, the build continue even if some symbols disapear from
the generated dot config file (.config).
This patch add a new check in order to stop the test if one
of the provided symbol is missing. This must be treated as error.
For example, if a symbol disapear due to new dependency constraints.
Inspired by is_toolchain_usable() function from genrandconfig:
https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
TestPythonPackageBase.config provide already the basic uClibc toolchain.
So by adding the symbols for the linaro toolchain some warning are printed
at while loading the configuration:
.config:16:warning: override: reassigning to symbol BR2_TOOLCHAIN_EXTERNAL
.config:16:warning: override: BR2_TOOLCHAIN_EXTERNAL changes choice state
.config:17:warning: override: reassigning to symbol BR2_TARGET_ROOTFS_CPIO
.config:21:warning: override: BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM changes choice state
So, some symbol disapear from the generated dot config (.config) leading
to an error due to a new check in the testsuite infra.
Since this test should use the Linaro toolchain, remove
TestPythonPackageBase.config add BR2_arm=y and disable the rootfs tar option.
While at it, re-order the options so that they appear in the same
order as they appear in a defconfig.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Adam Duskett <Aduskett@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Until now, the lxc test was using the ARM CodeSourcery 2014.05 armv5 toolchain.
But the recent systemd version bump to 245 added a toolchain dependency
on systemd package due to build issues with gcc < 5.0.
Before [1] the lxc test was failing to build with the ARM CodeSourcery 2014.05
toolchain. After [1], the test is faling at runtime since the
"BR2_INIT_SYSTEMD=y" symbol disapear from the dot config (.config) due to
the new toolchain dependency.
Fix this by using the same toolchain as for the systemd tests [2]
[1] 2196ee25ff
[2] b3d979c0d1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The BR2_GENERATE_LOCALE can only be used by uClibc based toolchains with the
internal toolchain backend [1].
The test_tmux is using a external uClibc toolchain, so the
"BR2_GENERATE_LOCALE="en_US.UTF-8"" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.
[1] https://git.buildroot.net/buildroot/commit/?id=bd0ffe2206fbd32baf7f4a1dc5fde81cfad70462
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The BR2_TOOLCHAIN_EXTERNAL_LOCALE is only defined for uClibc based
custom external toolchains.
The test_glxinfo is using a glibc toolchain, so the
"# BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.
There is the same problem with:
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The BR2_TOOLCHAIN_EXTERNAL_LOCALE is only defined for uClibc based
custom external toolchains.
The test_syslinux is using a glibc toolchain, so the
"# BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.
There is the same problem with:
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit adds four new tests for the syslinux bootloader:
- Building on x86, for legacy BIOS
- Building on x86, for EFI BIOS
- Building on x86-64, for legacy BIOS
- Building on x86-64, for EFI BIOS
Runtime testing in Qemu would certainly be possible, but is left as a
future addition to these tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Update the toolchain being used by the testsuite infra.
The new toolchain 2018.11-1 is based on gcc 8.2, uClibc-ng 1.0.30,
linux-headers 4.14 and binutils 2.31.1.
Enable BR2_TOOLCHAIN_HAS_THREADS_DEBUG that is now required.
The old toolchain 2017.05 is based on gcc 4.9, uClibc-ng 1.0.25,
linux-headers 3.10 and binutils 2.27.
Tested with gitlab
https://gitlab.com/kubu93/buildroot/pipelines/132376578
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Following commit eee96b0f0a that adds a
gcc patch for OpenRISC, the OpenRISC pre-built toolchain was
rebuilt. Let's use this new toolchain version for the autobuilders.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
make-4.3 shipped with a backward incompatible change in how sharp signs
are handled in macros. Previously, up to make 4.2, the sharp sign would
always start a comment, unless backslash-escaped, even in a macro or a
fucntion call.
Now, the sharp sign is no longer starting a comment when it appears
inside such a macro or function call. This behaviour was supposed to be
in force since 3.81, but was not; 4.3 fixed the code to match the doc.
As such, use of external toolchains is broken, as we use the sharp sign
in the copy_toolchain_sysroot macro, in shell variable expansion to
strip off any leading /: ${target\#/}.
Fix that by applying the workaround suggested in the release annoucement
[0], by using a variable to hold a sharp sign.
[0] https://lists.gnu.org/archive/html/info-gnu/2020-01/msg00004.html
Signed-off-by: Yaroslav Syrytsia <me@ys.lc>
[yann.morin.1998@free.fr:
- move the SHARP_SIGN definition out of Makefile and into support/
- expand the commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The older version of flake8 on the autobuilders does not support typehints.
As such, flake8 throws the following error when scanning sample_gst1_python:
E999 SyntaxError: invalid syntax
Remove the typehinting from on_message to fix this issue.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The config is missing the following packages:
- BR2_PACKAGE_GST1_PLUGINS_BASE
This is needed for the videotestsrc plugin which provides
videotestsrc.
- BR2_PACKAGE_GST1_PLUGINS_BAD
This is needed for the debugutils plugin which provides
fakevideosink.
- BR2_PACKAGE_GST1_PLUGINS_BASE_PLUGIN_VIDEOTESTSRC
videotestsrcplugin used by the example pipeline.
- BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_DEBUGUTILS
fakevideosink plugin used by the example pipeline.
The sample was also amended to fix:
- no call to main.
- using autovideosink instead of fakevideosink. Using Fakevideoskink
is preferred because its primary purpose is for debugging and sample
pipelines. Autovideosink does not work because there is no video
output device.
- No function on_message. Without this function, the script fails
because of the missing function.
- The script sets the pipeline to Gst.State.EOS instead of
Gst.State.NULL which results in a failed pipeline state. When the
state is set to NULL, gstreamer automatically calls EOS. Manually
setting the pipeline state to EOS results in the following error:
Trying to dispose element sink, but it is in PLAYING instead of
the NULL state. You need to explicitly set elements to the NULL
state before dropping the final reference, to allow them to
clean up. A refcounting bug may also cause this problem in the
application or some element.
In addition, the default timeout is set too low and would result in a
test failure, this has been changed to 200 seconds.
These issues are now properly fixed, and the sample script passes.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[yann.morin.1998@free.fr:
- meld the two patches together
- add the print() in the on_demand callback
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This reverts commit 25033cfb86.
This perl package is deprecated and gives following recommendations while
building:
*** THIS IS NOT AN ERROR, JUST A MESSAGE FOR YOUR INFORMATION ***
Do you really need Crypt::SSLeay?
Starting with version 6.02 of LWP, https support was unbundled into
LWP::Protocol::https. This module specifies as one of its prerequisites
IO::Socket::SSL which is automatically used by LWP::UserAgent unless
this preference is overridden separately. IO::Socket::SSL is a more
complete implementation, and, crucially, it allows hostname
verification. Crypt::SSLeay does not support this. At this point,
Crypt::SSLeay is maintained to support existing software that already
depends on it.
However, it is possible that your software does not really depend on
Crypt::SSLeay, only on the ability of LWP::UserAgent class to
communicate with sites over SSL/TLS.
If are using version LWP 6.02 or later, and therefore have installed
LWP::Protocol::https and its dependencies, and do not explicitly use
Net::SSL before loading LWP::UserAgent, or override the default socket
class, you are probably using IO::Socket::SSL and do not really need
Crypt::SSLeay.
Before installing Crypt::SSLeay, you may want to try specifying a
dependency on LWP::Protocol::https.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
support/testing/tests/package/test_gst1_python.py:29:1: W391 blank line at end of file
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
support/testing/tests/package/sample_python_gobject.py:5:1: E302 expected 2 blank lines, found 1
support/testing/tests/package/sample_python_gobject.py:8:7: E111 indentation is not a multiple of four
support/testing/tests/package/sample_python_gobject.py:11:1: E305 expected 2 blank lines after class or function definition, found 1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes the following flake8 warnings:
support/testing/tests/package/sample_gst1_python.py:5:1: F401 'time' imported but unused
support/testing/tests/package/sample_gst1_python.py:7:1: E402 module level import not at top of file
support/testing/tests/package/sample_gst1_python.py:21:12: W292 no newline at end of file
For the E402 warning, we add a "noqa" marker, as we really want the
gi.require_version() to be before.
There is still one remaining warning to be fixed:
support/testing/tests/package/sample_gst1_python.py:18:28: F821 undefined name 'on_message'
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
flake8 complains with:
support/scripts/pkg-stats:339:13: E722 do not use bare 'except'
Due to the construct:
try:
something
except:
print("some message")
raise
Which is in fact OK because the exception is re-raised. This issue is
discussed at https://github.com/PyCQA/pycodestyle/issues/703, and the
general agreement is that these "bare except" are OK, and should be
ignored from flake8 using a noqa statement.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
flake8 complains with:
pkg-stats:38:1: E402 module level import not at top of file
This is due to sys.path.append() being before the import from
getdeveloperlib, but we really need this sys.path.append() to be
before, so let's ignore this flake8 warning.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, we build a complete AArch64 system, including a kernel. This
can take quite some time.
Switch to an armv7 system, which allows us to use one a prebuilt kernel,
thus significantly reducing the test time.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[yann.morin.1998@free.fr:
- use a standalone config; don't inherit from TestPythonPackageBase
- use the default external toolchain (ARM, not Linaro)
- rewrite commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This test case runs a simple pipeline for 100 frames to ensure that
gst1-python works properly.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This commit updates all our toolchain configuration fragments for
pre-built Buildroot toolchains to use toolchains built with Buildroot
2020.02.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The C program inside check-kernel-headers.sh has two checking mode: a
strict and a loose one.
In strict mode, we want the kernel headers version declared by the
user to match exactly the one of the toolchain.
In loose mode, we want the kernel headers version of the toolchain to
be greater than or equal to the one declared by the user: this is used
when we have a toolchain that has newer headers than the latest
version known by Buildroot.
However, in loose mode, we continue to show the "Incorrect kernel
headers version" message, even though we then return a zero error
code. This is very confusing: you see an error displayed on the
terminal, but the build goes on.
We fix that by first doing the loose check first, and returning 0 if
it succeeds. And then we move on with the strict check where we want
the version to be identical.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This file was created by utils/scancpan while adding other packages but
apparently not yet added in the repo.
Assign this test case to Bernd in the DEVELOPERS file since he is
listed as the maintainer for this package.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Added via utils/scancpan, adding a host dependency to perl-try-tiny and
target dependency to openssl.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Dependency for perl-crypt-ssleay (added in subsequent commit).
Added via utils/scancpan, without changes.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Other changes:
- Convert the package to a meson package.
- Remove 0001-add-PYTHON_INCLUDES-override.patch as it no longer applies.
- Add gobject-introspection as a dependency.
- Add the package under myself in the DEVELOPERS file.
Because gobject-introspection is now a dependency of python-gobject, the test
must be updated at the same time.
- Change TestPythonPy2Gobject to TestPythonPy3Gobject as
gobject-introspection requires python3.
- Refactor test_python_gobject.py to no longer inherit the
TestPythonPackageBase class, as this class uses a base config that does not
support gobject-introspection.
- Update sample_python_gobject to use Glib to find the path of sh.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Two simple tests to ensure that openrc boots without any services crashing
with a read only and a read write filing system.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Carlos Santos <unixmania@gmail.com>
[yann.morin.1998@free.fr: really check the init process]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
If there is no infra set or infra is virtual the status is set to 'na'.
This is done for the follwing checks:
- license
- license-files
- hash
- hash-license
- patches
- version
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This value can be used for later processing.
In the buildroot-stats application this is used to create links pointing
to the git repo of buildroot.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Unify the status check information. The status is stored in a tuple. The
first entry is the status that can be 'ok', 'warning' or 'error'. The
second entry is a verbose message.
The following checks are performed:
- url: status of the URL check
- license: status of the license presence check
- license-files: status of the license file check
- hash: status of the hash file presence check
- patches: status of the patches count check
- pkg-check: status of the check-package script result
- developers: status if a package has developers in the DEVELOPERS file
- version: status of the version check
With that status information the following variables are replaced:
has_license, has_license_files, has_hash, url_status
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
