Commit Graph

469 Commits

Author SHA1 Message Date
Sergey Matyukevich
6d7f5c09f7 support/testing: switch TestATFAllwinner to mainline TF-A
Switch to mainline TF-A that provides basic support for H5 and A64.

Note that Allwinner platform layer in TF-A does not provide support
for GCC stack protection, so make sure to disable this TF-A feature.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 17:10:42 +02:00
José Luis Salvador Rufo
78f477d136 package/zfs: bump version to 2.0.5
As this version brings support for kernel up to 5.12, we update the
test cases to use the 5.12 kernel.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-06-26 15:13:03 +02:00
Thomas Petazzoni
81d1c6cf28 support/testing/tests/package/test_bmap_tools: add test for host bmap-tools
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 14:24:34 +02:00
Nicolas Carrier
1023f742b8 support/testing/tests/package/test_bmap_tools: new test
This patch implements a simple test in which a dummy file system image
is created, then `bmaptool create` and `bmaptool copy` are used to copy
it to another file.

Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: several reworks, add myself to DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 14:24:22 +02:00
Thomas Petazzoni
3cf2782906 support/testing/infra/emulator.py: update pre-built kernels
The pre-built vexpress kernel used by the testing infrastructure is a
4.0.0 kernel, which is getting old to be used with reasonably recent
toolchains.

This commit updates the pre-built kernels for both the versatile and
vexpress machines to 5.10.7 (they have already been put online).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 13:59:58 +02:00
Thomas Petazzoni
6a92726981 support/testing/tests/package/test_python_augeas: new test
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-24 23:42:04 +02:00
Francois Perrad
b4fd87f993 package/lua-zlib: new package
This module is only partially compatible with lzlib (which is no longer
maintained).

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: amend commit log about limited compatibility]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-14 22:31:32 +02:00
Peter Korsgaard
8d07baab43 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-06-07 17:14:37 +02:00
Marcin Niestroj
f40219ff69 support/testing: add python-dbus-next test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 16:41:10 +02:00
Romain Naour
45a13be315 support/testing: test_hardening disable PIC/PIE
Since [1], PIC/PIE is enabled by default but the TestRelroPartial
test expect implicitely PIC/PIE being disabled.

Disable PIC/PIE from the config fragment provided by
TestRelroPartial.

[1] 810ba387be

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661757

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-05-18 22:15:36 +02:00
Romain Naour
4a3639bad0 support/testing: test_glxinfo load X11 modules in the right order
From [1]
"Xorg does not implement real dynamic linking and requires that its
modules get loaded in the right order."

From /var/log/Xorg.0.0.log:
 (II) LoadModule: "modesetting"
 (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
 (EE) Failed to load /usr/lib/xorg/modules/drivers/modesetting_drv.so: /usr/lib/xorg/modules/drivers/modesetting_drv.so: undefined symbol: shadowRemove

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661899

[1] https://forums.gentoo.org/viewtopic-p-8245578.html#8245578

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:49 +02:00
Romain Naour
95b722eaa8 support/testing: test_glxinfo: switch to Gallium swrast
Since the mesa3d bump to version 21.0.3 [1], the
BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST option is not supported anymore
since the mesa DRI swrast driver has been removed upstream

So, switch to Gallium swrast.

[1]15a2f9b819806d38a7d8172a20f80130b1d60e63

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:48 +02:00
Dick Olsson
6131efc62c support/testing: new boot test for EDK2
Boot a QEMU sbsa-ref machine with ATF, EDK2, GRUB2 and a minimal
kernel. This is a simple but effective test of a compliant setup.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-17 17:26:37 +02:00
Peter Korsgaard
aa31d10808 test_docker_compose.py: Test the volume mount feature
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 23:00:16 +02:00
Peter Korsgaard
4915b692c8 test_docker_compose.py: Test the port publish feature
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 22:59:58 +02:00
Romain Naour
f5ea09e2a8 support/testing: remove TestPythonPy2Colorzero
The python2 support has been removed since the python-colorzero bump version to 2.0.

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:33 +02:00
Romain Naour
9217708122 support/testing: remove TestPythonPy2Gpiozero
The python2 support has been removed since the python-colorzero bump version to 2.0.

Remove the gpiozero test with python2

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:24 +02:00
Arnout Vandecappelle (Essensium/Mind)
74bae64dc5 support/testing: add sudo package test
Create a new user 'sudotest' to validate that sudo really works (i.e.
properly has setuid).

Creating the user and adding it to sudoers is done at runtime, otherwise
we'd need to add extra files to the config which complicates things a
little bit.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-10 22:08:15 +02:00
Adam Duskett
ab2d472cde testing/tests/package/test_openjdk.py: bump kernel version to 5.10.34
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.

Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-05 21:37:21 +02:00
Dick Olsson
c5497df7b3 support/testing: add s6-networking tests
Test that the TAICLOCK and TCP servers are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:58:51 +02:00
Dick Olsson
50c6e932dc support/testing: add s6-rc tests
Test that s6-rc service database compilation is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:56:49 +02:00
Dick Olsson
77c13ae989 support/testing: add s6-portable-utils tests
Test that a few basis utilities are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:53:52 +02:00
Dick Olsson
985d733f31 support/testing: add s6 tests
Test that directory scanning and supervision is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: properly indent, and use textwrap to dedent again.]
2021-05-04 21:50:40 +02:00
Dick Olsson
f7ea0af883 support/testing: add execline tests
Test that the interpreter can run a basic command.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:34:36 +02:00
Francois Perrad
b3dd1034d5 package/perl-libwww-perl: bump to version 6.53
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:44 +02:00
Francois Perrad
9962e3020c package/perl-io-socket-ssl: bump to version 2.070
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:43 +02:00
José Luis Salvador Rufo
4470bc9914 package/zfs: new package
OpenZFS is an advanced file system and volume manager which was originally
developed for Solaris and is now maintained by the OpenZFS community. This
repository contains the code for running OpenZFS on Linux and FreeBSD.

http://zfsonlinux.org/

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
[me:
  - fix test case on how to use a pre-built toolchain
  - reorder the test case config
  - add test case with glibc
  - drop superflous test timeout override
  - only select libtirpc when C library lacks native RPC
  - drop unused ZFS_MODULES variable
  - drop ZFS_CPE_ID_PREFIX and ZFS_AUTORECONF_OPTS which are defaults
  - drop NLS options, already set in a generic manner
  - drop incomplete/improper sysvinit support
  - some cosmetics
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-13 23:08:17 +02:00
Yann E. MORIN
b9db38d370 package: drop _CPE_ID_VALID, use _CPE_ID_VENDOR
FOO_CPE_ID_VALID really ought to be an internal implementaion detail.
Packages that really want to trigger their CPE defintitions really
should set one of the actual variables to a meaningful value.

There are two CPE-related variables that we could chose to set to
replace FOO_CPE_ID_VALID: FOO_CPE_ID_VENDOR and FOO_CPE_ID_PRODUCT.
Between those two, _VENDOR more often diverges from the default than
_PRODUCT does, so that's what we use.

    ---8<------8<------8<------8<------8<---
    #!/bin/bash
    # Replace FOO_CPE_ID_VALID = YES with FOO_CPE_ID_VENDOR = foo_project
    for i in $(git grep -l -E '[^)]_CPE_ID_VALID = YES' package support); do
        pkg="$(basename "${i%/*}")"
        sed -r -i -e "s/_CPE_ID_VALID = YES/_CPE_ID_VENDOR = ${pkg}_project/" "${i}"
    done
    ---8<------8<------8<------8<------8<---

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: update cpe-test comment to reflect pkg3 change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-03-06 17:18:07 +01:00
Peter Korsgaard
a0aff89ed2 support/scripts/gen-bootlin-toolchains: correct xtensa-lx60 toolchain dependencies
Fixes:
http://autobuild.buildroot.net/results/011/0111c2ed54618daaeedfc66b0ea04eda00a7e855/
http://autobuild.buildroot.net/results/e53/e53e3880b63a23fa3b3e6d34664d40d5ddbdff89/
..

As listed in the br_fragment file of the toolchain, this is built for a
little-endian "custom" xtensa variant rather than the (big-endian) fsf one:

BR2_xtensa=y
BR2_XTENSA_CUSTOM=y

So update the dependencies in the script and regenerate Config.in.options /
toolchain test.  Also fixup the autobuild config snippet to match.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-03-02 23:52:49 +01:00
Sergey Matyukevich
653b127d1e support/testing: TestATFAllwinner needs python3 for u-boot
New U-Boot version needs Python 3.x for pylibfdt.

Fixes:
- https://gitlab.com/buildroot.org/buildroot/-/jobs/1019385909

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-13 22:46:27 +01:00
Fabrice Fontaine
f7922a57f5 Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE
Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE to better
"comply" with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 23:21:38 +01:00
Fabrice Fontaine
4b6202f721 Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT
Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT to better "comply"
with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-21 22:43:24 +01:00
Sergey Matyukevich
5d54214049 support/testing: fix TestATFAllwinner test
Fix U-Boot config snippet in TestATFAllwinner. Bump U-Boot
version to fix DTC build on hosts with gcc 10 and add
pylibfdt dependency.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-16 22:35:20 +01:00
Sergey Matyukevich
22188ff6e0 support/testing: remove TestATFMarvell
There were only two users of Marvell ATF: SolidRun MacchiatoBin board
and ClearFrog GT 8k board. After mv-ddr-marvell package update both
boards switched to upstream ATF. Remove tests for now unused
Marvell ATF.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-16 22:35:18 +01:00
Yann E. MORIN
5b95a5dc27 support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.

However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.

To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.

So, we bump the git-specific format-version to -br1.

The %ci date  has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.

As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.

Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.

For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>

    ---8<------8<------8<------8<---
    #!/bin/sh
    # Find and download all packages using git as backend.
    # Manually fix hashes for affected packages.

    # Packages that only have a host variant
    HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'

    # Packages that have a non-git main _SOURCE, and/or which
    # have BR_NO_CHECK_HASH_FOR for the git _SOURCE
    NOT_GIT='aufs|aufs-util|xenomai|linux-headers'

    export BR2_DL_DIR=$(pwd)/temp-dl-dir

    make defconfig
    make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
                boot/vexpress-firmware/ package/ \
            |sed -r -e 's,.*/([^/]+)\.mk,\1,' \
            |sed -r -e '/^('"${NOT_GIT}"')$/d;' \
                    -e 's/^('"${HOST_ONLY}"')/host-\1/;' \
                    -e 's/$/-legal-info/;'
          )

    ---8<------8<------8<------8<---
2021-01-10 22:06:58 +01:00
Yann E. MORIN
728484538a support/testing: fix flake8 issue in CPE ID test
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-01-02 18:06:24 +01:00
Francois Perrad
2949f423a4 package/lualdap: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-31 15:13:06 +01:00
Marcin Niestroj
55a6ff34ba support/testing: add pytest-asyncio test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-31 15:10:16 +01:00
Thomas Petazzoni
3950c53cd0 support/testing/tests/core/test_cpeid: new test
This commit adds a number of test cases to verify that the CPE_ID_*
variables are properly handled by the generic package infrastructure
and that the "make show-info" JSON output matches what we expect.

A total of 5 different example packages are used to exercise different
scenarios of CPE_ID_* variables usage.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-11-26 16:36:56 +01:00
Romain Naour
1989a49feb support/testing/tests/init/test_systemd: update to BR2_PACKAGE_SYSTEMD_JOURNAL_REMOTE
The commit [1] moved systemd-journal-gatewayd into systemd-journal-remote
option. Update to BR2_PACKAGE_SYSTEMD_JOURNAL_REMOTE in the testsuite
when BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY is used.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981805
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981811

[1] e46fe9a6f2

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 11:49:59 +01:00
Romain Naour
f52a9835fa support/testing: update to the new text representation of capabilities
Since libcap 2.41, the text representation of capabilities now use
the '=' instead of '+' [1].

This break our capabilities tests since we still use the old
representation.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/morgan/libcap.git/commit/?id=177cd418031b1acfcf73fe3b1af9f3279828681c

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981737

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:12:48 +01:00
Romain Naour
7b9762f4ab support/testing/test_hardening: add missing Kconfig symbol
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is needed to use the
custom external toolchain x86-i686--glibc--bleeding-edge-2018.11-1.tar.bz2

Otherwise the symbol BR2_TOOLCHAIN_EXTERNAL_URL is lost.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981738
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981739
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981740
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981741
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981742
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981743

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:11:12 +01:00
Romain Naour
ddf40cffea support/testing/test_syslinux: add missing Kconfig symbol
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is needed to use the
custom external toolchain x86-i686--glibc--bleeding-edge-2018.11-1.tar.bz2

Otherwise the symbol BR2_TOOLCHAIN_EXTERNAL_URL is lost.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981734
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981733

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 09:06:03 +01:00
Romain Naour
14ed65e3a6 support/testing: TestInitSystemSystemdRwIfupdown test expect a RW rootfs
When running the TestInitSystemSystemdRwIfupdown test, the rootfs must
be in read-write mode. The commit log [1] introducing systemd tests say
so:

"basic systemd, read-write, network w/ ifupdown"

With systemd 246.5, the service systemd-update-done return an error code
when it can't write on the filesystem (/etc)

[1] 117835d5fc
[2] 8019995e9a

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981813

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-09 21:10:35 +01:00
Thomas Petazzoni
0a5c9f824a support/testing/tests: fix flake8 warnings
Fixes the following flake8 warnings:

support/testing/tests/core/test_selinux.py:21:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:38:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:51:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:62:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:65:14: E127 continuation line over-indented for visual indent
support/testing/tests/init/test_systemd_selinux.py:53:1: E302 expected 2 blank lines, found 1
support/testing/tests/init/test_systemd_selinux.py:64:1: E302 expected 2 blank lines, found 1

Interestingly, the "continuation line over-indented for visual indent"
shows up only once, while the same pattern is there at multiple places
in the file. We fix all places with that over-indentation pattern.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-06 08:11:25 +01:00
Thomas Petazzoni
03f556d7d2 support/testing/tests/toolchain/test_external_bootlin: regenerate with new PowerPC toolchains
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-15 00:00:57 +02:00
Antoine Tenart
751b660c1f support/testing/tests/core/test_selinux: new tests for the packages SELinux functionalities
Add tests to ensure the packages SELinux functionalities (being able to
select an extra SELinux module in the refpolicy, and being able to
provide a custom SELinux module) are working as expected.

We use a BR2_EXTERNAL folder, provided in the tests, to use a custom
SELinux enabled package.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
0f3b931624 support/testing/tests/core/test_selinux: new test for BR2_PACKAGE_REFPOLICY_CUSTOM_GIT
Add a test for BR2_PACKAGE_REFPOLICY_CUSTOM_GIT (which allows to select
a custom location for the SELinux refpolicy). The test uses the official
refpolicy as a test (we only want to test the functionality is working,
not that another refpolicy is correctly building; that is an user
problematic).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
4fe09f4cab support/testing/tests/core/test_selinux: new test for BR2_REFPOLICY_EXTRA_MODULES_DIRS
Add a test for the BR2_REFPOLICY_EXTRA_MODULES_DIRS functionality (which
allows to provide custom SELinux modules).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
e383f2835f support/testing/tests/core/test_selinux.py: new test for BR2_REFPOLICY_EXTRA_MODULES
This patch adds a test for the BR2_REFPOLICY_EXTRA_MODULES
functionality (which allows to select extra modules within the SELinux
refpolicy using Kconfig).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
31901a5afd support/testing/tests/init/test_systemd_selinux: new SELinuxSystemdSquashfs test
Add a test called 'SELinuxSystemdSquashfs' which will perform the same
tests as the Ext4 version, but using a Squashfs filesystem. Thanks to
this, we'll have a test on a real only filesystem.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
d466b9c0ff support/testing/tests/init/test_systemd_selinux: new SELinuxSystemdExt4 test
This adds a test called 'SELinuxSystemdExt4'. This test will build an
SELinux enabled image with systemd, boot it, and perform a few runtime
tests to check SELinux related capabilities.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Marcin Niestroj
fd89fcf944 support/testing: add pytest test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-09-06 14:46:41 +02:00
Peter Korsgaard
c287d789b7 Merge branch 'next'
A number of merge conflicts, but hopefully they are all sorted out now.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-09-02 18:14:46 +02:00
Yann E. MORIN
400ce4f905 support/runttime-tests: fix openssh test
When it was applied, commit 243d500f8d (support/testing: add openssh
runtime test) was amended to not provide a NIC to the emulated machine,
as the test did not require access to the outer world: it only uses the
lo interface. Also, there was a discrepancy between the NIC name in the
Buildroot configuration, and the drivers available in our default kernel
image, making the boot hang for a while whaiting for a NIC that would
never come.

However, that tweak was tested locally with a qmeu version more recent
than the one available in our buidroot/base Docker image. As a
consequence, that test fails to run in gitlab-ci.

Revert to using the old way of specifying no network: it works on
gitlab-ci, and qemu versions in standard distros still support it.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2020-09-01 21:35:19 +02:00
Thomas Petazzoni
efce79db65 support/testing/test/toolchain/test_external_bootlin: new test cases
This commit adds the new test cases generated automatically by the
bl-toolchains-gen script, to test the integration of the Bootlin
toolchains.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-08-29 15:47:46 +02:00
Yann E. MORIN
071dbfeb4d support/tests: add runtime test for python-rpi-gpio
Modeled after similar python packages.

However, this one is picky, and throws an exception when it
detects that it is not running on a Raspberry Pi. So we just
catch that exception and check this is what we expect.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Christian Stewart <christian@paral.in>
Cc: Michael Fischer <mf@go-sys.de>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Ian Haylock <haylocki@yahoo.co.uk>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-23 23:29:55 +02:00
Romain Naour
243d500f8d support/testing: add openssh runtime test
This new runtime test is based on test_dropbear.py. The only required change
is to use "-oStrictHostKeyChecking=no" instead of "-y" to accept the new key.

Since the base test infra only provide a uClibc-ng toolchain, add a second
test using a glibc based internal toolchain.

For example, this allow to trigger the openssh 8.1p bug with glibc 2.31 [1].

[1] https://bugs.archlinux.org/task/65386

Signed-off-by: Romain Naour <romain.naour@smile.fr>
yann.morin.1998@free.fr:
  - deduplicate the whole test
  - don't provide any NIC, we only need and use lo
  - simplify post-build script (append with cat, don't munge with sed)
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-08-19 22:13:43 +02:00
Thomas Petazzoni
679a1ba24b support/testing/tests/toolchain/test_external: support non-ELF toolchains
The TestExternalToolchain() base class implement a test checking if
the ELF interpreter that is advertised by Busybox really exists in the
rootfs. Of course, this only makes sense with ELF toolchains. Until
now, only ELF toolchains were tested, but we are going to use
TestExternalToolchain() with non-ELF toolchains as well, so let's make
this conditional.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: strip() lines during readlines()]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-08-16 23:43:48 +02:00
Thomas Petazzoni
cc061128de support/testing/tests/core/test_timezone.py: fix indentation
Fixes:

support/testing/tests/core/test_timezone.py:7:9: E117 over-indented

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-14 21:56:08 +02:00
Thomas Petazzoni
b0078c058a support/testing: consistently use raw strings for re.compile
Otherwise Python 3.x flake8 complains with:

W605 invalid escape sequence '\s'

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-14 21:56:02 +02:00
Thomas Petazzoni
9e3397cfab support/testing/tests/boot/test_atf: DTC needed for ATF Vexpress test
Since the bump of ATF to 2.2 for the ATF Vexpress test case in commit
fc3d6a3ed0
("support/testing/tests/boot/test_atf: update U-Boot/ATF use in
TestATFVexpress"), DTC is now needed otherwise the build fails with:

make[2]: dtc: Command not found
Makefile:873: recipe for target 'build/juno/release/fdts/juno_tb_fw_config.dtb' failed

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/674934470

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-09 22:14:24 +02:00
Thomas Petazzoni
fc3d6a3ed0 support/testing/tests/boot/test_atf: update U-Boot/ATF use in TestATFVexpress
This simply updates to more modern versions of ATF and U-Boot.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-06 00:05:14 +02:00
Peter Korsgaard
436daadc42 support/testing: drop explicit docker kernel options from docker test
Since commit 0390777bfa (package/docker-engine: needs some kernel
options), docker-engine now automatically ensures the needed kernel options
are enabled, so drop the explicit options from the kernel config.

23:19:27 TestDockerCompose                        Starting
23:19:28 TestDockerCompose                        Building
00:14:41 TestDockerCompose                        Building done
00:15:30 TestDockerCompose                        Cleaning up
.
----------------------------------------------------------------------
Ran 1 test in 3362.784s

OK

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-07-31 09:13:25 +02:00
Peter Korsgaard
4726cf9517 support/testing: drop explicit CGROUPFS_MOUNT from docker test
Since commit 4f8229653 (package/docker-engine: needs more runtime
dependencies), docker-engine now automatically pulls in cgroupfs-mount, so
drop the explicit handling of it in TestDockerCompose.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-07-31 09:13:21 +02:00
Eric Le Bihan
d7c944b2ae package/cargo: drop package
As no package depends on the standalone cargo package, it can be
dropped.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: David Pierret <david.pierret@smile.fr>
Tested-by: David Pierret <david.pierret@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-07-27 13:31:00 +02:00
Peter Seiderer
0d0af210f7 support/testing: add python-gpiozero test
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-06-18 21:45:00 +02:00
Peter Seiderer
dc5761beb0 support/testing: add python-colorzero test
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-06-18 21:45:00 +02:00
Adam Duskett
e9a02417f3 package/openjdk-bin: install to host/usr/lib/jvm
Buildroot currently installs openjdk-bin to $(HOST_DIR)/ instead of the more
traditional (for java installations) $(HOST_DIR)/usr/lib/jvm.

As described in https://bugs.busybox.net/show_bug.cgi?id=13001

"Openjdk-bin provides it's own libfreetype.so and places it into
$(HOST_DIR)/lib/. This library causes build failures with the
host-xapp_mkfontscale package due to the overwritten libfreetype.so.

mkfontscale.o: In function `doDirectory':
mkfontscale.c:(.text+0x1a80): undefined reference to `FT_Get_BDF_Property'
collect2: error: ld returned 1 exit status

Reproducing the error is done by repeating the following steps.
make host-freetype
make host-openjdk-bin
make host-xapp_mkfontscale"

There are two options for fixing this problem:

 1) add host-freetype and host-lksctp-tools as dependencies to host-openjdk-bin
    and then remove the provided libfreetype.so and libsctp.so libraries
    in a post_extract_hook.

 2) change the installation directory from $(HOST_DIR)/ to
    $(HOST_DIR)/usr/lib/jvm just like the target OpenJDK package and
    copy the entire source directories contents to the above location.

The second option provides the following advantages:
  - the directory structure is consistent with how we handle the target OpenJDK.

  - the HOST_OPENJDK_BIN_INSTALL_CMDS step is simplified.

  - packages such as Maven require directories of which we are currently not
    copying. These missing directories cause programs such as Maven to crash
    when running with an error such as
    "Can't read cryptographic policy directory: unlimited."

  - does not miss any other libraries that solution 1 would not cope with
    (e.g. libzip.so from host-libzip, or libnet.so from not-yet existing
    host-libnet, or libsctp.so from not-yet existing host-lksctp-tools)

Because the second option is both simple, easier to implement, is low-impact,
and fixes the problems described above wholly, it is the best to implement.

To implement the above changes, we must also modify the following files in the
same patch to match the host's new directory paths:

 - openjdk.mk
 - openjdk-jni-test.mk
 - openjdk-hello-world.mk

To avoid having to change all those packages in the future, expose two
new variables, HOST_OPENJDK_BIN_ROOT_DIR which contains the path where
the openjdk-bin was installed in, and JAVAC, which contains the path to
the javac compiler (modeled after the way the autoconf et al. variables
are set and exposed).

Tested with:
./support/testing/run-tests -o out -d dl tests.package.test_openjdk.TestOpenJdk

Fixes: https://bugs.busybox.net/show_bug.cgi?id=13001

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[yann.morin.1998@free.fr:
  - introduce HOST_OPENJDK_BIN_ROOT_DIR and JAVAC
  - expand and tweak the commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-06-18 11:01:41 +02:00
Titouan Christophe
cc267d7ab4 support/testing: add runtime test for Redis
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-06-06 00:17:40 +02:00
Francois Perrad
a3b3571fab support/testing: test_lua_utf8: fix test after bump to lua-utf8 0.1.2
The commit 0f0649140f introduced a change
in the module name: utf8 -> lua-utf8.

The packages-file-list.txt show the change in the files intalled on the rootfs:

[lua-utf8 0.1.2-2 latest version]
lua-utf8,./usr/lib/luarocks/rocks-5.1/luautf8/0.1.2-2/doc/LICENSE
lua-utf8,./usr/lib/luarocks/rocks-5.1/luautf8/0.1.2-2/doc/README.md
lua-utf8,./usr/lib/luarocks/rocks-5.1/luautf8/0.1.2-2/luautf8-0.1.2-2.rockspec
lua-utf8,./usr/lib/luarocks/rocks-5.1/luautf8/0.1.2-2/rock_manifest
lua-utf8,./usr/lib/lua/5.1/lua-utf8.so

[lua-utf8 1.2-0 previous version]
lua-utf8,./usr/lib/luarocks/rocks-5.1/utf8/1.2-0/doc/README.md
lua-utf8,./usr/lib/luarocks/rocks-5.1/utf8/1.2-0/utf8-1.2-0.rockspec
lua-utf8,./usr/lib/luarocks/rocks-5.1/utf8/1.2-0/rock_manifest
lua-utf8,./usr/lib/lua/5.1/utf8.so

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/526036036

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-27 22:03:21 +02:00
Romain Naour
3e6b97e09e support/testing: test_python_django: use the timeout_multiplier value
As reported by our gitlab runtime test [1] and on the mailing list
[2], the test_python_django is failing due to django server taking a
lot of time to start. Since the django server is started in background
through pexpect, we can't easily wait for the last startup line:

"January 01, 1970 - 00:00:41

Django version 3.0.4, using settings 'testsite.settings'

Starting development server at http://0.0.0.0:1234/

Quit the server with CONTROL-C."

In the failing gitlab job, we don't see such lines.

If we increase a lot the timout, the test passes.

Use timeout_multiplier introduced by [3] in order to adjust the
timeout.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/488816222

Tested:
https://gitlab.com/kubu93/buildroot/-/jobs/507458355

[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/488816222
[2] http://lists.busybox.net/pipermail/buildroot/2020-April/279598.html
[3] 6e45e33f27

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-12 12:08:51 +02:00
James Hilliard
b7d251293a package/python-txaio: drop python 2 support
As of upstream commit 735eb608637e7bbab4082a541ac802cc919fec22,
available since version v20.1.1, support for Python 2.x has been
dropped, and Python >= 3.5 is required. So we make python-txaio depend
on python3, and remove the python2 test of python-txaio.

We also remove the python-six dependency which is no longer used by
txaio, since upstream commit 62b0e7eaa22769687df1de8f57374cb0a42bdc4d.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-11 14:48:45 +02:00
James Hilliard
8a99ce44b4 package/python-autobahn: drop python 2 support
Upstream commit
c0223223f8,
which has been part of the releases since v20.1.2 has dropped support
for Python 2.x. So python-autobahn is now only available for Python
3.x.

We therefore remove the Python 2.x autobahn tests, and while at it,
drop the python-six dependency, which is no longer needed since
upstream commit
79bd2ba41b,
part of the releases since v20.1.3.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-08 22:55:07 +02:00
Romain Naour
50b747f212 support/testing: check if the defconfig provided for testing is valid
Currently, the build continue even if some symbols disapear from
the generated dot config file (.config).

This patch add a new check in order to stop the test if one
of the provided symbol is missing. This must be treated as error.

For example, if a symbol disapear due to new dependency constraints.

Inspired by is_toolchain_usable() function from genrandconfig:
https://git.busybox.net/buildroot/tree/utils/genrandconfig?h=2020.02#n164

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
8e217262a8 support/testing: don't use TestPythonPackageBase.config and linaro toolchain
TestPythonPackageBase.config provide already the basic uClibc toolchain.
So by adding the symbols for the linaro toolchain some warning are printed
at while loading the configuration:

.config:16:warning: override: reassigning to symbol BR2_TOOLCHAIN_EXTERNAL
.config:16:warning: override: BR2_TOOLCHAIN_EXTERNAL changes choice state
.config:17:warning: override: reassigning to symbol BR2_TARGET_ROOTFS_CPIO
.config:21:warning: override: BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM changes choice state

So, some symbol disapear from the generated dot config (.config) leading
to an error due to a new check in the testsuite infra.

Since this test should use the Linaro toolchain, remove
TestPythonPackageBase.config add BR2_arm=y and disable the rootfs tar option.

While at it, re-order the options so that they appear in the same
order as they appear in a defconfig.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Adam Duskett <Aduskett@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
abe170d94f support/testing: test_lxc: use ARM arm external toolchain
Until now, the lxc test was using the ARM CodeSourcery 2014.05 armv5 toolchain.
But the recent systemd version bump to 245 added a toolchain dependency
on systemd package due to build issues with gcc < 5.0.

Before [1] the lxc test was failing to build with the ARM CodeSourcery 2014.05
toolchain. After [1], the test is faling at runtime since the
"BR2_INIT_SYSTEMD=y" symbol disapear from the dot config (.config) due to
the new toolchain dependency.

Fix this by using the same toolchain as for the systemd tests [2]

[1] 2196ee25ff
[2] b3d979c0d1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
96ee994443 support/testing: test_tmux: remove BR2_GENERATE_LOCALE
The BR2_GENERATE_LOCALE can only be used by uClibc based toolchains with the
internal toolchain backend [1].

The test_tmux is using a external uClibc toolchain, so the
"BR2_GENERATE_LOCALE="en_US.UTF-8"" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.

[1] https://git.buildroot.net/buildroot/commit/?id=bd0ffe2206fbd32baf7f4a1dc5fde81cfad70462

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
3bba3bfc59 support/testing: test_glxinfo: remove BR2_TOOLCHAIN_EXTERNAL_LOCALE
The BR2_TOOLCHAIN_EXTERNAL_LOCALE is only defined for uClibc based
custom external toolchains.

The test_glxinfo is using a glibc toolchain, so the
"# BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.

There is the same problem with:
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
9cc3b0c7de support/testing: test_syslinux: remove BR2_TOOLCHAIN_EXTERNAL_LOCALE
The BR2_TOOLCHAIN_EXTERNAL_LOCALE is only defined for uClibc based
custom external toolchains.

The test_syslinux is using a glibc toolchain, so the
"# BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set" line disapear from
the generated dot config (.config) leading to an error due to
a new check in the testsuite infra.

There is the same problem with:
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y
BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
6fa5278511 support/testing: test_syslinux: fix flake8 errors
Reported by gitlab:
https://gitlab.com/kubu93/buildroot/-/jobs/499374911

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-06 22:16:36 +02:00
Romain Naour
b3d979c0d1 support/testing: fix systemd test by using a more recent gcc
As reported on the mailing list [0], there is a build issue with systemd
245 when using gcc < 5.0, due to the following build issue:

    ../src/shared/gpt.c:7:9: error: initializer element is not constant
             { GPT_ROOT_X86,              "root-x86"              },

The pre-built external toolchain we have for armv5 (the default with
just BR2_arm=y) is a very old toolchain from CodeSourcery, which has a
gcc 4.8; we have no other pre-built toolchains for armv5, except by
using a custom one, like those from the Bootlin toolchain builder. But
using a custom toolchain is not nice, as we want our runtime test to
test nominal configurations.

So, switch the systemd tests to use a Cortex-A9, so that we can use the
ARM 2019.12 toolchain, and with VFP, so that it can boot in the qemu
vexpress machine we use for the test-cases.

Fixes:
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144403
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144405
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144406
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144408
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144410
    https://gitlab.com/buildroot.org/buildroot/-/jobs/498144412

[0] http://lists.busybox.net/pipermail/buildroot/2020-April/278931.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
[yann.morin.1998@free.fr:
  - just use cortex-a9_VFP, instead of using a bootlin toolchain
  - adapt the commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-05 20:32:10 +02:00
Thomas Petazzoni
e9b393c676 support/testing: add build tests for the syslinux bootloader
This commit adds four new tests for the syslinux bootloader:

 - Building on x86, for legacy BIOS
 - Building on x86, for EFI BIOS
 - Building on x86-64, for legacy BIOS
 - Building on x86-64, for EFI BIOS

Runtime testing in Qemu would certainly be possible, but is left as a
future addition to these tests.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-04 23:07:36 +02:00
Romain Naour
0207a65323 support/testing: update basic toolchain to bootlin bleeding-edge 2018.11-1
Update the toolchain being used by the testsuite infra.

The new toolchain 2018.11-1 is based on gcc 8.2, uClibc-ng 1.0.30,
linux-headers 4.14 and binutils 2.31.1.
Enable BR2_TOOLCHAIN_HAS_THREADS_DEBUG that is now required.

The old toolchain 2017.05 is based on gcc 4.9, uClibc-ng 1.0.25,
linux-headers 3.10 and binutils 2.27.

Tested with gitlab
https://gitlab.com/kubu93/buildroot/pipelines/132376578

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-04 22:59:34 +02:00
Adam Duskett
f41ee450e5 support/testing/tests/package/sample_gst1_python.py: fix flake8 errors
The older version of flake8 on the autobuilders does not support typehints.
As such, flake8 throws the following error when scanning sample_gst1_python:
E999 SyntaxError: invalid syntax

Remove the typehinting from on_message to fix this issue.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-29 15:50:07 +02:00
Adam Duskett
207d276d98 support/testing: fix test_gst1_python
The config is missing the following packages:
  - BR2_PACKAGE_GST1_PLUGINS_BASE
    This is needed for the videotestsrc plugin which provides
    videotestsrc.

  - BR2_PACKAGE_GST1_PLUGINS_BAD
    This is needed for the debugutils plugin which provides
    fakevideosink.

  - BR2_PACKAGE_GST1_PLUGINS_BASE_PLUGIN_VIDEOTESTSRC
    videotestsrcplugin used by the example pipeline.

  - BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_DEBUGUTILS
    fakevideosink plugin used by the example pipeline.

The sample was also amended to fix:
  - no call to main.

  - using autovideosink instead of fakevideosink. Using Fakevideoskink
    is preferred because its primary purpose is for debugging and sample
    pipelines. Autovideosink does not work because there is no video
    output device.

  - No function on_message. Without this function, the script fails
    because of the missing function.

  - The script sets the pipeline to Gst.State.EOS instead of
    Gst.State.NULL which results in a failed pipeline state. When the
    state is set to NULL, gstreamer automatically calls EOS. Manually
    setting the pipeline state to EOS results in the following error:

        Trying to dispose element sink, but it is in PLAYING instead of
        the NULL state. You need to explicitly set elements to the NULL
        state before dropping the final reference, to allow them to
        clean up. A refcounting bug may also cause this problem in the
        application or some element.

In addition, the default timeout is set too low and would result in a
test failure, this has been changed to 200 seconds.

These issues are now properly fixed, and the sample script passes.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[yann.morin.1998@free.fr:
  - meld the two patches together
  - add the print() in the on_demand callback
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-27 18:31:45 +01:00
Thomas De Schampheleire
a570f63dd4 Revert "package/perl-crypt-ssleay: new package"
This reverts commit 25033cfb86.

This perl package is deprecated and gives following recommendations while
building:

    *** THIS IS NOT AN ERROR, JUST A MESSAGE FOR YOUR INFORMATION ***

        Do you really need Crypt::SSLeay?

        Starting with version 6.02 of LWP, https support was unbundled into
        LWP::Protocol::https. This module specifies as one of its prerequisites
        IO::Socket::SSL which is automatically used by LWP::UserAgent unless
        this preference is overridden separately. IO::Socket::SSL is a more
        complete implementation, and, crucially, it allows hostname
        verification. Crypt::SSLeay does not support this. At this point,
        Crypt::SSLeay is maintained to support existing software that already
        depends on it.

        However, it is possible that your software does not really depend on
        Crypt::SSLeay, only on the ability of LWP::UserAgent class to
        communicate with sites over SSL/TLS.

        If are using version LWP 6.02 or later, and therefore have installed
        LWP::Protocol::https and its dependencies, and do not explicitly use
        Net::SSL before loading LWP::UserAgent, or override the default socket
        class, you are probably using IO::Socket::SSL and do not really need
        Crypt::SSLeay.

        Before installing Crypt::SSLeay, you may want to try specifying a
        dependency on LWP::Protocol::https.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-26 22:26:20 +01:00
Marcin Niestroj
803fc6ad51 support/testing: add netdata test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 22:25:40 +01:00
Thomas Petazzoni
1214109cdc support/testing/tests/package/test_python_gobject.py: fix flake8 warning
Fixes:

   support/testing/tests/package/test_python_gobject.py:4:1: E302 expected 2 blank lines, found 1

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 15:54:29 +01:00
Thomas Petazzoni
4a180c34b7 support/testing/tests/package/test_gst1_python.py: fix flake8 warning
Fixes:

  support/testing/tests/package/test_gst1_python.py:29:1: W391 blank line at end of file

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 15:54:29 +01:00
Thomas Petazzoni
75371a4a12 support/testing/tests/package/sample_python_gobject.py: fix flake8 warnings
Fixes:

  support/testing/tests/package/sample_python_gobject.py:5:1: E302 expected 2 blank lines, found 1
  support/testing/tests/package/sample_python_gobject.py:8:7: E111 indentation is not a multiple of four
  support/testing/tests/package/sample_python_gobject.py:11:1: E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 15:54:29 +01:00
Thomas Petazzoni
15f354e7df support/testing/tests/package/sample_gst1_python.py: fix flake8 warnings
Fixes the following flake8 warnings:

  support/testing/tests/package/sample_gst1_python.py:5:1: F401 'time' imported but unused
  support/testing/tests/package/sample_gst1_python.py:7:1: E402 module level import not at top of file
  support/testing/tests/package/sample_gst1_python.py:21:12: W292 no newline at end of file

For the E402 warning, we add a "noqa" marker, as we really want the
gi.require_version() to be before.

There is still one remaining warning to be fixed:

  support/testing/tests/package/sample_gst1_python.py:18:28: F821 undefined name 'on_message'

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 15:54:13 +01:00
Thomas Petazzoni
7e025dd80b support/testing/tests/init/init_openrc: fix flake8 F401 warning
Fixes:

support/testing/tests/init/test_openrc.py:1:1: F401 'infra.basetest' imported but unused

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-24 15:28:21 +01:00
Adam Duskett
1101d7cb09 support/testing/: change python-gobject to use prebuilt kernel
Currently, we build a complete AArch64 system, including a kernel. This
can take quite some time.

Switch to an armv7 system, which allows us to use one a prebuilt kernel,
thus significantly reducing the test time.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[yann.morin.1998@free.fr:
  - use a standalone config; don't inherit from TestPythonPackageBase
  - use the default external toolchain (ARM, not Linaro)
  - rewrite commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-22 15:25:34 +01:00
Adam Duskett
3f9bf62953 support/testing: add gst1-python test case
This test case runs a simple pipeline for 100 frames to ensure that
gst1-python works properly.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-22 11:55:31 +01:00
Francois Perrad
6500cb3cc6 package/lua-lyaml: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-20 23:00:30 +01:00
Thomas De Schampheleire
0c839dc11c support/testing: add missing test file for perl-html-parser
This file was created by utils/scancpan while adding other packages but
apparently not yet added in the repo.

Assign this test case to Bernd in the DEVELOPERS file since he is
listed as the maintainer for this package.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-15 21:38:56 +01:00
Thomas De Schampheleire
25033cfb86 package/perl-crypt-ssleay: new package
Added via utils/scancpan, adding a host dependency to perl-try-tiny and
target dependency to openssl.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-15 21:38:56 +01:00
Thomas De Schampheleire
f8638b99ae package/perl-lwp-protocol-https: new package
Dependency for perl-crypt-ssleay (added in subsequent commit).

Added via utils/scancpan, without changes.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-03-15 21:38:56 +01:00
Adam Duskett
179e6766a6 package/python-gobject: bump version to 3.36.0
Other changes:
  - Convert the package to a meson package.
  - Remove 0001-add-PYTHON_INCLUDES-override.patch as it no longer applies.
  - Add gobject-introspection as a dependency.
  - Add the package under myself in the DEVELOPERS file.

Because gobject-introspection is now a dependency of python-gobject, the test
must be updated at the same time.

 - Change TestPythonPy2Gobject to TestPythonPy3Gobject as
   gobject-introspection requires python3.

 - Refactor test_python_gobject.py to no longer inherit the
   TestPythonPackageBase class, as this class uses a base config that does not
   support gobject-introspection.

 - Update sample_python_gobject to use Glib to find the path of sh.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-12 22:21:55 +01:00
Adam Duskett
23d3433302 support/testing: add openrc tests
Two simple tests to ensure that openrc boots without any services crashing
with a read only and a read write filing system.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Carlos Santos <unixmania@gmail.com>
[yann.morin.1998@free.fr: really check the init process]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-11 18:50:38 +01:00
Romain Naour
b1e4404c04 support/testing: test_systemd.py: add linux fragment to enable CONFIG_BINFMT_MISC
While investigating [1] one units failed due to missing kernel option
CONFIG_BINFMT_MISC needed by "proc-sys-fs-binfmt_misc.mount" service.

It's because the kernel support autofs4 but not MISC binaries.

Since the systemd test infra use the default defconfig (vexpress),
we need to provide a linux fragment to enable CONFIG_BINFMT_MISC.

[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/454255917

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
[yann.morin.1998@free.fr:
  - move the kernel config with the others in conf/
]
Tested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-01 18:43:42 +01:00
Yegor Yefremov
593bba41ab support/testing: add libftdi1 test case
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-02-23 11:12:09 +01:00
Yegor Yefremov
5abe7e4ce3 support/run-tests: reorder imports
Reorder imports using the isort utility to fix a warning from pylint3:

wrong-import-order: standard import "import multiprocessing" should be
placed before "import nose2"

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-02-17 10:13:08 +01:00
Yegor Yefremov
ea31dc1cd4 support/run-tests: check for empty sequences in a pythonic way
According to PEP8 empty sequences should be checked as booleans.

Fixes the following PEP8 warning:
Do not use `len(SEQUENCE)` to determine if a sequence is empty

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-02-16 15:38:03 +01:00
Romain Naour
faec5c583e support/testing/glxinfo: explicitely enable GLX
Since [1], the GLX support is enabled by BR2_PACKAGE_MESA3D_OPENGL_GLX
symbol.

Since [2], only one swrast provider can be built.
Keep BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/400391349

[1] 5cb821d563
[2] 09a0a28507

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-02-15 11:59:13 +01:00
Titouan Christophe
1b2319f931 support/testing: add runtime test for Crudini (py2 and py3)
This also adds the new tests to the gitlab CI configuration.

Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-05 16:23:42 +01:00
Matt Weber
496a43c777 support/testing: new opkg test case
- Validates an archive can be installed and removed
- Builds an archives that uses postinst and prerm scripts

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-03 22:17:42 +01:00
Yegor Yefremov
96bbea75a3 support/testing: add python-can test case
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-01-31 08:05:21 +01:00
Yann E. MORIN
ed56b7b547 support/testing: really fix python-gitdb2 test
The test-case for python-gitdb2 consists solely in verifying that the
module can indeed be imported.

However, flake8 errors out on unused imports. Furthermore, it also
errors about wildcard imports, as it can detect unused symbols.

Commit d8c86be9cd (support/testing: fix python-gitdb2 test) tried to
address this issue, by explicitly squelching the two errors, F401 and
F403.

While that works on recent distros, the image used by our docker
pipeline is laggign behind and the flake8 there only handles at most a
single error in the noqa list.

Do as is done with the other python samples, and just blindly ignore
all errors.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-01-04 09:29:36 +01:00
Titouan Christophe
9fa2add810 support/testing: add test for python-avro
This adds a test case for python-avro, with a script that
performs a simple deserialization.

Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2019-12-29 21:38:33 +01:00
Yann E. MORIN
d8c86be9cd support/testing: fix python-gitdb2 test
Cthe test-case for python-gitdb2 consists solely in verifying that the
module can indeed be imported.

However, flake8 errors out on unused imports. Furthermore, it also
errors about wildcard imports, as it can detect unused symbols.

Squelch those errors.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2019-12-18 21:12:22 +01:00
Adam Duskett
5ef1679635 support/testing: add gitdb2 test
Add a simple test case that imports the module.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-12-16 23:36:45 +01:00
Adam Duskett
f81f673a78 support/testing: add smmap2 test
Add a simple test case that imports the module and instantiates a new
SlidingWindowMapManager class.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-12-16 23:28:08 +01:00
Arnout Vandecappelle (Essensium/Mind)
17638bc3ad support/testing: test_python_django: add missing line
flake8 complains with:

test_python_django.py:25:1: E302 expected 2 blank lines, found 1

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-12-06 00:32:34 +01:00
Adam Duskett
8583b2c6d8 support/testing: add python-django test
This test comprises of four simple steps:
  1: Start a new simple project called testsite.
  2: Run ./manage.py migrate on the new testsite.
  3: Run ./manage.py runserver 0.0.0.0:1234 & sleep 30
    - The sleep 30 is necessary as it may take several seconds for
      the django server to fully start.
  4: Run netstat to ensure the server opened port 1234.

Signed-off-by: Adam Duskett <aduskett@greenlots.com>
[Thomas: use self.assertRunOk() when appropriate]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-12-05 23:02:14 +01:00
Adam Duskett
600132f8c0 support/testing: add python-gobject test
Add a simple test case that imports the module.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-12-03 23:49:34 +01:00
Pierre-Jean Texier
bda4008975 support/testing: basetest.py: fix code style
Fix these warnings:

W291 trailing whitespace

Fixes:
 - https://gitlab.com/buildroot.org/buildroot/-/jobs/360824861

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-25 21:40:52 +01:00
Pierre-Jean Texier
bd18e74eeb support/testing: test_lxc.py: fix code style
Fix these warnings:

E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E265 block comment should start with '# '
E302 expected 2 blank lines, found 1
F401 'pexpect' imported but unused

Fixes:
 - https://gitlab.com/buildroot.org/buildroot/-/jobs/360824861

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-25 21:40:41 +01:00
Patrick Havelange
1ca16b1feb support/testing: add lxc test
The test starts a simple container with an iperf3 server.
The container is using the tini init system, with a shared rootfs.
An iperf3 client is started from the host to check that the container
is really up and running.

Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-20 22:38:25 +01:00
Patrick Havelange
63966e56a3 support/testing: add assertRunOk method to BRTest class
This method asserts that the given command ran successfully.
The goal is for it to be used by the different tests when needed.

Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-20 22:34:53 +01:00
Ricardo Martincoski
4a40d36f13 support/testing: switch to Python 3 only
Python 2.7 will not be maintained past 2020.

Many scripts on the tree are used during the build and should keep
Python 2 compatibility for a while.
This is not the case for the runtime test infra. It's meant to be run in
modern distros only, so it can safely switch to support Python 3 only.

An advantage of this approach is to have less scenarios to test in.
Otherwise every change to the test infra or runtime tests would need to
be tested against both versions of the interpreter, increasing the
effort of the developers, to ensure the compatibility to Python 2 was
not broken.

In order to accomplish the change to Python 3:
 - change the shebang for run-tests;
 - use Python 3 urllib as a drop-in replacement for Python 2 urllib2;
 - when writing the downloaded binary files, explicitly open the output
   file as binary;
 - when subprocess is used to retrieve the text output from commands,
   explicitly ask for text output. For this, use 'universal_newlines'
   because 'text' was added only on Python 3.7;
 - when pexpect is used to retrieve the text output from qemu or git,
   explicitly ask for text output using 'encoding';
 - the code using csv currently follows the example in the documentation
   for the Python 2 module, change it to follow the example in the
   documentation for the Python 3 module;
 - fix the relative import for test_git.py to be Python 3 compliant.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-28 22:14:07 +01:00
Peter Korsgaard
1cab59a5e8 package/docker-compose: bump version to 1.24.1
Fixes https://gitlab.com/buildroot.org/buildroot/-/jobs/332656041

The recent bump of a number of python packages broke docker-compose, as
docker-compose specifies both minimum and maximum versions for (most of) its
dependencies:

Dependencies of docker-compse 1.20.1 (! = unmet):
cached-property: < 2 (currently 1.51)
docopt: < 0.7 (currently 0.6.2)
! pyyaml: < 4.0, patched to < 4.3 (currently 5.1.2)
requests: < 2.19, patched to < 3 (currently 2.22.0)
! texttable: < 0.10 (currently 1.6.2)
websocket-client: < 1.0 (currently 0.56.0)
! docker: < 4.0 (currently 4.1.0)
dockerpty: < 0.5 (currently 0.4.1)
six: < 2 (currently 1.12.0)
jsonschema: < 3 (currently 2.5.1)
enum34: < 2 (currently 1.1.6)
backports.ssl-match-hostname: >= 3.5 (currently 3.7.0.1)
ipaddress: >= 1.0.16 (currently 1.0.23)

To fix this, bump docker-compose to the most recent release (1.24.1).  This
is unfortunately not enough, as our docker, pyyaml, requests and texttable
packages are too new, so add 3 patches from upstream to relax the version
checks of dependencies.  Notice that patch 0003 is from
https://github.com/docker/compose/pull/6623 and has not been merged yet.

Discussions around the problem of these maximum versions of the dependencies
and the fact that all downstream users have to patch it is ongoing here:

https://github.com/docker/compose/issues/6756

docker-compose 1.24.1 added a requirement for ssh support in python-docker in:
7b82b2e8c7

So add a dependency for python-paramiko and update the toolchain dependency
for C++ (from python-paramiko -> python-cryptography) and adjust the
toolchain configuration of the runtime test to match.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-26 20:17:14 +02:00
Francois Perrad
76b4fcb5dc support/testing: add luvi test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-26 16:33:12 +02:00
Francois Perrad
d1e01107a7 support/testing: add lua-sdl2 test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-26 16:31:36 +02:00
Francois Perrad
308ef3166b support/testing: add lua-gd test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-26 16:31:03 +02:00
Francois Perrad
eaa2c3a6f7 support/testing: add lzlib test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-26 16:30:58 +02:00
Ricardo Martincoski
f3c4a9e1cd support/testing: test_syslog_ng: improve commands
There is no need for double grep, so choose a better regexp. Use &&
instead of ; between commands so the sequence of commands fail faster.

Break the last sequence of commands in 2 calls run() so the proper
return code can be tested for each.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-16 21:33:04 +02:00
Ricardo Martincoski
d3be2087d0 support/testing: add tmux test
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-12 15:37:51 +02:00
Marcin Niestroj
f8b8a9c529 support/testing: add turbolua test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-10-02 20:57:08 +02:00
Francois Perrad
101ae29c07 support/testing: add lua-cqueues test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-29 09:15:08 +02:00
Sergey Matyukevich
2a220de466 binaries-marvell: bump version and switch to common firmware
Since release 18.12 binaries-marvell repository provides
common firmware supporting both A7K and A8K SoC families.

This commit bumps package version to 18.12 and removes
platform specific binary selections from Config.in.
Single firmware image suitable for both A7K and A8K
platforms is now specified in mk file explicitely.

Legacy handling is not needed, as configs which did have
the option set will continue to work without change.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
[Arnout:
 - Remove BINARIES_MARVELL_IMAGE entirely;
 - Add remark about legacy handling;
 - Remove the deprecated option from the defconfigs and test that use
   it.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-09-21 18:31:26 +02:00
Peter Korsgaard
d8e6851f11 Merge branch 'next' 2019-09-03 15:03:02 +02:00
Victor Huesca
152776b420 support/testing: new test for root password
Add support to test that the root passowrd is working as expected.
- Buildtime test: Check the hash present in the generated '/etc/shadow'.
- Runtime test: Build an armv7 image and try to login with a password.

Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-13 22:30:41 +02:00
Ricardo Martincoski
620c98a214 support/testing: factor out run_cmd_on_host
Currently many test cases call subprocess.check_output on their own.
Factor out that code to an infra method so the call get standardized.

This will be handful when switching the test infra to use Python 3.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:58 +02:00
Ricardo Martincoski
1b2928201f support/testing: do not print non-utf-8 chars on cbor test
The python-cbor sample script to be run on the target dumps a binary
variable to the target stdout. This is done for debug purposes-only, in
the case the test on test infra fails.
This non-utf-8 is currently silently ignored by the Emulator class /
pexpect.spawn from the test infra because the infra uses Python 2, that
in turn do not differentiate between string and byte data.

Make the code Python3-friendly (Python 3 in the host) by doing the right
thing and encoding the data before printing it.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:55 +02:00
Ricardo Martincoski
8de4291bd2 support/testing: check ubi image before booting
Do the same as other fs tests and minimally check the ubi image before
booting.
The call to 'file' was already there, but the output wasn't tested for
some unknown reason. Add the assert for the output of the command.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:53 +02:00
Ricardo Martincoski
a565917046 support/testing: provide entropy to perl tests
Newer versions of perl-io-socket-ssl require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
perl tests.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139402

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
eb6b0fd87a support/testing: provide entropy to lua tests
Newer versions of lua-http require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
lua tests.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139374
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139376

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
d23056ffe1 support/testing: make crossbar to use umsgpack
Since version 19.3.3, python-autobahn assumes that when CPython is used
msgpack will be used as well [1]. But it still allows the user to
override this behavior by setting an environment variable [2] to use
umsgpack.

Make the test to explicitly use umsgpack since it is part of the minimal
config (python-crossbar selects python-u-msgpack).

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139430

[1] ea019b8042
[2] fe70ceebe0

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
ef5e87bb92 support/testing: make treq tests to ignore warning
Newer versions of python-treq display a warning at runtime when
service_identity is not installed:
"Without the service_identity module, Twisted can perform only
rudimentary TLS client hostname verification."

This warning message confuses the test that looks for another string in
stdout. Make the test ignore other messages while still expecting
"Connection refused".

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139449
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139450

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
5c39c90ae1 support/testing: fix hardening tests
Since "2467822c85 package/checksec: bump to version 2.1.0" the hardening
tests fail because upstream slightly changed the way the script is
called.
According to README.md: "- All options now require `--$option=$value`
instead of `--$option $value`"

Instead of just replacing '--output json' with '--output=json' take into
account that upstream also changed the usage example to show --format
instead of --output. Both options do exactly the same, but following the
usage example seems to be more future-proof.

Upstream also improved the json output. Now when a file is passed as
parameter, the json has the file name as the main key, instead of the
string "file". Adjust the test cases accordingly.

Fixes:
tests.core.test_hardening.TestFortifyConserv
tests.core.test_hardening.TestFortifyNone
tests.core.test_hardening.TestRelro
tests.core.test_hardening.TestRelroPartial
tests.core.test_hardening.TestSspNone
tests.core.test_hardening.TestSspStrong

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-05 22:15:18 +02:00
Ricardo Martincoski
1ad5e0a6a6 support/testing: test check-package ignores external.mk
Check external.mk is ignored only when in the root path of a
br2-external.

Add a file called external.mk as a fixture to be used by the test case.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: wrap at 80 columns]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
4323fbaf8d support/testing: add test for check-package
Check the basic usage for check-package.

It can be called using either absolute path, relative path or from PATH.
Files to be checked can be passed with either absolute path or relative
path (also including files in the current directory).

Also check it ignores some special files when checking intree files,
i.e. package/pkg-generic.mk, while still generating warnings for out-of-tree
files when called with -b.
In order to allow the later, add an empty line to the Config.in in the
br2-external being tested so the script does generate a warning.

Catches bug #11271.

More tests can be added later, for example compatibility to Python 3.

Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: wrap at 80 columns where appropriate; merge into a single
class.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
8669243da3 support/testing: allow run-tests to be called from anywhere
Currently run-tests must be called from the Buildroot top directory.

Derive the top directory from the script path, so run-tests can be called from
any path.
As a consequence the test infra will always test the repo it belongs to.

Suggested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
9609a7c2c2 support/testing: test atop with builtin kernel
Use the new builtin armv5 4.19 kernel to test atop.
The atop package cannot be tested using BASIC_TOOLCHAIN_CONFIG because
it needs kernel headers >= 3.14. So use an updated version of it,
copying the config fragment from
support/config-fragments/autobuild/br-arm-full.config

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
650e9903a9 support/testing: drop haveged for python tests
The new builtin armv5 kernel 4.19 uses VirtIORNG to provide entropy, so we
no longer need haveged.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508536
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508537
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508542
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508543

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Peter: reword]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
56e38a705e support/testing: test syslog-ng with builtin kernel
Use the new builtin kernel 4.19 with VirtIORNG to provide entropy to
test syslog-ng.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
7acb32dabb support/testing: add builtin armv5 kernel 4.19 with entropy
More and more packages being tested by the test infra, e.g. syslog-ng,
need entropy at startup, usually reading from /dev/random.

Some test cases can also depend on a kernel version newer than the
builtin ones already provided by the test infra:
 - 3.11.0 for armv5;
 - 4.0.0 for armv7.

Add a new builtin kernel to be used by such test cases.
Add it for armv5 so most test cases that switch to use this kernel can
keep using BASIC_TOOLCHAIN_CONFIG.
Use the same kernel version and kernel config as qemu_arm_versatile plus
HW_RANDOM_VIRTIO for VirtIORNG to be usable.
Copy the actual binary file from the syslog-ng runtime test at current
master @ 29e1cb8884.

Since there is already a 'kernel-versatile' file on autobuild.buildroot.net
and we must keep it with this name for reproducibility purposes, create a
simple naming convention for newer builtin kernel images and dtb files:
kernel-<defconfig>-<kernel_series_version>
<dtb_name>-<kernel_series_version>.dtb
Pass '-device virtio-rng-pci' to qemu when this kernel is used.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Peter: use this new kernel instead of the old builtin/armv5 kernel]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Francois Perrad
1a49dcb528 support/testing: improve test_luasyslog
Lua has a builtin lsyslog module, so let's test this one as well.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-02 17:35:44 +02:00
Ricardo Martincoski
450863097f support/testing: use virtio-rng-pci to test syslog-ng
Recent versions of syslog-ng need some entropy on startup.
So use VirtIORNG to provide it. In order to accomplish this:
 - build the kernel containing the driver;
 - pass '-device virtio-rng-pci' to qemu.

Use the same kernel version and kernel config as qemu_arm_versatile.
It already has PCI enabled but it does not have HW_RANDOM_VIRTIO, so add
a defconfig fragment to enable the drivers.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/259856394

At the same time, fix a typo (missing '#') that resulted in the
generation of root.tar. This file is not used in the test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-01 20:31:54 +02:00
Francois Perrad
0b32cd888e package/luaposix: bump to version 34.1.1
the module std.normalize is no longer a dependency

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-01 18:11:33 +02:00
Francois Perrad
a0b0976c93 support/testing: add luaposix test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-01 10:11:18 +02:00
Daniel J. Leach
6aee78c894 support/testing: openjdk JNI test cases
This test case builds a native library and ensures a Java class can load
and interact with the native library. The test also verifies Java code
can make system calls via the native library.

Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Acked-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Tested-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-06-30 17:32:34 +02:00
Thomas Petazzoni
345c29a4b7 support/testing/infra/emulator: increase memory size used by Qemu
By default, Qemu emulates a system with 128 MB of RAM. This is not
sufficient for some test cases we have, such as TestPerlDBDmysql,
where the initramfs is quite large. Therefore, this commit extends the
RAM size emulated by Qemu to 256 MB.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/237108668

Thanks to Arnout for the analysis of the issue.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-06-22 19:28:36 +02:00
Francois Perrad
c8ba0f7488 package/luasyslog: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-06-20 18:25:15 +02:00
Thomas Petazzoni
17809276b5 support/testing/tests: drop Python 2.x test in test_ipython
Since commit 6ebaef3818
("package/python-ipython: bump to version 7.4.0"), ipython is no
longer available for Python 2.x, as it requires Python 3.x.

However, the corresponding test case that was testing iPython under
Python 2.x was not removed at the same time, causing a failure of
TestIPythonPy2 test. Let's drop the test that is no longer relevant.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/210208754

Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-05-18 22:41:22 +02:00
Romain Naour
ec68531774 support/testing: add test for xserver/Mesa OpenGL/glxinfo
This test allow to check if the xserver with GLX is working properly.
This is a basic test but it allow to trigger the current bug reported
by [1].

To test if the glxinfo test is working, you can change "-display :0" by
"-display :1" in the glxinfo command line.

[1] https://bugs.buildroot.org/show_bug.cgi?id=11591

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Youssef Harmouch <youssef.harmouch@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-04-11 18:23:40 +02:00
Francois Perrad
47db24b93a support/testing/tests/package/test_perl_*: regeneration
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-31 14:01:50 +02:00
Francois Perrad
6aaba51f3b support/testing/tests/package/test_lua*: regeneration
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-31 12:15:02 +02:00
Francois Perrad
5593ed554f support/testing: add lua-http test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 19:01:00 +01:00
Francois Perrad
6de6f21dc3 support/testing: add rings test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
7ad1a47f9f support/testing: add luasocket test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
97032e2f4f support/testing: add luasec test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
d2db91b334 support/testing: add luaossl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
3a3a661303 support/testing: add luafilesystem test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
596465e61a support/testing: add luaexpat test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
f4dc945fbc support/testing: add lua-utf8 test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
f830d89121 support/testing: add lua-curl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
e7e6b2478d support/testing: add lsqlite3 test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Francois Perrad
ac2bfe7643 support/testing: add lpeg test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-28 17:20:00 +01:00
Adam Duskett
df7bf8fb41 support/testing/tests/package/test_openjdk: new test
This test is a simple "Hello, World" integration test of the OpenJDK
package.

It compiles the Java app on the host, then runs it on an emulated
AARCH64 target and verifies "Hello, World" is printed.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-17 14:02:18 +01:00
Ricardo Martincoski
7b2dd942d0 Revert "testing/tests/download: test git special ref"
This reverts commit 81771cfcdc.

The download of sha1 of a special ref currently works or not depending
on the git client version in use.
With git version 2.11.0 (present in the docker image) it does not work.
With git version 2.17.1 it works.

For the sake of reproducibility, remove this part of the TestGitRefs
test case until some code gets added to the download infra to handle
sha1 of a special ref for any git client version.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/158295269

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-03-01 11:09:41 +01:00
Peter Korsgaard
11c55c94da support/testing: build a glibc toolchain for docker / docker-compose tests
runc (which is a reverse dependency of docker-engine) is about to gain a
!uclibc dependency, so move to a glibc toolchain instead.

There are currently no prebuilt x86_64 / core2 / glibc toolchains available,
so instead use the internal toolchain backend to build one.

While we are at it, drop the infra.basetest.BASIC_TOOLCHAIN_CONFIG
reference, as that ARM toolchain configuration doesn't make any sense for
this x86-64 based test.

add docker / docker-compose tests

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-12 20:03:46 +01:00
Ricardo Martincoski
81771cfcdc testing/tests/download: test git special ref
Add a special ref to the static repo and check on the git refs test case
the download of a git package:
 - with the sha1 of a special ref as version;

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: change to use the sha1 of a special ref instead of the name]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 11:13:20 +01:00
Ricardo Martincoski
ed3cda38cd testing/tests/download: test git tag
Add a tag to the static repo and check on the git refs test case the
download of a git package:
 - with the name of a tag as version;
 - with the sha1 of a tag itself as version;
 - with the partial sha1 of a tag itself as version;
 - with the sha1 of a commit pointed by a tag as version;
 - with the partial sha1 of a commit pointed by a tag as version;
 - with the sha1 of a commit reachable only by a tag as version;
 - with the partial sha1 of a commit reachable only by a tag as version.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 10:57:26 +01:00
Matt Weber
c6386be05b testing/download: GIT use BRConfigTest
Enables the test to use the new non-emulator base class which takes
significantly less test time.

Cc: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 10:40:15 +01:00
Ricardo Martincoski
0461ef4dea testing/infra: Add BRConfigTest as superclass of BRTest
The git tests don't need to do a full build, they only need to do a
configure and download and/or legal-info. More tests of that type will
be added in the future. Therefore, we want to have a test base class
that doesn't automatically do a full build in the setUp().

Add this new class as a superclass of the existing BRTest class, so we
don't need to update existing tests. Only the code in run-tests that
iterates over all subclasses of BRTest has to be adapted to use
BRConfigTest instead.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 10:40:15 +01:00
Ricardo Martincoski
9cc7a12bde testing/tests/download: test git submodules
Add two submodules as static repos, add a branch to the main static repo
and check on the git refs test case the download of a git package:
 - repo with submodule but without support in the package;
 - repo with recursive submodules with support in the package.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: remove handling of inconsistent tarball hashes - that's an
 actual bug that should be fixed]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 09:56:58 +01:00
Peter Korsgaard
a9219660d3 support/testing: add docker / docker-compose tests
Build for x86-64 as public containers in general are only available for
x86-64.  Docker needs a number of kernel options enabled, so use a custom
kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to
expose entropy to the guest.  The default RAM amount (128M) is not enough to
run docker / docker-compose, so bump to 512MB.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-05 23:05:00 +01:00
Peter Korsgaard
092b2b5ada support/testing: test_dropbear.py: drop ssh port forward to the host
It is not used and may conflict with other services running on port 2222 on
the host machine.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-05 19:04:07 +01:00
Peter Korsgaard
cc84a0dac6 support/testing: test_dropbear.py: add test for ssh connection to localhost
Use sshpass to pass the password on the command line.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-05 19:01:55 +01:00
Peter Korsgaard
a34005699b support/testing: test_dropbear.py: use a constant for the password
The password is used in multiple places, so add a constant for it instead of
hardcoding it multiple times.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-05 19:01:42 +01:00
Ricardo Martincoski
5692e6afa1 testing/tests/download: test git branch
Add a branch to the static repo and check on the git refs test case the
download of a git package:
 - with a sha1 reachable by a branch name, but not pointed by it, as
   version. This is the most common use case for git refs in the tree;
 - with a partial sha1 of a commit reachable by a branch as version;
 - with a sha1 of the commit head of a branch as version;
 - with a partial sha1 of the commit head of a branch as version;

Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-05 10:23:03 +01:00
Ricardo Martincoski
0a90740488 testing/tests/download: test case for git refs
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.

Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
 - trying to download an invalid sha1 generates an error;
 - downloading a valid sha1 that contains unexpected content generates
   an error.

In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-04 20:44:29 +01:00
Thomas Petazzoni
7dd2f4f3e1 support/testing/tests/download/gitremote: remove trailing whitespace
Fixes the following flake8 warnings:

support/testing/tests/download/gitremote.py:29:67: W291 trailing whitespace
support/testing/tests/download/gitremote.py:30:60: W291 trailing whitespace

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-02-04 17:10:35 +01:00
Ricardo Martincoski
f284b5e7a1 testing/tests/download: add git hash test
Add one test case to ensure the hash is checked for git packages:
 - correct hash;
 - wrong hash;
 - no hash file.

Add required infra:
 - a GitRemote class, that can start a git server in the host machine to
   emulate a remote git server under the control of the test;
 - a new base class, called GitTestBase, that inherits from BRTest and
   must be subclassed by all git test cases.
   Its setUp() method takes care of configuring the build with a
   br2-external, avoiding to hit http://sources.buildroot.net by using
   an empty BR2_BACKUP_SITE. It also avoids downloading not
   pre-installed dependencies (i.e. lzip) every time by calling 'make
   dependencies' using the common dl directory, and it instantiates the
   GitRemote object.

Besides the Python scripts, add some fixtures used during the tests:
 - a br2-external (git-hash) with one package for each part of the test
   case;
 - a static git bare repo (repo.git) to be served using GitRemote class.

Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.

Thanks to Arnout for the hint about how to add a bare repo to test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Arnout: split long line; reorder imports to satisfy flake8]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-04 16:52:58 +01:00
Ricardo Martincoski
4d5d0124c7 support/testing/infra/basetest: support br2-external
Some upcoming test cases can use one or more br2-external trees as
fixtures that provide packages used only in runtime tests.

Add support for br2-external into the BRTest class. Any test case can
then provide a list of paths for being used as br2-external trees
during the build of the image to test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Thomas: use named argument for make_extra_opts.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-29 22:16:33 +01:00
Ricardo Martincoski
07bbf1b4dd support/testing/infra/builder: configure and build with make target and environment
Make the builder able to call 'VAR1=1 make VAR2=2 target'.

Allow sending extra parameters to be added to the end of make command
line. Uses for these purposes:
 - to configure a br2-external, using the 'BR2_EXTERNAL="dir" variable.
 - to specify a make target, such as 'foo-source.'

Allow adding variables to the environment when calling make.
These added variables allow a user to override default values from BuildRoot,
such as 'BR2_DL_DIR="dl"'.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-29 22:14:55 +01:00
Ricardo Martincoski
e2b98dfede support/testing: add atop test
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-28 10:00:45 +01:00
Thomas Petazzoni
ba99ccaae4 support/testing: drop docstring in TestPythonPackageBase::test_run()
Having a docstring in the test_run() method clutters the "run-tests
-l" output:

test_run (tests.package.test_python_crossbar.TestPythonPy3Crossbar)
Test a python package. ... ok
[...]
test_run (tests.package.test_python_pexpect.TestPythonPy2Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_pexpect.TestPythonPy3Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy2Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy3Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy2Pynacl)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy3Pynacl)
Test a python package. ... ok

So let's simply drop this docstring that is not particularly useful.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-08 20:50:08 +01:00
Francois Perrad
9d08630128 support/testing: add prosody test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 23:08:59 +01:00
Francois Perrad
bcfbf6437a support/testing: add lua test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:37:10 +01:00
Francois Perrad
2d1f32f084 support/testing: add perl-io-socket-ssl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:46 +01:00
Francois Perrad
35c858bd82 support/testing: add perl-io-socket-multicast test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:40 +01:00
Francois Perrad
d5196b8ee9 support/testing: add perl-encode-decode test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:36 +01:00
Francois Perrad
7046c20476 support/testing: add perl-dbd-mysql test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:30 +01:00
Francois Perrad
fe4b91db57 support/testing/tests/packages/test_perl_*: regeneration
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:19:59 +01:00
Francois Perrad
a03f66f431 support/testing: add perl-xml-libxml test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872335]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:52:15 +01:00
Francois Perrad
d9f5748a07 support/testing: add perl-x10 test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872334]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:52:10 +01:00