Commit Graph

1325 Commits

Author SHA1 Message Date
Kory Maincent
c656373aeb support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS
The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
Add dedicated Grub2 builtin config for the EFI compatible cases.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-03 14:45:10 +02:00
Kory Maincent
d6d7cbb8e0 support/testing: don't fail on tests emitting invalid utf-8 sequences
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.

However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:

    Traceback (most recent call last):
      [...]
        emulator.login()
      File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
        index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
        return self.expect_list(compiled_pattern_list,
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
        return exp.expect_loop(timeout)
      File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
        incoming = spawn.read_nonblocking(spawn.maxread, timeout)
      File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
        return super(spawn, self).read_nonblocking(size)
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
        s = self._decoder.decode(s, final=False)
      File "/usr/lib/python3.8/codecs.py", line 322, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.

We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.

[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
  - don't change encoding, use codec_errors
  - rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-03 14:26:12 +02:00
Yann E. MORIN
556a0a1104 Revert "make: support: use command -v' instead of which'"
This reverts commit ca6a2907c2.

Switching to using 'command -v' instead of 'which', opened a can of
worms that is hard to fix in a timely manner:

  - recursive call to 'make' from a post-build, post-iamge script, fails
    because of a redefinition of HOSTCC_NOCCACHE (a bug on its own that
    needs a separate fix anyway) [0];

  - 'make' believeing it can call "simple" commands with execve() et al.
    instead of passing them through a shell via system(), and thus
    failing to find 'command' in the PATH [1].

[0] https://lore.kernel.org/buildroot/20211001175329.GA1973888@lbrmn-mmayer.ric.broadcom.net/T/#m95c17eb8374e4e3dd6eee700d397aa12cca0739e
[1] https://lore.kernel.org/buildroot/20211001180304.GV1504958@scaer/T/#m3a8f36bd76ec7d8e5038a6c8932bb6ffe23ea268

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-01 20:09:58 +02:00
Yann E. MORIN
6cddc00399 Revert "support/dependencies: don't check for `which'"
This reverts commit a95c353a46.

Switching to using 'command -v' instead of 'which', opened a can of
worms that is hard to fix in a timely manner:

  - recursive call to 'make' from a post-build, post-iamge script, fails
    because of a redefinition of HOSTCC_NOCCACHE (a bug on its own that
    needs a separate fix anyway) [0];

  - 'make' believeing it can call "simple" commands with execve() et al.
    instead of passing them through a shell via system(), and thus
    failing to find 'command' in the PATH [1].

[0] https://lore.kernel.org/buildroot/20211001175329.GA1973888@lbrmn-mmayer.ric.broadcom.net/T/#m95c17eb8374e4e3dd6eee700d397aa12cca0739e
[1] https://lore.kernel.org/buildroot/20211001180304.GV1504958@scaer/T/#m3a8f36bd76ec7d8e5038a6c8932bb6ffe23ea268

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-01 20:09:55 +02:00
Kory Maincent
b68810e70c boot/grub2: add support to build multiple Grub2 configurations in the same build
When Grub2 is build it is configured only for one boot set-up, BIOS Legacy,
EFI 32 bit or EFI 64 bit. It can not deal with several boot set-up on the
same image.

This patch allows to build Grub2 for different configurations simultaneously.
To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI
BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more
reasonable to just extend the grub2 package into 3 packages.

We can no longer use autotools-package as a consequence of this multi-build, and
we have to resort to generic-package and a partial duplication of
the autotools-infra. Grub2 was already using custom option like --prefix or
--exec-prefix so this won't add much more weirdness.

We use a GRUB2_TUPLES list to describe all the configurations selected.
For each boot case described in the GRUB2_TUPLES list, it configures and
builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.

We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the
BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
  - keep sub-options properly indented
  - fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-09-27 21:36:06 +02:00
Asaf Kahlon
502b8576c4 support/testing: remove python2 tests for automat and attrs
Since commit ef5c8be25a those packages
depend on python3. Hence, we can remove the python2 tests.

Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
2021-09-27 16:34:22 +02:00
Petr Vorel
a95c353a46 support/dependencies: don't check for `which'
It's requirement has been removed in previous commit.

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-26 23:37:21 +02:00
Petr Vorel
ca6a2907c2 make: support: use command -v' instead of which'
`which' has been discontinued after 2.21 release in 2015 due this (git
repository is empty [1]) and version shipped in Debian produces warning
[2]:

/usr/bin/which: this version of `which' is deprecated; use `command -v' in scripts instead.

`command is POSIX [3] and supported on all common shells (bash, zsh,
dash, busybox sh, mksh).

Patch tested on dash as the default shell.

[1] https://git.savannah.gnu.org/cgit/which.git
[2] 3a8dd10b45
[3] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-26 23:37:21 +02:00
Thomas Petazzoni
0132f4748c package/python-git: new package
The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS, as this commit is based on initial work
from Nicolas.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-23 21:59:26 +02:00
Thomas Petazzoni
e6ee07f41a package/python-flask-expects-json: new package
This commit adds a new package called python-flask-expects-json, which
also to validate the JSON blurbs submitted to a Flask web
application. A runtime test is added as well, making sure that the
package minimally works with an example Flask application.

The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS file, as Nicolas is also interested in
this package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-23 21:53:57 +02:00
Thomas Petazzoni
57260dab0c support/testing/tests/package/test_python_flask: new test
This commit adds a test for python-flask package. As we are about to
add python-flask-expects-json together with a test, it made sense to
also add a test for python-flask itself.

As far as the DEVELOPERS file is concerned, the test files are added
both to the existing maintainer of package/python-flask, as well as to
myself.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-23 21:48:22 +02:00
Thomas Petazzoni
8bc8f4c416 package/python-{smmap2, gitdb2}: bump versions to 4.0.0 and 4.0.7
This commit bumps the python-smmap2 and python-gitdb2 packages in
lockstep, as the new version of gitdb2 requires a newer version of
smmap2, but the current version of gitdb2 cannot work with the newer
version of smmap2 (sigh).

Also, upstream the projects have been renamed: gitdb2 is now named
gitdb on PyPi (see https://pypi.org/project/gitdb2/) and smmap2 is now
named smmap (https://pypi.org/project/smmap2/). However, to avoid
needless churn, we don't rename the Buildroot packages, but that
rename is visible in the name of the tarballs being downloaded.

Also, since version 4.0.0, smmap supports only Python 3.x, so we add a
dependency on Python 3.x and drop the test case of gitdb2 on Python
2.x.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-23 21:41:36 +02:00
Romain Naour
71b8322712 support/docker: bump docker image for the gitlab-ci to Debian bullseye
Starting with Qemu 6.1.0, gcc 7.5 is needed to build.
Since we build host-qemu package for qemu defconfig, we have to
upgrade to (at least) Debian buster that provide gcc 8 as host compiler.

While testing this upgrate, the test_edk2 failed since it actually
requires Qemu >= 4.1.0 to support arm SBSA reference machine [1].
Debian Buster only provide Qemu 3.1.

Finally, upgrade to Debian bullseye but it requires some linux
kernel version bump in several defconfigs since host gcc is based
on gcc-10 [2].

[1] https://git.qemu.org/?p=qemu.git;a=commit;h=64580903c2b3aee08d74d64e6248a313b246cb69
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38

Signed-off-by: Romain Naour <romain.naour@gmail.com>
[Arnout: don't install python]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-22 21:58:26 +02:00
Petr Vorel
35f15db30a support: utils: use python3 explicitly
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).

Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import

Tested with python3 -m py_compile.

[1] https://www.python.org/doc/sunset-python-2/

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-22 21:30:24 +02:00
Romain Naour
521b6f8550 support/testing: switch to prebuilt toolchain, core2duo to Nehalem
To avoid spending some time to build the x86_64 toolchain (~20min),
switch to corei7 cpu (Nahalem) and use the prebuilt Bootlin toolchain.

We have to use the "stable" Bootlin toolchain to use the same kernel version
for the toolchain kernel headers and the running kernel.

With the "bleeding-edge" toolchain we have the "kernel too old" issue
(running kernel 4.19 vs kernel headers 5.4)

Runtime tested locally.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:24:31 +02:00
Romain Naour
184d20404e support/testing: test_openssh: use a prebuilt toolchain
Switch from the Buildroot internal toolchain for armv5 to
the prebuilt Bootlin external toolchain.

The test doesn't require to build a toolchain, there was
no prebuilt glibc toolchain recent enough at the time this
test has been introduced.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:23:50 +02:00
Romain Naour
f6d438d59f support/testing: test_rust: remove kernel build
This test already use builtin kernel provided by the testsuite infra:

    self.emulator.boot(arch="armv7",
                       kernel="builtin",
                       options=["-initrd", img])

But a second kernel is build from the its defconfig. This second kernel
is not used by the test.

The TestRust (using BR2_PACKAGE_HOST_RUST=y) is really long to build,
save some cpu time by removing the kernel build.

This unused kernel (based on 4.11.3 release) doesn't even build with
host gcc >= 10.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:23:26 +02:00
Romain Naour
883d5a2f3d support/testing: test_lxc bump to kernel 4.19.204
The kernel 4.19.79 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

The issue was fixed in 4.19.114 [2]
Bump to the latest 4.19.x version.

[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:22:44 +02:00
Romain Naour
23a8baa779 support/testing: test_iso9660: bump to kernel 4.19.204
The kernel 4.11 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

Update to the latest 4.19 LTS version but doing so requires to
disable CONFIG_RETPOLINE since the toolchain is not retpoline
capable [2].

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1564202078

[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8c4ad5d39144776c2987e81609204e1766ed4190

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:22:15 +02:00
Romain Naour
e0ad7c6411 support/testing: test_openrc: switch to ARM arm toolchain and builtin kernel
The kernel 5.5.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

But we can't just update this test to the next linux kernel LTS 5.10.y since
the minimum gcc version has been updated to gcc 4.9 since 5.8 kernel [2]
and the Sourcery CodeBench ARM 2014.05 is used (gcc 4.8 based).

Enable arm cortex A9 and VFP support to switch to the ARM arm prebuilt
toolchain (the Bootlin toolchain could be used).
While at it use the prebuilt buildin kernel for the vexpress target
recently updated to 5.10.7.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1564202094

[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6ec4476ac82512f09c94aff5972654b70f3772b2
[3] 3cf2782906

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:21:00 +02:00
Romain Naour
73278c8a70 support/testing: test_glxinfo: bump to kernel 4.19.204
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

Bump to the next LTS release.

[1] https://gcc.gnu.org/gcc-10/porting_to.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:20:15 +02:00
Romain Naour
e0a64dfc0e support/testing: test_f2fs: bump to kernel 4.19.204
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

Bump to the next LTS release.

[1] https://gcc.gnu.org/gcc-10/porting_to.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:20:04 +02:00
Romain Naour
69de111c46 support/testing: test_systemd: bump to kernel 4.19.204
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

Bump to the next LTS release.

[1] https://gcc.gnu.org/gcc-10/porting_to.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:19:53 +02:00
Romain Naour
322b40405e support/testing: test_file_capabilities: bump to kernel 4.19.204
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].

/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here

Bump to the next LTS release.

[1] https://gcc.gnu.org/gcc-10/porting_to.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-21 22:19:04 +02:00
Matthew Weber via buildroot
5ec9542415 support/testing/tests/fs: test OCI format
This patch adds runtime testing of the OCI archive created by the
sloci scripting. It launches a containerd instance, imports, and
runs the OCI container.

The existing QEMU AARCH64 kernel config was extended to enable common
options used by a container runtime (cgroup and overlayfs).

Signed-off-by: Matthew Weber <matthew.weber@collins.com>
[Arnout: adapt file name which is arm64 now; add to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-11 14:52:41 +02:00
Yann E. MORIN
4ad23552c6 package/e2fsprogs: bump version to fix regression
Version 1.46.3 had a regression, which meant the file that would store
the filesystem image had to pre-exist, or mkfs.ext2 would fail to
generate the filesystem:

    mkfs.ext4: No such file or directory while trying to determine
    filesystem size

The regression was fixed upstream, and is now part of the 1.46.4
release, so bump to that release.

Fixes: #14196

Additionally, as noticed by Romain, the defaults settings for generating
"small" filesystems have changed: the inode size has been increased fom
128 to 256 bytes in 1.46.4 [0]. This causes the number of inodes to
diverge slightly from the requested number; instead of 8 more inodes,
there are now 8 fewer than requested.

Adapt our test accordingly.

[0] a23b50cdb5

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reported-by: Kevin Tomary <kevin.tomary@hotmail.com>
Reported-by: Leon de Rooij <leon@exquisip.nl>
Reported-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Michael Walle <michael@walle.cc>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-09-10 18:22:09 +02:00
Peter Korsgaard
72e1a75820 Merge branch 'next' 2021-09-04 19:57:30 +02:00
Peter Korsgaard
aa433d1c5c Update for 2021.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-09-04 11:53:24 +02:00
Thomas Petazzoni
c938698b4a support/testing/tests: add test for check_bin_arch
This tests build a bogus package that installs a binary built for the
host architecture into $(TARGET_DIR), which should cause a build
failure, at least as long as the host architecture isn't ARM.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
[yann.morin.1998@free.fr: drop uneeded subprocess import to fix flake8]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-08-29 11:37:32 +02:00
Thomas Petazzoni
1259d40b0a support/testing/infra: add log_file_path() function
Some tests will need to grep through the build log to verify that some
features are working are expected. In order to allow them to open the
build log, we provide a new function called log_file_path(), which
returns the path to the log file if available.

We also use this function in open_log_file().

Note that open_log_file() cannot be used directly to grep through the
log file at the end of a build: because it opens in "a+" mode, it
greps starting from the end of the file.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-08-29 11:37:15 +02:00
Romain Naour
12c7a05da1 utils/test-pkg: add gitlab-ci support
The gitlab-ci support in test-pkg allows to parallelize the test-pkg
work into several gitlab jobs. It's much faster than local serialized
testing.

To trigger this, a developer will have to add, in the latest commit of
their branch, a token on its own line, followed by a configuration
fragment, e.g.:

    test-pkg config:
    SOME_OPTION=y
    # OTHER_OPTION is not set
    SOME_VARIABLE="some value"

This configuration fragment is used as input to test-pkg.

To be able to generate one job per test to run, we need the list of
tests in the parent pipeline, and the individual .config files (one per
test) in the child pipeline. We use the newly-introduced --prepare-only
mode to test-pkg, and collect all the generated .config files as
artefacts; those are inherited in the child pipeline via the
"needs::pipeline" and "needs::job" directives. This is a bit tricky,
and is best described by the Gitlab-CI documentation [0].

We also list those .config files to generate the actual list of jobs to
run in the child pipeline.

Notes:
  - if the user provides an empty fragment, this is considered an error:
    indeed, without a fragment (and the package name), there is no way
    to know what to test;
  - if that fragment yields an empty list of tests, then there is
    nothing to test either, so that is also considered an error.

[0] https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
  - split the change to test-pkg to its own patch
  - generate the actual yml snippet in support/scripts/generate-gitlab-ci-yml,
    listing the .config files created by test-pkg
  - some code-style-candies...
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-28 17:03:13 +02:00
Romain Naour
e5494f1fac support/testing: test_atf: bump the custom version to v2.5
This version bump is needed to pass the ATF test with
hardening option enabled (-fstack-protector-strong)

With the version v2.2, ATF fail due to undefined references:

./build/juno/release/bl2u/arm_tzc400.o: In function `arm_tzc400_setup':
arm_tzc400.c:(.text.arm_tzc400_setup+0x10): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0x18): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xb8): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xcc): undefined reference to `__stack_chk_fail'

Since commit ccac9a5bbb, Buildroot no
longer forces ENABLE_STACK_PROTECTOR. However, we rely on the ATF build
system to handle it correctly, and this wasn't the case in v2.2.

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1524842591

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-28 14:24:25 +02:00
Arnout Vandecappelle (Essensium/Mind)
7dac56b76e Merge branch 'master' into next
Conflicts:
	package/go/go.hash
	package/go/go.mk

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-26 23:40:57 +02:00
Romain Naour
5d60e07e27 support/testing: test_docker_compose: bump the kernel to 4.19.204
gcc 10.x is now used by default but the kernel 4.19 used by
test_docker_compose doesn't build with it.

Bump the kernel to 4.19.204 release that contains a lot of
fixes for newer gcc.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-24 23:59:09 +02:00
Romain Naour
f4ff135e78 support/testing: revert the last change of check_network()
check_network() must check the error code of the command
used to check the network configuration with the value
passed as argument "exitCode".

But this argument is ignored since this commit [1].

Revert the last change of check_network().

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1522848308
https://gitlab.com/kubu93/buildroot/-/jobs/1522848306

[1] afc1ed4d51

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-24 23:54:53 +02:00
Romain Naour
26f6110740 support/testing: remove TestPythonPy2{Cryptography, ServiceIdentity, Treq, Twisted, Txtorcon}
The python2 support has been removed since the python-idna bump to version 3.2 [1]

[1] 0c7e30b43a

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-24 00:26:05 +02:00
Peter Korsgaard
5284a826ae Update for 2021.05.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d5127a4de7)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-08-10 09:48:00 +02:00
Romain Naour
208e98beb2 support/misc/gitlab-ci.yml.in: templatise the make command
In a followup commit, the make command used to log and display the last
lines on error will be used in another job.
Factorize it by introducing .run_make template.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-05 22:30:50 +02:00
Raphaël Mélotte
58c99c957b package/python-rsa: new package
Python-RSA is a pure-Python RSA implementation.

Signed-off-by: Raphaël Mélotte <raphael.melotte@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-04 22:27:28 +02:00
Gleb Mazovetskiy
6fbdf51596 Makefile: Parallelize glibc locale generation
Parallelizes locale generation based on `BR2_JLEVEL` setting.

Locale generation always runs during the finalize stage and can consume
a significant amount of time. Parallelizing it greatly reduces that time
on multi-core machines.

To parallelize it, we first invoke `localedef` for every locale in
parallel with the `--no-archive` option. This creates the intermediate
locale data instead of writing to the finally archive directly.

Then, we invoke `localedef` again once to create the archive from the
intermediate compiled locale data files.

We have to do it this way because `localedef` does not do any locking
when writing to the archive file, so calling it without `--no-archive`
concurrently could result in a corrupt archive file or an archive file
that is missing some locales.

While we're at it, make two additional improvements:
- Remove locale-archive before adding to it. Otherwise, repeated
  applications of target-finalize will keep on growing the file.
- Sort the locales when creating locale-archive so its contents are
  reproducible.

We use `find` to collect the installed locales rather than LOCALES. This
makes it possible for something else (skeleton, overlay, custom package)
to create and install additional locales and still have them added to
locale-archive.

Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
[Arnout:
 - Remove -j$(PARALLEL_JOBS), it's already part of $(MAKE)
 - Remove HOST_DIR, TARGET_DIR, STAGING_DIR, they're already exported
 - Extend commit message
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-07-25 16:46:30 +02:00
Adam Duskett
db1ded1084 support/testing: add polkit tests
This test script tests polkit with and without systemd.

The Systemd test does the following:
  - The brtest user attempts to restart the systemd-timesyncd service and is
    denied.

  - A systemd-timesyncd-restart.rules file provided by polkit-rules-test
    is copied from /root/ to /etc/polkit-1/rules.d

  - The brtest user attempts to restart the systemd-timesyncd service and should
    now succeed.

The initd test does the following:
- The brtest user attempts to run the test application "hello-polkit" with the
  command "pkexec hello-polkit" and is denied.

- A hello-polkit.rules file provided by polkit-rules-test is copied from /root/
  to /etc/polkit-1/rules.d

- The brtest user attempts to re-run the test hello-polkit binary with
  "pkexec hello-polkit" and succeeds.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-07-24 23:29:21 +02:00
Yann E. MORIN
d0edfec1e2 boot/uboot: add missing dependency to host-pkgconf
Commit 2eaa6d0f36 (boot/uboot: fix uboot building host tools on x86
architecture) added use of $(PKG_CONFIG_HOST_BINARY), but forgot to add
the corresponding build-ordr dependency.

Add this missing depenency now.

Additionally, the associated test had an explicit host pkgconf enbled in
its configuration. This is superfluous now that uboot properly depends
on host-pkgconf, so drop that from the test.

Note: it hapenned to work, because host-pkgconf, when explicitly enabled
in the configuration, and without per-package directories, would build
before uboot and thus be available. This would fail with PPD, though,
and thus would break for TLPB.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-07-17 13:18:04 +02:00
Francois Perrad
d3e2e7e9a6 package/lua-lunix: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-07-13 22:19:07 +02:00
Kory Maincent
2eaa6d0f36 boot/uboot: fix uboot building host tools on x86 architecture
The make all command run the tools/makefile on the process.
This makefile use "pkg-config" command to support static link.
The issue is the use of pkg-config configured for crosscompiling
to build binaries tools for host architecture.
To fix it, I add pkg-config environment variable to configure it for host.

Add a test to avoid future regress on the build of U-boot.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
  - fix mixed space-TAB indentation
  - fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-30 22:45:18 +02:00
Yann E. MORIN
e7f0408f49 support/tests: fix squashfs test
Commit 3cf2782906 (support/testing/infra/emulator.py: update pre-built
kernels) bumped the default kernels used by the testing infra.

However, the newer armv7 kernel (at least) no longer has support for
lz4-compressed squashfs filesystems.

This breaks the squashfs test:

    Filesystem uses "lz4" compression. This is not supported
    List of all partitions:
    1f00          131072 mtdblock0
     (driver?)
    1f01           32768 mtdblock1
     (driver?)
    b300            2048 mmcblk0
     driver: mmcblk
    No filesystem could mount root, tried:
     squashfs

    Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,0)

Updating the kernel again is a little bit cumbersome, while fixing the
actual test is relatively trivial, so this is what we do: we switch
over to lzo, which is supported by the new kernel:

    # zcat /proc/config.gz |grep SQUA
    CONFIG_SQUASHFS=y
    CONFIG_SQUASHFS_ZLIB=y
    # CONFIG_SQUASHFS_LZ4 is not set
    CONFIG_SQUASHFS_LZO=y
    # CONFIG_SQUASHFS_XZ is not set

While at it, also drop the superfluous line disabling gzip compression:
it is part of a choice, so enabling one (lzo here) forcibly disables the
others (of which gzip).

Fixes: 3cf2782906

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-27 11:50:01 +02:00
Thomas Petazzoni
f37e811cd1 support/testing/tests/fs/test_ext: add missing "format" option for Qemu
Will avoid the following warning:

   WARNING: Image format was not specified for
   '/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and
   probing guessed raw. Automatically detecting the format is
   dangerous for raw images, write operations on block 0 will be
   restricted. Specify the 'raw' format explicitly to remove the
   restrictions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 22:17:07 +02:00
Thomas Petazzoni
abe32cfdf0 support/testing/tests: fix tests to use infra.img_round_power2()
All the tests that are using if=sd as a Qemu options are changed to
use infra.img_round_power2() instead of simply extending the size of
the image to the next MB boundary, which is not longer sufficient with
Qemu >= 5.1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: drop now-useless imports]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 22:16:40 +02:00
Thomas Petazzoni
37a1af7a74 support/testing/infra: add img_round_power2() function
Since Qemu 5.1, SD card images must have a size that are a power of
two. While some filesystem (such as ext2/3/4) allow to specify the
expected size of the filesystem, others such as SquashFS do not have
this capability.

We were already extending the size of such images to the next 1 MB
boundary using "truncate -s %1M", but that is no longer sufficient. So
instead, we introduce a helper function that extends the size of an
image to the next power of two.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - use f.trunctate() rather than subprocess.call([truncate,...])
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 21:27:48 +02:00
Thomas Petazzoni
78c42cdca2 support/testing/tests/fs/test_ext: specify 16 MB as ext filesystem size
Since Qemu 5.1, the SD card size must be a power of two, so the
default size for ext2/3/4 filesystem images of 60 MB is not
suitable. Since 16 MB is used for the Ext4 test, let's use the same
value for the other tests as well (ext2, ext2r1, ext3). Without this
change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >=
5.1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 21:26:16 +02:00
Thomas Petazzoni
afc1ed4d51 support/testing: use .assertRunOk() when possible
The BRTest() class implements an assertRunOk() method that does the
very common work of running a command inside the emulator, and
checking that it is successful.

This commit changes all locations where this .assertRunOk() method can
be used, instead of open-coding the same logic.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 17:41:10 +02:00