Commit Graph

1755 Commits

Author SHA1 Message Date
Sebastian Weyer
24b2ef0c67 support/testing: test_rust.py: Add test to check correct vendoring
Currently the tests TestRust and TestRustBin do check for vendoring by
requiring package ripgrep to be built but only if the download
directory is already empty, otherwise the existing contents of the download
directory will be reused and therefore not be redownloaded.

This new test will only verify that the required packages are downloaded
and vendored correctly without doing a runtime test. It does so by setting a
path to a folder "dl" inside the build directory (output-directory/testname/)
and then setting the environment variable BR2_DL_DIR to this path before the
build starts. BR2_DL_DIR is not set in the config options because it would be
overridden by the user's own environment variable if defined. This code was
essentially copied from the file test_gitforge.py which was added in commit
1ca6ab6ace

We want the package ripgrep to be built since it requires vendoring
directly. Additionally we want the package python-cryptography to be
built because it has rust dependencies and therefore indirectly also requires
vendoring.

Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr>
Signed-off-by: Antoine Coutant <antoine.coutant@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-01-11 12:13:58 +01:00
Yann E. MORIN
9fbd3d8574 Revert "support/download: generate even more reproducible tarballs"
Commit 768f9f80f6 (support/download: generate even more reproducible
tarballs) causes non-reproducibility in tarballs we previousy
generated, especially the archives for two cargo-vendored packages,
ripgrep and sentry-cli.

The cause is that those two pakcages eventually vendor a file that has
the u+x bit set, but is otehrwise go-x. With 768f9f80f6, the files are
now go+x, so the hash for those generated archives has changed.

Besides, that commit was wrong: it did not account for the 'r' bit for
go part, leaving some non-reproducibility still unaccounted for.

So, to generate really reproducible archives, we would need to fix that
read bit as well, and that has the potential to affect all the archives
we generated so far. If we wanted to do so, we'd need a way to version
all generated archives, like we do for git and svn, but now for all the
different CVSes, as well as for all the vendoring post-processes.

For 768f9f80f6, all that was of conern was the working copies of CVSes
(i.e. git, svn, cvs...) that we cache in the Buildroot download dir, not
the temporary files during post-processing. Indeed, in that latter case,
the user has virtually no way to mangle with the mode of the
intermediate extract before repack.

And we do have a big fat warning that users should not attempt to meddle
with the git tree that Buildroot caches.

As 768f9f80f6 however demonstrates, is that it took quite a long time
between the introduction of the git caching, and the time someone
eventually discovered they could meddle in there. This shows that the
issue it not actually critical in most setups.

Also, the tar manual [0] hints at a better solution to handle
reproducibility, which even avoids touching the files on disk which is
even nicer:

    ‘--mode='go+u,go-w'’
        Omit irrelevant information about file permissions.

If we were to actually handle the mode bit for reproducibility, we'd
need to:
  - introduce archive versioning for all download backends and
    prost-processing
  - use the tar officially suggested method

So, revert that change, as it was incomplete, was not really fixing much
issues, and causes actual issues.

This reverts commit 768f9f80f6.

[0] https://www.gnu.org/software/tar/manual/tar.html#Reproducibility

Thanks to Vincent and Arnout for pointing at the tar manual.

Reported-by: Antoine Coutant <antoine.coutant@smile.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Antoine Coutant <antoine.coutant@smile.fr>
2024-01-11 12:00:25 +01:00
Fabrice Fontaine
7ea38660e0 support/dependencies/dependencies.sh: libopenssl needs perl Math::BigInt on s390
libopenssl needs perl Math::BigInt for s390x asm to avoid the following
build failure since commit a5cacb6308:

Can't locate bigint.pm in @INC (you may need to install the bigint module) (@INC contains: /home/buildroot/autobuild/instance-0/output-1/build/libopenssl-3.2.0/crypto/poly1305/asm/../.. /home/buildroot/autobuild/instance-0/output-1/host/lib/perl /usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at /home/buildroot/autobuild/instance-0/output-1/build/libopenssl-3.2.0/crypto/poly1305/asm/../../perlasm/s390x.pm line 16.
BEGIN failed--compilation aborted at /home/buildroot/autobuild/instance-0/output-1/build/libopenssl-3.2.0/crypto/poly1305/asm/../../perlasm/s390x.pm line 16.

Fixes:
 - http://autobuild.buildroot.org/results/986cb07d368c7214ffbc9d60c378e7ac00797f00

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-01-10 21:22:57 +01:00
Julien Olivain
5cd74d6a4e support/testing: test_python_ml_dtypes: remove BR2_PACKAGE_PYTHON3_ZLIB
The test_python_ml_dtypes.py enabled BR2_PACKAGE_PYTHON3_ZLIB=y in its
configuration to workaround the fact that the toolchain used to
testing was tainted with zlib.

The commit 7e0e6e3
"toolchain/toolchain-external/toolchain-external-bootlin: update to
2023.11-1" updated the toolchains which are no longer tainted with
zlib.

The workaround is no longer needed and this commit removes this
config, for consistency.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-29 21:49:28 +01:00
Thomas Petazzoni
7e0e6e3b86 toolchain/toolchain-external/toolchain-external-bootlin: update to 2023.11-1
New toolchains have been released, with the following changes:

- The bleeding-edge toolchains are based on gcc 13.2, binutils 2.41,
  gdb 14.1, kernel headers 5.10, glibc 2.38, musl 1.2.4 or uclibc-ng
  1.0.45.

- The stable toolchains are based on gcc 12.3, binutils 2.40, gdb
  13.2, kernel headers 4.14, glibc 2.38, musl 1.2.4 or uclibc-ng
  1.0.45.

- The glibc version is no longer affected by CVE-2023-4911

- The gdb build has been fixed to no longer rely on uninstalled
  libbfd.so and libopcodes.so libraries

- The zlib library, which was incorrectly present in the toolchain
  sysroot, is gone, fixing various build failures encountered with
  2023.08 toolchains.

- There are now toolchains for m68k 68xxx based on uclibc and musl in
  addition to glibc, which was already supported

The careful reviewer will notice that a number of

    depends on !BR2_ARCH_NEEDS_GCC_AT_LEAST_14

are being added to the toolchains that use gcc 13.x, as per
a0d2a5cfec
("support/scripts/gen-bootlin-toolchains: generate
BR2_ARCH_NEEDS_GCC_AT_LEAST_X guard").

All 214 test cases were successfully run:

  https://gitlab.com/tpetazzoni/buildroot/-/pipelines/1120323562

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-28 21:57:50 +01:00
Julien Olivain
ebe47e7605 package/python-ml-dtypes: new package
ml_dtypes is a stand-alone implementation of several NumPy
dtype extensions used in machine learning libraries.

https://github.com/jax-ml/ml_dtypes

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-12-25 11:20:24 +01:00
Colin Foster
4d025f88e6 support/testing: add python-tftpy runtime test
This runtime test verifies the existence of the tftpy module when
selected.

Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-12-23 15:05:01 +01:00
Romain Naour
0c0cd720c5 support/testing: TestDtbocfg: bump kernel to 5.10.202
The prebuilt kernel has been updated to 5.10.202, sync the kernel
built by TestDtbocfg.

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-09 21:35:37 +01:00
Romain Naour
f6254689f8 support/testing: InitSystemSystemdBaseOverlayfs: bump kernel to 5.10.202
The prebuilt kernel has been updated to 5.10.202, sync the kernel
built by InitSystemSystemdBaseOverlayfs.

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-09 21:35:32 +01:00
Romain Naour
241a8b0615 support/testing: bump prebuilt kernel to 5.10.202
As reported on the mailing list [1], TestPolkitSystemd and
TestPolkitInitd are failing since we bumped the Bootlin toolchain
2023.08 [2].

The issue is caused by expat (XML library) package detecting
arc4random_buf() introduced by glibc 2.36 [3].

With arc4random_buf() support enabled, expat hang Polkit while reading
its policy files (XML files) due to a lack of entropy on the system.

Upgrading the kernel allows to avoid such issue thanks to
random_get_entropy_fallback() introduced in 5.10.119 (backpored) [4].

Build the vexpress-v2p-ca9 (armv7) 5.10.202 using the kernel
configuration file found in /proc/config.gz but with additional kernel
options provided by SYSTEMD_LINUX_CONFIG_FIXUPS. Indeed some kernel
options requested by systemd were missing in 5.10.7 kernel.

Build the versatile-pb (armv5) kernel using the same kernel
configuration as qemu_arm_versatile_defconfig but with additional
kernel options like for vexpress-v2p-ca9. While at it, enable
IKCONFIG_PROC option to provide the kernel configuration at runtime
in /proc/config.gz.

Runtime tested:
https://gitlab.com/kubu93/buildroot/-/pipelines/1097887826 (tests.package.test_polkit.TestPolkit*)
https://gitlab.com/kubu93/buildroot/-/pipelines/1097705399 (tests.init.*)

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/5689309948 (TestPolkitSystemd)
https://gitlab.com/buildroot.org/buildroot/-/jobs/5689309947 (TestPolkitInitd)

[1] http://lists.busybox.net/pipermail/buildroot/2023-December/680445.html
[2] 452365a385
[3] https://sourceware.org/git/?p=glibc.git;a=commit;h=6f4e0fcfa2d2b0915816a3a3a1d48b4763a7dee2
[4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=fdca775081527364621857957655207d83035376

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-09 21:35:28 +01:00
Peter Korsgaard
0c6f3d7f2e Merge branch 'next' 2023-12-04 21:11:12 +01:00
Peter Korsgaard
954aeb7c72 Update for 2023.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-12-04 09:08:32 +01:00
Romain Naour
3503d48897 support/misc/gitlab-ci.yml.in: print error log if the runtime test fail
When an error occurs, the gitlab-ci job log doesn't contain any useful
information than the name of the failing test:

  FAIL: test_run (tests.package.test_python_paho_mqtt.TestPythonPahoMQTT)

In order to encourage contributors to investigate issues reported by
gitlab-ci, we want to print the last lines of the log file (build or
runtime).

Unfortunately, gitlab-ci job log completely strips lines ending with
CRCRLF [1][2]. We have to take a look at the gitlab-ci raw log to see
the complete log [3].

To workaround this issue, remove crlf from qemu serial stdio log
while printing in the gitlab-ci job log (we don't want to change
the log file generated by support/testing/run-tests and saved as
artefacts).

[1] https://gitlab.com/gitlab-org/gitlab/-/issues/218771
[2] https://gitlab.com/kubu93/buildroot/-/jobs/5492937691
[3] https://gitlab.com/kubu93/buildroot/-/jobs/5492937691/raw

Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-12-03 20:20:53 +01:00
José Luis Salvador Rufo
a44d1a1252 package/zfs: bump version to 2.2.0
Removed backported patch:
- bc3f12bfac.patch

Updated ZFS test to pass this new version; drop the explicit /pool
mountpoint option to rely on the default location (which happens to be
/pool already).

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
[yann.morin.1998@free.fr:
  - needed on master to further bump to a data-corruption fix
]
(cherry picked from commit d153e58d13)
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-12-01 11:29:53 +01:00
José Luis Salvador Rufo
d153e58d13 package/zfs: bump version to 2.2.0
Removed backported patch:
- bc3f12bfac.patch

Updated ZFS test to pass this new version; drop the explicit /pool
mountpoint option to rely on the default location (which happens to be
/pool already).

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-11-18 15:51:39 +01:00
Peter Korsgaard
41978f360a Update for 2023.08.3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 8ad64e724c)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-11-14 19:15:57 +01:00
Peter Korsgaard
f1ee7015a4 support/dependencies/check-host-tar.sh: blacklist tar 1.35+
GNU tar 1.35 changed the behaviour for the devmajor/devminor fields,
breaking the download hash validation.  For details, see:

https://lists.gnu.org/archive/html/info-gnu/2023-07/msg00005.html
https://patchwork.ozlabs.org/project/buildroot/patch/20231018141155.533944-1-vfazio@gmail.com/

To work around this issue, blacklist tar 1.35+ similar to how we do it for
pre-1.27 versions so Buildroot falls back to building host-tar (which is
currently 1.34).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-11-13 22:51:20 +01:00
Arnout Vandecappelle
3b65462631 support/testing: add missing systemd-overlay-fstab
Commit 083e65a67c introduced tests for the
various read-only root options under systemd, but while applying the
fs-overlay that is used in one of the tests wasn't included. Include it
now.

Fixes: 083e65a67c
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-11-13 12:17:43 +01:00
Waldemar Brodkorb
14ae33c669 support/testing/tests/package/test_ruby.py: new runtime test
Signed-off-by: Waldemar Brodkorb <wbx@openadk.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-08 23:18:52 +01:00
Daniel Lang
c3c5320ce2 support/scripts/pkg-stats: check all files for warnings
Instead of only checking .mk and Config.in{,.host}, check
all files in a package directory.
.checkpackageignore isn't considered here, therefore the shown number
includes ignored warnings as well.
Add another css class to signal some warning, compared to a lot (>5),
similar to patches.

Signed-off-by: Daniel Lang <dalang@gmx.at>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-08 22:14:27 +01:00
Daniel Lang
1aa198f728 support/scripts/pkg-stats: ignore more makefiles that aren't packages
Some packages are grouped and have a general makefile that defines
reusable variables. These makefiles have no relevant information for
pkg-stats and should be excluded.

Signed-off-by: Daniel Lang <dalang@gmx.at>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-08 21:59:50 +01:00
Yann E. MORIN
f91e89b6e6 support/download: teach dl-wrapper to handle more than one hash file
Currently, we expect and only use hash files that lie within the package
directory, alongside the .mk file. Those hash files are thus bundled
with Buildroot.

This implies that only what's known to Buildroot can ever get into those
hash files. For packages where the version is fixed (or a static
choice), then we can carry hashes for those known versions.

However, we do have a few packages for which the version is a free-form
entry, where the user can provide a custom location and/or version. like
a custom VCS tree and revision, or a custom tarball URL. This means that
Buildroot has no way to be able to cary hashes for such custom versions.

This means that there is no integrity check that what was downloaded is
what was expected. For a sha1 in a git tree, this is a minor issue,
because the sha1 by itself is already a hash of the expected content.
But for custom tarballs URLs, or for a tag in a VCS, there is indeed no
integrity check.

Buildroot can't provide such hashes, but interested users may want to
provide those, and currently there is no (easy) way to do so.

So, we need our download helpers to be able to accept more than one hash
file to lookup for hashes.

Extend the dl-wrapper and the check-hash helpers thusly, and update the
legal-info accordingly.

Note that, to be able to pass more than one hash file, we also need to
re-order the arguments passed to support/download/check-hash, which also
impies some shuffling in the three places it is called:
  - 2 in dl-wrapper
  - 1 in the legal-info infra

That in turn also requires that the legal-license-file macro args get
re-ordered to have the hash file last; we take the opportunity to also
move the HOST/TARGET arg to be first, like in the other legal-info
macros.

Reported-by: "Martin Zeiser (mzeiser)" <mzeiser@cisco.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-11-07 11:48:45 +01:00
Adam Duskett
49db37ae5a support/testing/tests/package/test_firewalld.py: new test
This test case runs firewalld using both system and sysvinit.

run `firewalld-cmd --state` and ensure the output is "running" with a return
code of 0.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-06 20:43:55 +01:00
Adam Duskett
9a81c7a7a4 support/testing/tests/package/test_python_django.py: fix timeout calculation.
timeout = 35 * self.emulator.timeout_multiplier
[...]
        self.assertRunOk(cmd, timeout=timeout)

Gets re-multiplied by self.emulator.timeout_multiplier in self.emulator.run().
Drop multiplying the timeout by self.emulator.timeout_multiplier to fix this
issue.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-06 20:43:47 +01:00
Adam Duskett
4889ac89b8 support/testing/tests/package/test_lxc.py: use timeout argument for emulator.login
self.emulator.timeout_multiplier *= 10 is equivilent to 60 * 10 or 600.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-06 20:43:41 +01:00
Adam Duskett
c9316f21c7 support/testing/test/init/test_systemd_selinux.py: use timeout argument for emulator.login
self.emulator.timeout_multiplier *= 10 is equivilent to 60 * 10 or 600.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-06 20:43:31 +01:00
Adam Duskett
b1d554b9a7 support/testing/infra/emulator.py: add a timeout argument for the login method
to override the current value of 60 seconds

As per a suggestion by Thomas, add a timeout argument to override the current
value of 60 seconds for the emulator.login method.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-06 20:42:17 +01:00
Adam Duskett
c1d5eada4d package/nodejs: bump version to 20.9.0
- Refactor 0001-add-qemu-wrapper-support.patch for 20.9.0

- NodeJS now requires GCC 10.1 header. However, as there is no
  BR2_TOOLCHAIN_GCC_AT_LEAST_10_1, we round up to BR2_TOOLCHAIN_GCC_AT_LEAST_11.

- Drop the --without-dtrace and --without-etw config options as they no longer
  exist.

- Update support/testing/tests/package/test_nodejs.py to use
  BR2_TOOLCHAIN_EXTERNAL_BOOTLIN, otherwise the test fails as the
  gcc version selected will be 7.

- Update the hash file for LICENSE.txt due to numerous changes, such
  as the addition of new libraries and updated dates. No new license
  types are added.

Tested with: ./support/testing/run-tests tests.package.test_nodejs

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-03 20:59:28 +01:00
Adam Duskett
3204b01f53 package/openrc: add split-user support
Also, introduce a new test in support/testing/tests/init/test_openrc.py that
ensures split-user support works properly.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-01 19:12:22 +01:00
Adam Duskett
095cab35a9 package/openjdk{-bin}: drop OpenJDK11 and add OpenJDK21
OpenJDK 21 is out and with it, OpenJDK11 is now EOL.
See: https://endoflife.date/oracle-jdk As such, drop support for 11 and do the
following:

  - The 0001-Add-ARCv2-ISA-processors-support-to-Zero.patch patch now applies to
    both 17 and 21. Move it out of the version-specific directoriy.

  - BR2_OPENJDK_VERSION_LTS is now set to 17.

  - BR2_OPENJDK_VERSION_LATEST is now set to 21.

  - Drop --disable-hotspot-gtest as it has been removed, and was ignored in 17.

  - Add two separate HOST_OPENJDK_BIN_VERSION defines in openjdk-bin.mk as
    there is not a point release yet for OpenJDK 21.

  - Update the expectedVersion variable in JniTest.java from 0x000A0000 to
    0x00150000

Tested with:
./support/testing/run-tests tests.package.test_openjdk.TestOpenJdk.test_run

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-11-01 19:00:50 +01:00
Peter Korsgaard
cc31a8057a Update for 2023.08.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit fe50c054bc)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-10-16 11:34:39 +02:00
Yann E. MORIN
083e65a67c support/testing: add systemd tests for factory and overlayfs
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
[Arnout:
 - use a simple mount unit for
   TestInitSystemSystemdRoFullOverlayfsVarBacking;
 - change the test of TestInitSystemSystemdRoFullOverlayfsVarBacking to
   check that the exact expected mount was performed;
 - add a test of var backing with fstab instead of mount unit.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-10-08 20:15:13 +02:00
Thomas Petazzoni
3a1b68955c support/testing/tests/package/test_nodejs: test both host-nodejs-bin and host-nodejs-src
The TestNodeJSModule test triggers the build of host-nodejs to be able
to install third party modules. Now that host-nodejs has two
providers, it makes sense to test both cases, so we duplicate
TestNodeJSModule into TestNodeJsModuleHostBin (which tests the
host-nodejs-bin) and TestNodeJSModuleHostSrc (which tests the
host-nodejs-src).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-10-06 17:31:15 +02:00
Romain Naour
f9f6da877a support/testing: TestPythonPy3MakoExt: new test for mako external plugins
Mako provide some external plugins that requires additionnal and
optional runtime dependencies, make sure we test these situations.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-30 18:48:26 +02:00
Romain Naour
164801997e support/testing: add python-mako runtime test
This new runtime test allows to make sure that the python-mako package
minimally works at runtime.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-30 18:48:24 +02:00
Yann E. MORIN
0e65515862 support/runtime-test: fix weston test
The weston runtime test uses the CRC of the framebuffer to detect that
"something" is being drawned on the framebuffer. This requires that the
sampling of the CRC happens does not happen too early after trigerring
an action, or the rendering may be not be finishe, either:

  - weston may not have had time to initialise, or
  - the test application may not have started rednering,

The sequence of rendering that has been observed yields this sequence of
CRCs (elided for brevity):

  - boot:
    - alternating between 0x4c4126bf and 0x5d2f9aa5: console cursor
      blinking

  - start weston:
    - 0x4c4126bf: weston switches to a cleared vt, no blinking cursor
      ...
    - 0xe54b7895: weston is starting
      ...
    - 0xe54b7895: wayland socket appears!
      ...
    - 0x6bf28bdf: weston is ready
      ...

  - start weston-simple-egl:
    - 0x6bf28bdf: application is starting
      ...
    - 0xNNNNNNNN: random CRCs while the application renders
      ...

  - stop weston-simple-egl:
    - 0xNNNNNNNN: zero, one, or two random CRCs while the application
        renders before it handles SIGTERM
    - 0x6bf28bdf: application is stopped
      ...

  - stop weston:
    - 0x6bf28bdf: a few CRC identical to when weston was started, while
        weston is processing SIGTERM
    - oscillating between 0x4c4126bf and 0x5d2f9aa5: console cursor
        blinking, back to initial vt, weston dead.

So, we need to wait "enough" after each action. Moreover, when the
wayland socket appears, weston may not have stabilised yet, so we also
need to wait after the socket appears.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Julien Olivain <ju.o@free.fr>
2023-09-30 10:59:56 +02:00
Adam Duskett
7f0af11cee support/testing/tests/package/test_flutter.py: new runtime test
This is a simple test that builds and runs the futter-gallery application and
checks if the service is active.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
[yann.morin.1998@free.fr: fix flake8 warnings]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-09-30 00:12:07 +02:00
Peter Korsgaard
55fbb5519c Update for 2023.08.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 9266ab06e0)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-09-28 00:22:36 +02:00
Julien Olivain
2031fc8809 support/testing/tests/package/test_tcl.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
[Arnout: use f-string instead of string.format]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-09-17 22:09:40 +02:00
Yann E. MORIN
bcee3ca6d6 support/download/git: fix shellcheck errors
The quoting around the expansion of ${relative_dir} was indeed incorrect
since it was introduced back in 8fe9894f65 (suport/download: fix git
wrapper with submodules on older git versions): it is in fact already
quoted as part of the whole sed expression.

${GIT} can contain more than one item, but we don't care about splitting
on spaces when we just print it for debug, so we can just quote it
rather than add an exception.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-14 23:02:20 +02:00
Yann E. MORIN
daa341cb9b support/download/git: properly catch failures
Since commit b7efb43e86 (download/git: try to recover from
utterly-broken repositories), we catch errors through an ERR
trap, so we can try and recover from a broken repository. In
that commit, we switched from using "set -e" to "set -E", so
that trap is inherited in functions, command substitutions,
and subshells.

However, the trap is not defined until we have parsed the
options, created the cache directory, and eventually chdir()ed
into it. Athough improbable, it is possible for the git helper
to fail in any of those steps, and that would not get caught.

Fix that

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-14 23:02:13 +02:00
Yann E. MORIN
768f9f80f6 support/download: generate even more reproducible tarballs
When we generate the taballs off a local working copy of a VCS tree,
the umask is the one that we enforce in out top-level Makefile.

However, it is possible that a user manually tinkers in said working
copy (e.g. to check an upstream bug fix, or regression). If the user
umask is different from the one Buildroot enfirces, such tinkering
can impact the mode bits of the files, even if their content is not
modified.

When we eventually need to create a tarball from said working copy,
the VCS (e.g. git) will only be interested in checking whether the
content of the files have changed before chcking them out, and will
not look at, and restore/fix the mode bits.

As a consequence, we may create non-reproducible archives.

We fix that by enforcing the mode bits on the files before we create
the tarball: we disable the write and execute bits, and only set the
execute bit if the user execute bit is set.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-14 23:02:07 +02:00
Julien Olivain
02497626ad support/testing/tests/package/test_libgpgme.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-10 18:34:52 +02:00
Julien Olivain
fc942d5033 support/testing/tests/package/test_screen.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-10 18:32:49 +02:00
Julien Olivain
b492a5414e support/testing/tests/package/test_less.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-10 18:32:27 +02:00
Peter Korsgaard
600e36f8f2 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-09-07 16:50:14 +02:00
Peter Korsgaard
d8ce1deff9 Update for 2023.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-09-06 21:54:35 +02:00
Julien Olivain
d44a888c32 support/testing: python-magic-wormhole: fix random failures
The magic-wormhole "receive" command can output "waiting" messages
when key receival or verification are longer than a predefined
timeout:
https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L135

The intent is to have an interactive user experience.

This behavior makes the runtime test unreliable as the test always
expect the sent message as the exact output. When the test execution
is slower, it sometimes get the "waiting" message instead of the
expected message.

Some test jobs are succeeding:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4968059737
while some other are failing.

magic-wormhole can override those timers with environment variables.
See:
https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L26

This commit sets those environment variable to larger values
(100 seconds instread of 1 by default), to make sure the test will
always pass.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4962923235

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-09-02 09:43:55 +02:00
Peter Korsgaard
386b72ca22 Update for 2023.05.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 3923a4fac8)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-08-31 20:12:26 +02:00
Daniel Lang
6fa3a239ac support/scripts/gen-missing-cpe: remove rarely used script
The intention of this script is to generate the XML that can be sent to
NVD to request a new CPE identifier.

As discussed on the mailing list [0] keeping up with version numbers of
all registered CPE ID won't work.
In addition the feed used to generated the XML files will be retired
[1]. In the future an API needs to be used for fetching the data in
connection with a local database.
All of this works against keeping this script and porting it to the new
API.
As a last blow Matthew, the original author concluded [2]:
> Makes sense to drop it.  There never got to be enough momentum in the overall
> software community to make CVE or even the new identifier really accurate.

The intention is to ignore the version part of CPE IDs in the future,
and only look at the version range specified on a CVE. Therefore, a tool
to add new CPE ID versions isn't useful to us. It might still be useful
to have a tool to create the vendor and project parts of a CPE ID.
However, the current gen-missing-cpe tool doesn't support that, and the
API is anyway going to be retired. So there is no reason at all to keep
this around.

Remove gen-missing-cpe and the cpedb module. Remove the Makefile target
to call the script.

Since the cpedb module is removed, the CPEDB_URL definition must be
moved to the place where it is still used, in pkg-stats.

[0]: https://lists.buildroot.org/pipermail/buildroot/2023-August/672620.html
[1]: https://nvd.nist.gov/General/News/change-timeline
[2]: https://lists.buildroot.org/pipermail/buildroot/2023-August/672651.html

Signed-off-by: Daniel Lang <dalang@gmx.at>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-08-30 22:39:23 +02:00