Commit Graph

1484 Commits

Author SHA1 Message Date
José Pekkarinen
ef321ff4a9 package/avocado: new package
Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
[Thomas: add test case, add missing dependencies]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-26 10:07:52 +02:00
Heiko Thiery
28973f28ac support/scripts/pkg-stats: tweak infras field when running with -c
When we use the statistics output to generate a CVE/CPE customer
report showing whether a product is affected by CVEs, we are primarily
interested in whether they are relevant to the target
system. Currently we cannot see if the package is configured for the
build (infra==host) and/or the target system (infra==target).

Therefore this commit extends the pkg-stats script to leverage the
information available in "make show-info" output to tweak the list of
package infrastructures for each package. Thanks to this commit, the
script now has a more consistent behavior:

 * When pkg-stats is run without -c, i.e without a defined Buildroot
   configuration, it continues to operate as it did, i.e it lists all
   package infrastructures supported by the package (such as autotools
   host+target, or kconfig target, etc.)

 * When pkg-stats is run with -c, i.e with a defined Buildroot
   configuration which defines the list of packages that should be
   considered, then for each package it now lists only the package
   infrastructures used by the package in that current
   configuration. For example if you have a package with a host and
   target variant, but only the host variant is used in your
   configuration, now the pkg-stats output will only say that the host
   variant of this package is used;

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[Thomas: pretty much rework the entire implementation and how the
result is presented.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-25 19:08:52 +02:00
Yann E. MORIN
d9ff62c4cd pacakge: drop remnants of $(HOST_DIR)/usr
It's been ages (5 years at the next release) that we've not installed
host packages in $(HOST_DIR)/usr, but we still have a few packages that
reference it or install things in there.

Drop all of those in one fell swoop.

The run-time test still succeeds, and the following defconfig, which
should exercise all touched packages [*], does build:

    BR2_x86_i686=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_INIT_NONE=y
    BR2_SYSTEM_BIN_SH_NONE=y
    # BR2_PACKAGE_BUSYBOX is not set
    BR2_PACKAGE_GAWK=y
    BR2_PACKAGE_GETTEXT=y
    BR2_PACKAGE_ABOOTIMG=y
    BR2_PACKAGE_DBUS_PYTHON=y
    BR2_PACKAGE_OLA=y
    BR2_PACKAGE_JIMTCL=y
    BR2_PACKAGE_LUA=y
    # BR2_PACKAGE_LUA_32BITS is not set
    BR2_PACKAGE_ARGPARSE=y
    BR2_PACKAGE_PERL=y
    BR2_PACKAGE_PHP=y
    BR2_PACKAGE_PHP_APCU=y
    BR2_PACKAGE_PHP_LUA=y
    BR2_PACKAGE_PHP_PAM=y
    BR2_PACKAGE_PHP_PECL_DBUS=y
    BR2_PACKAGE_PYTHON3=y
    BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
    BR2_PACKAGE_PYTHON_PLY=y
    BR2_PACKAGE_PYTHON_PYBIND=y
    BR2_PACKAGE_LIBVA=y
    BR2_PACKAGE_BIND=y
    BR2_PACKAGE_BIND_SERVER=y
    BR2_PACKAGE_BIND_TOOLS=y
    BR2_PACKAGE_APPARMOR=y
    BR2_PACKAGE_APPARMOR_BINUTILS=y
    BR2_PACKAGE_APPARMOR_UTILS=y
    BR2_PACKAGE_APPARMOR_UTILS_EXTRA=y
    BR2_PACKAGE_APPARMOR_PROFILES=y
    BR2_PACKAGE_REFPOLICY=y
    BR2_PACKAGE_URANDOM_SCRIPTS=y
    BR2_PACKAGE_BASH=y
    # embiggen-disk to exercise go
    BR2_PACKAGE_EMBIGGEN_DISK=y
    BR2_TARGET_GRUB2=y
    BR2_TARGET_GRUB2_I386_PC=y
    BR2_TARGET_GRUB2_I386_EFI=y

[*] exceptions:
  - zfs was not tested: it needs a kernel to be built;
  - compiler-rt was not tsted: it needs llvm to be built, that takes
    ages, and other packages already reference the correct location for
    llvm-config, so it was assumed that is OK.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Cc: Anisse Astier <anisse@astier.eu>
Cc: Antoine Tenart <atenart@kernel.org>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Cc: Christian Stewart <christian@paral.in>
Cc: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Guillaume William Brs <guillaume.bressaix@gmail.com>
Cc: Hervé Codina <herve.codina@bootlin.com>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Cc: Julien Boibessot <julien.boibessot@armadeus.com>
Cc: Julien Olivain <ju.o@free.fr>
Cc: Matt Weber <matthew.weber@collins.com>
Cc: Nicolas Carrier <nicolas.carrier@orolia.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vadim Kochan <vadim4j@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>

---
Changes v1 -> v2:
  - fix new instance that have crept in  (Romain)
2022-07-24 11:30:40 +02:00
Arnout Vandecappelle (Essensium/Mind)
bf178754ef support/scripts/pkg-stats: fix flake8 errors
Commit f1bcb2a45c introduced a number of
flake8 errors. Fix these by:
- adding noqa to the multi-line string containing tabs;
- replacing other tabs with spaces;
- removing space after opening parenthesis;
- splitting the long lines.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-07-23 19:47:11 +02:00
Thomas Petazzoni
7dcd979098 support/testing/tests/boot: new test cases for Grub
This commit adds four test cases for Grub:

 - Grub i386 legacy BIOS
 - Grub i386 UEFI
 - Grub x86-64 UEFI
 - Grub AArch64 UEFI

There is some overlap with the ISO9660 filesystem test cases, some of
which use Grub, but we found it relevant to have separate test cases
for Grub, which were useful to test Grub in non-ISO9660 situations.

The Grub ARM UEFI case is not tested, as it requires Grub to be
chain-loaded by U-Boot. Implementing this test case is left as an
exercise for the reader.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - use EDK2 to build the OVMF blurbs from source, instead of the
    binary blobs
  - add host-dosfstools
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-07-23 19:04:41 +02:00
Sen Hastings
f1bcb2a45c support/scripts/pkg-stats: migrate to CSS grid and inline javascript
This migrates pkg-stats.html from html tables to CSS grid, allowing
the use of newer, simpler javascript that is short enough to be
inlined, instead of relying on externally hosted javascript.

Javascript sorting function was rewritten from scratch in ~55 lines,
short enough to be inlined directly in the html.

Tables were redone in CSS grid, but with care taken to mimic existing
"look and feel" of prevous implementation, albeit with slightly
better responsive behavior and default styling characteristics.

Column labels are now "sticky" and stay stuck to the top of the
viewport as you scroll down the page.

Also, css was rewritten in fewer lines and table elements were changed
to divs (for grid support).

Other small misc fixes include quoted hrefs and document language
declarations to make the w3c html validator happy.

Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-07-23 18:09:52 +02:00
Vincent Fazio
3fed424566 package/python3: use the provided pyc compiler
Previously, we used support/scripts/pycompile.py to generate the pyc
files for the python libraries.

While the script worked, it did not follow the PEP 3147 layout
requirements for py+pyc deployments.

Now, use the package's own compileall.py script. This will follow
PEP 3147 guidelines. It also supports "legacy" pyc only deployments as
described here:
  https://peps.python.org/pep-3147/#case-4-legacy-pyc-files-and-source-less-imports

With this change, we no longer need to hack support for side-by-side pyc
files because files will be deployed as appropriate.

This also has the added benefit of not requiring python3 on the host to
build host-python3.

Fixes: #14911

Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
  - build-tested in a python-less environment
  - build+run-tested with the runtime-test infra
]
Tested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-07-23 17:23:57 +02:00
Yann E. MORIN
32fb25423b support/testing: introduce py-only and py-pyc tests
Our current python3 builds only tests the pyc-only case, so add two new
tests, one for py-only and one for py+pyc. For orthogonality, rename the
current test.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-07-23 17:23:57 +02:00
Yann E. MORIN
caaa56c99f support/testing: drop legacy python2 code
Now that we only have python3, we will never have to test a
python2-based build, so we can drop python2 compatibility
cruft.

In python3, print already is a function, we don't need to
import it from the future.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-07-23 17:23:57 +02:00
Ricardo Martincoski
9bb647297a support/testing/tests: add get-developers tests
Add a runtime test in order to detect undesired changes in behavior of
the get-developers script.

The test uses a .patch file generated against the buildroot tree as a
fixture to check how get-developers operates when called to check it.
The test also overrides the DEVELOPERS file in order to be fully
reproducible and a -d option is added to get-developers in order to
allow this. Since get-developers only looks to already committed
files to compare against patch files, the fixture uses a package that
is very unlikely to be removed from buildroot tree: binutils.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: extracted from a larger patch from Ricardo, submitted at
https://patchwork.ozlabs.org/project/buildroot/patch/20220528014832.289907-1-ricardo.martincoski@gmail.com/]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-23 16:32:59 +02:00
Thomas Petazzoni
4ed7bca6a0 support/scripts/gitlab-ci.yml.in: use get-developers -v in check-DEVELOPERS
Now that we have a working mechanism to validate the DEVELOPERS file
with the get-developers -v option, let's use it.

This brings back proper validation of the DEVELOPERS file, as
get-developers without argument no longer did any validation following
commit 45aabcddc5 utils/get-developers: really make it callable from
elsewhere than the toplevel directory".

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-23 16:32:58 +02:00
Raphaël Mélotte
dfa0c8e3a8 support/testing: add new test for python-jmespath
Add a new rudimentary test inspired by the examples from jmespath's
README file ([1]).

[1]: https://github.com/jmespath/jmespath.py/blob/develop/README.rst

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
[Thomas: add entry in DEVELOPERS file]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-22 23:18:59 +02:00
Raphaël Mélotte
c15734e3d2 support/testing: add new test for python-s3transfer
Many actual tests require network usage, so just import the module to
check that at least that part works.

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-22 22:52:03 +02:00
Vincent Stehlé
98ecdd1d11 package/zerofree: new package
zerofree is a utility which scans the free blocks in an ext2 filesystem
and fills any non-zero blocks with zeroes.

https://frippery.org/uml/

The ext2fs/ext2fs.h header guards the inclusion of <sys/types.h> behind
HAVE_SYS_TYPES_H, which is an autotools-defined macro that is only
supposed to be defined by the package itself, i.e. e2fsprogs, and that
should not leak into installed headers. However, e2fsprogs does leak it,
so we work it around, liek gentoo does.

Tested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - fix MMU dependency for comment; reword comment
  - fix multi-line assignment of ZEROFREE_CFLAGS
  - do not add comment trailing after assignment
  - extend commit log to explain why we need the workaround
  - use TARGET_CONFIGURE_OPTS, drop explicit CC=
  - install to explicit destination file
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-07-17 14:05:54 +02:00
Vincent Stehlé
019d4b312d support/testing: add test for msr-tools
Add a simple test to verify that msr-tools are working.

The test needs to build a custom x86_64 kernel with support for CPUID and
MSR.

As the TSC_AUX MSR is emulated on qemu we can use it to test that a value
written with wrmsr can indeed be read back with rdmsr.

Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-07-16 23:48:25 +02:00
Vincent Stehlé
49fde7057e support/testing: add test for pixz
Add a simple compress-uncompress test to verify that pixz is working.

Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-07-16 23:44:04 +02:00
Nicolas POIROT
4de196a1ce support/scripts/br2-external: add kernel extensions
Buildroot documentation section 9.2.1.6 "Additional kernel extensions"
indicates support for kernel extensions defined in external buildroot
trees but unfortunately, there didn't seem to be any support in
br2-external script.

This patch copies 'init' code support to include external kernel
extensions defined in 'linux' dir at the br2-external root directory as
explained in documentation.

Signed-off-by: Nicolas POIROT <ni.poirot@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-06-22 22:22:30 +02:00
Matt Weber
570aa42559 testing/tests: CLANG compiler-rt runtime test
This patch adds a test case that
 1) Builds the complete LLVM and CLANG set of host tools
 2) Cross-compiles the compiler-rt runtime using CLANG
 3) Builds a cross-compiled application using CLANG and the libfuzzer
    compiler-rt library.
 4) Executes the fuzz application (part of the libfuzzer package) on
    target and checks expected output for a heap-buffer-overflow.

Note: The libfuzzer package is just a tutorial example of how to use
      the toolkit provided by llvm (Thus not adding it as a full
      Buildroot package).

Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Romain Naour <romain.naour@smile.fr>
[Arnout: add Matt to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-20 21:32:43 +02:00
James Hilliard
56a27d7d00 support/misc/toolchainfile.cmake.in: allow variables to be overridden
Some packages such as libclc need to override cmake toolchain
variables, to avoid errors caused by trying to set overriden
variables ensure that they are not defined before being set.

This prevents difficult to debug silent dropping of overriden
variables.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-20 20:59:21 +02:00
Thomas Petazzoni
18a6a38394 toolchain/toolchain-external/toolchain-external-andes-nds32: remove package
As we're about to remove the nds32 architecture support from
Buildroot, drop the toolchain-external-andes-nds32 external toolchain
package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-19 18:26:44 +02:00
Thomas Petazzoni
df13e24c9f support/testing, toolchain/toolchain-external/toolchain-external-bootlin: regenerate with ARMv7 big endian toolchain
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 17:28:30 +02:00
Thomas Petazzoni
3008505865 support/scripts/gen-bootlin-toolchains: add support for ARMv7 big endian toolchain
This Bootlin toolchain has been available for a while, but was not
supported until now.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 17:28:30 +02:00
Thomas Petazzoni
db37e75aa3 support/scripts/gen-bootlin-toolchains: drop "mips64" architecture variant
These toolchains have been marked obsolete by toolchains.bootlin.com
as they are replaced by the mips64-n32 toolchains. See commit:

  121e78806b

Will allow to fix:

  http://autobuild.buildroot.net/results/d13b35ba5a0f68f72e6592bdd9218b625a3c6554/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 17:28:30 +02:00
Thomas Petazzoni
3c11e6f3c8 support/scripts/gen-bootlin-toolchains: check that toolchains exists for a certain arch
The "arches" dict lists the architectures for which Bootlin toolchains
are available, along with the corresponding Buildroot conditions for
those architecture variants.

However, there is nothing that checks that such architectures really
have Bootlin toolchains available. Even if no toolchain is available,
the architecture is considered as support, and will be listed in the
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS option, making menuconfig
believe that some Bootlin toolchain is available for the selected
architecture variant.

This is currently the case with the "mips64" architecture (as
identified by the Bootlin toolchains project). Such toolchains have
been made obsolete in toolchains.bootlin.com and replaced by the
mips64-n32 toolchains. But "mips64" is still listed in "arches",
causing this architecture to be considered as having Bootlin
toolchains available.

To avoid this to happen in the future, this commit adds a check that
verifies that an architecture listed in "arches" really has at least
one toolchain available.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 17:28:30 +02:00
Thomas Petazzoni
a5599d727e support/scripts/gen-bootlin-toolchains: make use of BR2_TOOLCHAIN_EXTERNAL_HAS_NO_GDBSERVER
The microblazeel, microblazebe and openrisc toolchains need to select
this option as they don't have any gdb/gdbserver.

Helps in fixing:

  http://autobuild.buildroot.net/results/6315ef7b66ee4ae8f870c92186bc674d65f62f2c/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 17:28:30 +02:00
Thomas Petazzoni
26dee54e42 support/scripts/gen-bootlin-toolchains: adjust dependencies of i686 toolchains
The Bootlin i686 toolchain was already made available only on BR2_i386
with !BR2_x86_i486 && !BR2_x86_i586 && !BR2_x86_1000. However, this
was not sufficient as a few other architecture variants of BR2_i386
are "lower" than i686, and they need to be excluded as well.

Allows to fix:

  http://autobuild.buildroot.net/results/56ac1a8fa5b34a9ca10eef98ae9fb090b8c762c4/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 16:49:32 +02:00
Thomas Petazzoni
f2be0bb278 support/scripts/gen-bootlin-toolchains: improve dependencies of ARM toolchains
The current description of the Bootlin ARM toolchains allows them to
be selected for ARM big endian configurations, which obviously doesn't
work as these Bootlin ARM toolchains are little endian only.

We fix this by adding BR2_arm in the list of conditions for those
toolchains.

Will allow to fix:

  http://autobuild.buildroot.net/results/7befbb686bb972016ba4e742976dcdb3fed1be11/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 16:49:27 +02:00
Thomas Petazzoni
f55b660f9b support/testing, toolchain/toolchain-external/toolchain-external-bootlin: regenerate with latest gen-bootlin-toolchains script
This commit allows to get a proper description of the dependencies for
the RISC-V 64-bit toolchain, that includes the BR2_USE_MMU dependency.

Fixes:

  http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 16:49:25 +02:00
Thomas Petazzoni
1f4bba5967 support/scripts/gen-bootlin-toolchains: fix RISC-V 64-bit toolchain description
We now support both MMU-enabled and MMU-less RISC-V 64-bit
configurations. However, the Bootlin toolchain for RISC-V 64-bit only
supports MMU-enabled configurations, but the current logic in
toolchain/toolchain-external/toolchain-external-bootlin/ does not take
this into account, and allows selecting the Booltin toolchain for
MMU-less RISC-V 64-bit configurations.

To fix this, the gen-bootlin-toolchains script is modified to add the
BR2_USE_MMU dependency to the description of the RISC-V 64-bit
toolchain.

However, the BR2_USE_MMU dependency was also added for glibc and musl
toolchains unconditionally, so to avoid duplicating the dependency, we
now only add it only if not already present in the list of
dependencies for this toolchain.

This will allow to fix:

  http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-14 16:49:22 +02:00
Peter Korsgaard
730baf05bf Update for 2022.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-06-06 22:14:04 +02:00
Thomas Petazzoni
fc63ec9292 support/scripts/gen-bootlin-toolchains: properly take into account !BR2_STATIC_LIBS for glibc toolchains
The Config.in options created for each toolchain were properly taking
into account the !BR2_STATIC_LIBS dependency of glibc
toolchains. However, this dependency was not taken into account into
the main BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS
option. Consequently, if an architecture is only supported by glibc,
but BR2_STATIC_LIBS is enabled, the main "Bootlin toolchain" option
was visible... but with no selectable toolchain.

We fix this by making sure that
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS is only true for all
architectures supported, taking into account the fact that some
architectures can only be supported if !BR2_STATIC_LIBS, when the only
available C library is glibc.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-06-06 15:15:01 +02:00
Charles Hardin
2a9ef1f572 support/scripts: handle paths with regex characters in per-package
Some constraints on a setup ended up with a plus sign in the path
for historical reasons and would then fail to match on the comparison
of the host/lib dir match. So, the =~ for bash can be augmented
with a double quote expansion to preserve the literal value of
the characters in the variable.

Example Path: /home/vagrant/test+buildroot/per-package

Signed-off-by: Charles Hardin <ckhardin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-06-04 09:55:34 +02:00
Peter Korsgaard
1c416477db Update for 2022.02.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 1335d46be5)
[Peter: skip Makefile bump]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-05-29 22:54:53 +02:00
Romain Naour
3613c3cb33 support/testing: test_oci: update to _OCI_CMD option
BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS option has been
replaced by BR2_TARGET_ROOTFS_OCI_CMD in commit [1].

Since BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS contains
only one item, we can safely replace by
BR2_TARGET_ROOTFS_OCI_CMD in the defconfig fragment
used by the test_oci test case.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2491321058

[1] 08d65d81d8

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin@orange.com>
Cc: Sergio Prado <sergio.prado@e-labworks.com>
Cc: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-05-29 22:29:41 +02:00
Romain Naour
5468cf1095 package/python-avro: fix capitalizations of Parse
python-avro 1.11.0 deprecated schema.Parse [1] and actually error
out when used.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429013770

[1] 3e79dfec84

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-05-17 22:50:33 +02:00
Linus Kaschulla
001e0aab23 fs/squashfs: needs more test cases
For the new patch, adding block size options (commit 555f8dfd),
Yann E. MORIN requested updated testcases that specifically ensure
the extreme blocksizes (4K and 1024K) don't cause issues.

This patch splits the current test case in 2, testing with both
block sizes and ensuring the block size was applied in the same
fashion as for the specified compression.

Signed-off-by: Linus Kaschulla <linus@cosmos-ink.net>
[yann.morin.1998@free.fr: keep exisitng test with default size]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-05-17 22:40:45 +02:00
Romain Naour
447e6f0811 support/testing: test_zfs: add missing libelf dependency
While building the kernel tools, libelf header is missing:

output/TestZfsGlibc/build/linux-5.15.35/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
   10 | #include <gelf.h>

Select BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF to build host-libelf.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014008

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-14 11:44:48 +02:00
Romain Naour
e66cac7631 support/testing: test_zfs: don't run TestZfsBase as a test
The commit [1] introcuded TestZfsBase as a common function
between all Zfs tests. But TestZfsBase test is executed
as a test itself.

Rename test_run() to base_test_run() to avoid this issue.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014006

[1] 593e8cb71f

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-14 11:44:48 +02:00
Romain Naour
ca135c9939 support/testing: test_lxc: bump kernel version to 5.15.38 for complete pidfds support
As reported by [1], the lxc test is broken since lxc >= 4.0.11.
A patch was added to lxc 4.0.11 to use the new mount api for devpts
setup [2] but the fall back code doesn't work when this new mount
API is not supported. This API was added in kernel 5.6.

(kernel 5.5)
DEBUG conf - conf.c:lxc_setup_devpts_child:1682 - No new devpts instance will be
mounted since no pts devices are required
lxc-start lxc_iperf3  DEBUG conf - conf.c:lxc_setup_dev_console:1966 - Cleared
all (0) mounts from "/dev/console"
lxc-start lxc_iperf3  ERROR mount_utils - mount_utils.c:mount_at:661 - No such
file or directory - Failed to mount "/proc/self/fd/44" to "/proc/self/fd/43"
lxc-start lxc_iperf3  ERROR conf - conf.c:lxc_setup_dev_console:1988 - No such
file or directory - Failed to mount "10(/dev/pts/0)" on "43"
lxc-start lxc_iperf3  ERROR conf - conf.c:lxc_setup_console:2143 - No such file
or directory - Failed to setup console

(kernel 5.6)
lxc-start lxc_iperf3  TRACE mount_utils - mount_utils.c:can_use_mount_api:582 -
Kernel supports mount api
lxc-start lxc_iperf3  TRACE mount_utils - mount_utils.c:move_detached_mount:328
- Attach detached mount 45 to filesystem at 43
lxc-start lxc_iperf3  TRACE conf - conf.c:lxc_setup_dev_console:1990 - Setup
console "/dev/pts/0"

Bump the kernel to the current LTS 5.15.38 version that fully support the
mount API needed by lxc.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429013708

[1] http://lists.busybox.net/pipermail/buildroot/2022-January/635251.html
[2] be606e16fd

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-05-13 21:08:30 +02:00
Arnout Vandecappelle (Essensium/Mind)
1264dd50ef support/testing/tests/package/test_octave.py: fix flake8 error
Fixes:
support/testing/tests/package/test_octave.py:46:31: W605 invalid escape sequence '\ '

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-04 22:01:44 +02:00
José Luis Salvador Rufo
49f9a08223 package/zfs: add test case for musl toolchain
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-03 22:33:47 +02:00
José Luis Salvador Rufo
593e8cb71f package/zfs: refactor test cases
Add TestZfsBase that contains the common parts of the test.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-03 22:32:15 +02:00
Julien Olivain
40ea68b4b2 package/octave: new package
GNU Octave is a high-level language, primarily intended for numerical
computations. It provides a convenient command line interface for
solving linear and nonlinear problems numerically, and for performing
other numerical experiments using a language that is mostly compatible
with Matlab. It may also be used as a batch-oriented language. Octave
has extensive tools for solving common numerical linear algebra
problems, finding the roots of nonlinear equations, integrating
ordinary functions, manipulating polynomials, and integrating ordinary
differential and differential-algebraic equations. It is easily
extensible and customizable via user-defined functions written in
Octave's own language, or using dynamically loaded modules written in
C++, C, Fortran, or other languages.

https://www.octave.org/

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-03 22:04:36 +02:00
José Luis Salvador Rufo
68dfd09708 package/zfs: bump version to 2.1.4
Removed 0001-Correct-a-flaw-in-the-Python-3-version-checking.patch
because is already merged.

Select libcurl as required dependency, because keylocation now
supports https. OpenSSL was already a dependency, so libcurl will be
built with https support.

Add upstream patch to support uClibc.

We update the test cases to use the latest LTS kernel, 5.15.x.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-04-25 22:16:45 +02:00
Andreas Ziegler
6ab8b1e263 meson-package: prevent cmake find_package() picking up host libraries
The meson build system uses (1) pkg-config and (2) cmake find_package to locate
dependencies and stops only if a package is found. Cmake uses a toolchain file
that is generated by meson from the existing configuration [1].

The cmake toolchain file misses CMAKE_FIND_ROOT_PATH and CMAKE_FIND_ROOT_PATH_*
settings, and find_package() falls back to the default behaviour and looks for
dependencies on the build machine (the Buildroot HOST_DIR).

Use a feature introduced in meson 0.56.0 [2] to pass the Buildroot toolchain
file to cmake instead of using a meson-generated one.

Fixes the following build failures:
http://autobuild.buildroot.org/results/056/05673ed04c6f044f1b56b9d5342d61653be43a18/
http://autobuild.buildroot.org/results/f0a/f0a9e719114f19dc9d20622ed85dd4f8e968c20f/
http://autobuild.buildroot.org/results/527/527ee199813abbacd61c3fa32b517ea60af60659/

[1[ see mesonbuild/cmake/toolchain.py
[2] see https://mesonbuild.com/CMake-module.html#cross-compilation

Signed-off-by: Andreas Ziegler <br015@umbiko.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-21 21:42:49 +02:00
Peter Korsgaard
11b6b65d50 Update for 2022.02.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 9da87c5293)
[Peter: skip Makefile bump]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-04-13 00:09:28 +02:00
Thomas Petazzoni
bd1798ad95 support/scripts/pkg-stats: reimplement CPE parsing in pkg-stats
pkg-stats currently uses the services from support/scripts/cpedb.py to
match the CPE identifiers of packages with the official CPE database.

Unfortunately, the cpedb.py code uses regular ElementTree parsing,
which involves loading the full XML tree into memory. This causes the
pkg-stats process to consume a huge amount of memory:

thomas   1310458 85.2 21.4 3708952 3450164 pts/5 R+   16:04   0:33  |   |   \_ python3 ./support/scripts/pkg-stats

So, 3.7 GB of VSZ and 3.4 GB of RSS are used by the pkg-stats
process. This is causing the OOM killer to kick-in on machines with
relatively low memory.

This commit reimplements the XML parsing needed to do the CPE matching
directly in pkg-stats, using the XmlParser functionality of
ElementTree, also called "streaming parsing". Thanks to this, we never
load the entire XML tree in RAM, but only stream it through the
parser, and construct a very simple list of all CPE identifiers. The
max memory consumption of pkg-stats is now:

thomas   1317511 74.2  0.9 381104 152224 pts/5   R+   16:08   0:17  |   |   \_ python3 ./support/scripts/pkg-stats

So, 381 MB of VSZ and 152 MB of RSS, which is obviously much better.

The JSON output of pkg-stats for the full package set, before and after
this commit, is exactly identical.

Now, one will probably wonder why this isn't directly changed in
cpedb.py. The reason is simple: cpedb.py is also used by
support/scripts/missing-cpe, which (for now) heavily relies on having
in memory the ElementTree objects, to re-generate a snippet of XML
that allows us to submit to NIST new CPE entries.

So, future work could include one of those two options:

 (1) Re-integrate cpedb.py into missing-cpe directly, and live with
     two different ways of processing the CPE database.

 (2) Rewrite the missing-cpe logic to also be compatible with a
     streaming parsing, which would allow this logic to be again
     shared between pkg-stats and missing-cpe.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - add missing import of requests
  - import CPEDB_URL from cpedb, instead of duplicating it
  - fix flake8 errors
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-02 19:14:17 +02:00
Thomas Petazzoni
387c496b98 support/scripts/pkg-stats: add a timeout on HTTP requests for upstream URLs
Some upstream sites are very slow to respond, and the default timeout
of 300 seconds of the aiohttp.ClientSession() is too long. Let's
reduce it to 15 seconds.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-02 16:36:09 +02:00
Thomas Petazzoni
1adcf56598 support/scripts/pkg-stats: allow disabling package warnings retrieval
This is useful when debugging/developing the pkg-stats script.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-02 16:34:37 +02:00
Thomas Petazzoni
cd8a576c6d support/scripts/pkg-stats: allow disabling CPE matching
This is useful when debugging/developing the pkg-stats script.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-02 16:34:07 +02:00