Add a new rudimentary test inspired by the examples from jmespath's
README file ([1]).
[1]: https://github.com/jmespath/jmespath.py/blob/develop/README.rst
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
[Thomas: add entry in DEVELOPERS file]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Many actual tests require network usage, so just import the module to
check that at least that part works.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
zerofree is a utility which scans the free blocks in an ext2 filesystem
and fills any non-zero blocks with zeroes.
https://frippery.org/uml/
The ext2fs/ext2fs.h header guards the inclusion of <sys/types.h> behind
HAVE_SYS_TYPES_H, which is an autotools-defined macro that is only
supposed to be defined by the package itself, i.e. e2fsprogs, and that
should not leak into installed headers. However, e2fsprogs does leak it,
so we work it around, liek gentoo does.
Tested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- fix MMU dependency for comment; reword comment
- fix multi-line assignment of ZEROFREE_CFLAGS
- do not add comment trailing after assignment
- extend commit log to explain why we need the workaround
- use TARGET_CONFIGURE_OPTS, drop explicit CC=
- install to explicit destination file
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a simple test to verify that msr-tools are working.
The test needs to build a custom x86_64 kernel with support for CPUID and
MSR.
As the TSC_AUX MSR is emulated on qemu we can use it to test that a value
written with wrmsr can indeed be read back with rdmsr.
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a simple compress-uncompress test to verify that pixz is working.
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Buildroot documentation section 9.2.1.6 "Additional kernel extensions"
indicates support for kernel extensions defined in external buildroot
trees but unfortunately, there didn't seem to be any support in
br2-external script.
This patch copies 'init' code support to include external kernel
extensions defined in 'linux' dir at the br2-external root directory as
explained in documentation.
Signed-off-by: Nicolas POIROT <ni.poirot@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This patch adds a test case that
1) Builds the complete LLVM and CLANG set of host tools
2) Cross-compiles the compiler-rt runtime using CLANG
3) Builds a cross-compiled application using CLANG and the libfuzzer
compiler-rt library.
4) Executes the fuzz application (part of the libfuzzer package) on
target and checks expected output for a heap-buffer-overflow.
Note: The libfuzzer package is just a tutorial example of how to use
the toolkit provided by llvm (Thus not adding it as a full
Buildroot package).
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Romain Naour <romain.naour@smile.fr>
[Arnout: add Matt to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Some packages such as libclc need to override cmake toolchain
variables, to avoid errors caused by trying to set overriden
variables ensure that they are not defined before being set.
This prevents difficult to debug silent dropping of overriden
variables.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
As we're about to remove the nds32 architecture support from
Buildroot, drop the toolchain-external-andes-nds32 external toolchain
package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This Bootlin toolchain has been available for a while, but was not
supported until now.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
These toolchains have been marked obsolete by toolchains.bootlin.com
as they are replaced by the mips64-n32 toolchains. See commit:
121e78806b
Will allow to fix:
http://autobuild.buildroot.net/results/d13b35ba5a0f68f72e6592bdd9218b625a3c6554/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The "arches" dict lists the architectures for which Bootlin toolchains
are available, along with the corresponding Buildroot conditions for
those architecture variants.
However, there is nothing that checks that such architectures really
have Bootlin toolchains available. Even if no toolchain is available,
the architecture is considered as support, and will be listed in the
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS option, making menuconfig
believe that some Bootlin toolchain is available for the selected
architecture variant.
This is currently the case with the "mips64" architecture (as
identified by the Bootlin toolchains project). Such toolchains have
been made obsolete in toolchains.bootlin.com and replaced by the
mips64-n32 toolchains. But "mips64" is still listed in "arches",
causing this architecture to be considered as having Bootlin
toolchains available.
To avoid this to happen in the future, this commit adds a check that
verifies that an architecture listed in "arches" really has at least
one toolchain available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The microblazeel, microblazebe and openrisc toolchains need to select
this option as they don't have any gdb/gdbserver.
Helps in fixing:
http://autobuild.buildroot.net/results/6315ef7b66ee4ae8f870c92186bc674d65f62f2c/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The Bootlin i686 toolchain was already made available only on BR2_i386
with !BR2_x86_i486 && !BR2_x86_i586 && !BR2_x86_1000. However, this
was not sufficient as a few other architecture variants of BR2_i386
are "lower" than i686, and they need to be excluded as well.
Allows to fix:
http://autobuild.buildroot.net/results/56ac1a8fa5b34a9ca10eef98ae9fb090b8c762c4/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The current description of the Bootlin ARM toolchains allows them to
be selected for ARM big endian configurations, which obviously doesn't
work as these Bootlin ARM toolchains are little endian only.
We fix this by adding BR2_arm in the list of conditions for those
toolchains.
Will allow to fix:
http://autobuild.buildroot.net/results/7befbb686bb972016ba4e742976dcdb3fed1be11/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit allows to get a proper description of the dependencies for
the RISC-V 64-bit toolchain, that includes the BR2_USE_MMU dependency.
Fixes:
http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We now support both MMU-enabled and MMU-less RISC-V 64-bit
configurations. However, the Bootlin toolchain for RISC-V 64-bit only
supports MMU-enabled configurations, but the current logic in
toolchain/toolchain-external/toolchain-external-bootlin/ does not take
this into account, and allows selecting the Booltin toolchain for
MMU-less RISC-V 64-bit configurations.
To fix this, the gen-bootlin-toolchains script is modified to add the
BR2_USE_MMU dependency to the description of the RISC-V 64-bit
toolchain.
However, the BR2_USE_MMU dependency was also added for glibc and musl
toolchains unconditionally, so to avoid duplicating the dependency, we
now only add it only if not already present in the list of
dependencies for this toolchain.
This will allow to fix:
http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The Config.in options created for each toolchain were properly taking
into account the !BR2_STATIC_LIBS dependency of glibc
toolchains. However, this dependency was not taken into account into
the main BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS
option. Consequently, if an architecture is only supported by glibc,
but BR2_STATIC_LIBS is enabled, the main "Bootlin toolchain" option
was visible... but with no selectable toolchain.
We fix this by making sure that
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS is only true for all
architectures supported, taking into account the fact that some
architectures can only be supported if !BR2_STATIC_LIBS, when the only
available C library is glibc.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some constraints on a setup ended up with a plus sign in the path
for historical reasons and would then fail to match on the comparison
of the host/lib dir match. So, the =~ for bash can be augmented
with a double quote expansion to preserve the literal value of
the characters in the variable.
Example Path: /home/vagrant/test+buildroot/per-package
Signed-off-by: Charles Hardin <ckhardin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS option has been
replaced by BR2_TARGET_ROOTFS_OCI_CMD in commit [1].
Since BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS contains
only one item, we can safely replace by
BR2_TARGET_ROOTFS_OCI_CMD in the defconfig fragment
used by the test_oci test case.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2491321058
[1] 08d65d81d8
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin@orange.com>
Cc: Sergio Prado <sergio.prado@e-labworks.com>
Cc: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
For the new patch, adding block size options (commit 555f8dfd),
Yann E. MORIN requested updated testcases that specifically ensure
the extreme blocksizes (4K and 1024K) don't cause issues.
This patch splits the current test case in 2, testing with both
block sizes and ensuring the block size was applied in the same
fashion as for the specified compression.
Signed-off-by: Linus Kaschulla <linus@cosmos-ink.net>
[yann.morin.1998@free.fr: keep exisitng test with default size]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
While building the kernel tools, libelf header is missing:
output/TestZfsGlibc/build/linux-5.15.35/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
10 | #include <gelf.h>
Select BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF to build host-libelf.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014008
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The commit [1] introcuded TestZfsBase as a common function
between all Zfs tests. But TestZfsBase test is executed
as a test itself.
Rename test_run() to base_test_run() to avoid this issue.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014006
[1] 593e8cb71f
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
As reported by [1], the lxc test is broken since lxc >= 4.0.11.
A patch was added to lxc 4.0.11 to use the new mount api for devpts
setup [2] but the fall back code doesn't work when this new mount
API is not supported. This API was added in kernel 5.6.
(kernel 5.5)
DEBUG conf - conf.c:lxc_setup_devpts_child:1682 - No new devpts instance will be
mounted since no pts devices are required
lxc-start lxc_iperf3 DEBUG conf - conf.c:lxc_setup_dev_console:1966 - Cleared
all (0) mounts from "/dev/console"
lxc-start lxc_iperf3 ERROR mount_utils - mount_utils.c:mount_at:661 - No such
file or directory - Failed to mount "/proc/self/fd/44" to "/proc/self/fd/43"
lxc-start lxc_iperf3 ERROR conf - conf.c:lxc_setup_dev_console:1988 - No such
file or directory - Failed to mount "10(/dev/pts/0)" on "43"
lxc-start lxc_iperf3 ERROR conf - conf.c:lxc_setup_console:2143 - No such file
or directory - Failed to setup console
(kernel 5.6)
lxc-start lxc_iperf3 TRACE mount_utils - mount_utils.c:can_use_mount_api:582 -
Kernel supports mount api
lxc-start lxc_iperf3 TRACE mount_utils - mount_utils.c:move_detached_mount:328
- Attach detached mount 45 to filesystem at 43
lxc-start lxc_iperf3 TRACE conf - conf.c:lxc_setup_dev_console:1990 - Setup
console "/dev/pts/0"
Bump the kernel to the current LTS 5.15.38 version that fully support the
mount API needed by lxc.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429013708
[1] http://lists.busybox.net/pipermail/buildroot/2022-January/635251.html
[2] be606e16fd
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add TestZfsBase that contains the common parts of the test.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
GNU Octave is a high-level language, primarily intended for numerical
computations. It provides a convenient command line interface for
solving linear and nonlinear problems numerically, and for performing
other numerical experiments using a language that is mostly compatible
with Matlab. It may also be used as a batch-oriented language. Octave
has extensive tools for solving common numerical linear algebra
problems, finding the roots of nonlinear equations, integrating
ordinary functions, manipulating polynomials, and integrating ordinary
differential and differential-algebraic equations. It is easily
extensible and customizable via user-defined functions written in
Octave's own language, or using dynamically loaded modules written in
C++, C, Fortran, or other languages.
https://www.octave.org/
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Removed 0001-Correct-a-flaw-in-the-Python-3-version-checking.patch
because is already merged.
Select libcurl as required dependency, because keylocation now
supports https. OpenSSL was already a dependency, so libcurl will be
built with https support.
Add upstream patch to support uClibc.
We update the test cases to use the latest LTS kernel, 5.15.x.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
pkg-stats currently uses the services from support/scripts/cpedb.py to
match the CPE identifiers of packages with the official CPE database.
Unfortunately, the cpedb.py code uses regular ElementTree parsing,
which involves loading the full XML tree into memory. This causes the
pkg-stats process to consume a huge amount of memory:
thomas 1310458 85.2 21.4 3708952 3450164 pts/5 R+ 16:04 0:33 | | \_ python3 ./support/scripts/pkg-stats
So, 3.7 GB of VSZ and 3.4 GB of RSS are used by the pkg-stats
process. This is causing the OOM killer to kick-in on machines with
relatively low memory.
This commit reimplements the XML parsing needed to do the CPE matching
directly in pkg-stats, using the XmlParser functionality of
ElementTree, also called "streaming parsing". Thanks to this, we never
load the entire XML tree in RAM, but only stream it through the
parser, and construct a very simple list of all CPE identifiers. The
max memory consumption of pkg-stats is now:
thomas 1317511 74.2 0.9 381104 152224 pts/5 R+ 16:08 0:17 | | \_ python3 ./support/scripts/pkg-stats
So, 381 MB of VSZ and 152 MB of RSS, which is obviously much better.
The JSON output of pkg-stats for the full package set, before and after
this commit, is exactly identical.
Now, one will probably wonder why this isn't directly changed in
cpedb.py. The reason is simple: cpedb.py is also used by
support/scripts/missing-cpe, which (for now) heavily relies on having
in memory the ElementTree objects, to re-generate a snippet of XML
that allows us to submit to NIST new CPE entries.
So, future work could include one of those two options:
(1) Re-integrate cpedb.py into missing-cpe directly, and live with
two different ways of processing the CPE database.
(2) Rewrite the missing-cpe logic to also be compatible with a
streaming parsing, which would allow this logic to be again
shared between pkg-stats and missing-cpe.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- add missing import of requests
- import CPEDB_URL from cpedb, instead of duplicating it
- fix flake8 errors
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some upstream sites are very slow to respond, and the default timeout
of 300 seconds of the aiohttp.ClientSession() is too long. Let's
reduce it to 15 seconds.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This is useful when debugging/developing the pkg-stats script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This is useful when debugging/developing the pkg-stats script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This commit adds support for a new type of graph, showing the timeline
of a build. It shows, with one line per package, when each of this
package steps started/ended, and therefore allows to see the
sequencing of the package builds.
For a fully serialized build like we have today, this is not super
useful (except to show that everything is serialized), but it becomes
much more useful in the context of top-level parallel build.
We chose to order the graph by the time-of-configure, as it is the
closest to the actual cascade-style of a true dependency graph, which is
tiny bit more complex to achieve properly. The actual result still looks
pretty good.
The graph-build make target is extended to also generate this new
timeline graph.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- sort by start-of-configure time
- re-use existing colorsets (default or alternate)
- fix python2isms
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The check_package_get_latest_version_by_distro() function analyzes the
data returned by release-monitoring.org. For two of our
packages (bento4 and qextserialport), release-monitoring.org returns
something that is a bit odd: it returns an entry with a
"stable_versions" field that contains an empty array. Our code was
ready to have or not have a "stable_versions" entry, but when it is
present, we assumed it was not an empty array. These two packages, for
some reason, break this assumption.
In order to solve this problem, this commit is more careful, and uses
the stable_versions field only if it exists and it has at least one
entry. The code is also reworked as a sequence of "if...elif...else"
to be more readable.
This fixes the following exception when running pkg-stats on the full
package set:
Task exception was never retrieved
future: <Task finished name='Task-10772' coro=<check_package_latest_version_get() done, defined at ./support/scripts/pkg-stats:532> exception=IndexError('list index out of range')>
Traceback (most recent call last):
File "./support/scripts/pkg-stats", line 535, in check_package_latest_version_get
if await check_package_get_latest_version_by_distro(session, pkg):
File "./support/scripts/pkg-stats", line 489, in check_package_get_latest_version_by_distro
version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
IndexError: list index out of range
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: non-sequence tests as True]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
TestRust and TestRustBin has been introduced at the time when there was
no cargo package infrastructure or any package using rust compiler
(Buildroot 2018.02).
Since then the ripgrep package has been introduced, initially using
the generic package infrastructure and converted later to the cargo
package infrastructure.
Due a recent change in rust/cargo removing the cargo config file [1]
the test TestRust and TestRustBin now fail to compile since they build
an hello-world crate outside of the cargo package infrastructure
without the correct environment for cross-compiling.
Replace the 'hello-world' crate by ripgrep package and check if it
can run properly in Qemu.
Fixes tests.package.test_rust.TestRustBin:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545
But doesn't fixes tests.package.test_rust.TestRust due another bug:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
[1] b6378631c2
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
