Z3, also known as the Z3 Theorem Prover, is a cross-platform
satisfiability modulo theories (SMT) solver.
https://github.com/Z3Prover/z3
Signed-off-by: Julien Olivain <ju.o@free.fr>
[yann.morin.1998@free.fr:
- python bindings 'depends on' python, not 'select' it
- fix check-package in test_z3.py
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25680e6aa8)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Compilation of Perl-related packages fails if `PERL_MM_OPT` is defined.
We previously issued an error in this case.
Instead, simply `unexport` the variable.
Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Back when support/testing/tests/package/test_gdb was introduced, there
was a significant difference in how gdb < 10 and gdb >= 10 were
handled in gdb.mk, which explained why we were testing both gdb 9.x
and gdb 11.x.
However, support for gdb 9.x has now been dropped, and we only support
gdb >= 10.x, so testing gdb 9.x and 11.x separately no longer make
much sense. In addition:
- other GDB tests in the same file already test the default version,
which is now 11.x, meaning we in fact have duplicated tests between
the ones testing the default version and the ones testing 11.x
specifically
- GDB 9.x has been removed, which means all the tests testing GDB 9.x
are failing, with a Config.in.legacy build error.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828456 (TestGdbHostOnly9x)
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828454 (TestGdbHostGdbserver9x)
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828451 (TestGdbHostGdbTarget9x)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The CPIO filesystem generated by the test_python_crossbar test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.
It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828587
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It seems like on Gitlab CI, the runners are quite slow, and the Flask
server does not startup in the 15 seconds we give it. So increase this
to 30 seconds before trying to contact the Flask server.
Hopefully fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828594
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Docker compose up outputs terminal control characters intended for
interactive output viewing.
Wget similarly can use the -q option to produce quieter logs.
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Docker compose is now invoked as "docker compose" not "docker-compose."
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828442
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use the output of `cargo vendor` to generate the vendor configuration.
Fixes the need to patch the generated configuration if there are
non-crates.io dependencies.
Note:
`cargo vendor` currently prints a newline before it prints the
needed configuration.
This is fixed in +nightly, will end up in +stable soon and must
be considered when updating cargo.
See: https://github.com/rust-lang/cargo/pull/11273
Until then it is needed to remove this first line to make sure
that the contents of .cargo/config will be the same as they were
generated with the earlier version of the script. Thus, the
hashes of the packages that use this script remain the same.
Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
[yann.morin.1998@free.fr: add comment in rust-bin.mk and rust.mk]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Startig with glibc 2.34, the gconv modules description has been split in
two:
- a common definition in the old location, /usr/lib/gconv/gconv-modules
- specific definitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/
This is done so as to simplify the handling of glibc gconv modules, and
eventually to segregate those outside of glibc, and so that third-parties
may also provide their own gconv converters and their definitions.
And starting with that same glibc version, most of the gconv modules
definitions are moved to an extra configuration file in that
sub-directory.
It is thus no longer possible to use special code pages, like cp850,
which are very useful to access FAT-formatted devices.
Add support for this new gconv layout, while keeping support for older
glibc versions. Note that the modules themselves are not moved or
renamed, just the definition files have changed.
Instead of passing the one old gonv modules definitions file on stdin,
we pass the base directory to that file, and move into the script the
responsibility to find all the gconv definition files.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When only a subset of the glibc gconv modules are installed, we need to
generate a trimmed-down list of available modules. We currently use gawk
for that.
However, we are not using any GNU extension in that awk script, and it
happens to work as expected when using mawk (which has no GNU
extension).
Commit 11c1076db9 (toolchain: add option to copy the gconv libraries)
did not explain why it used gawk explicitly, and given the age for that
commit, we doubt we'd be able to have the involved participants recall
anything from that period...
Besides, gawk is not a requirement for Buildroot.
Switch over to using plain awk.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Newest versions requires a bit more time to finish.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25d865996d)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Any .pyc files generated by the pycompile script during target
finalization are currently counted in the "Unknown" package,
because packages-file-list.txt only contains the source .py file.
If a .py file is added to filesdict, add the corresponding .pyc
file as well.
Signed-off-by: Michael Klein <m.klein@mvz-labor-lb.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Polkit has two directories that are used to store rules.
Add the second directory to the existing tests, to ensure
that both work in the future.
Signed-off-by: Daniel Lang <d.lang@abatec.at>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Check that dependencies that are DT_NEEDED by a program are
automatically copied by dracut. We use cramfs, the package,
as it is small and just depends on libz.
Test more than one dracut config file.
Reported-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
[Arnout: remove test for features that haven't been merged yet]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Since we do not pivot_root/switch_root from the cpio, whatever we get
in the rootfs was exactly what we got by listing the cpio archive.
Drop the test for the presence of pv, it's redundant.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Due to a bug in the CodeSourcery ARM toolchain packaging, this
toolchain currently appears as available even for noMMU
configurations, which is obviously wrong. Due to this, the
bootlin-armv7m-uclibc.config fragment ends up using the CodeSourcery
ARM toolchain, which is obviously wrong for an ARM noMMU
configuration, causing a build failure when matching the toolchain
capabilities with the configuration.
Even though we will separately fix the CodeSourcery ARM toolchain
packaging, it makes sense to ensure that the
bootlin-armv7m-uclibc.config fragment explicitly selects the Bootlin
toolchain.
Reported-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This tests valdates that we can publish a message and read it back.
Signed-off-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
- don't manually start mosquitto, there's a startup script for that
- don't pass custom timeout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bf0d8c9659)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes flake8 errors:
support/scripts/pkg-stats:1013:133: E501 line too long (164 > 132 characters)
support/scripts/pkg-stats:1018:36: F541 f-string is missing placeholders
support/scripts/pkg-stats:1110:199: E261 at least two spaces before inline comment
For the first and the last one, we chose to split the long lines rather
than adding noqa: 501. Indeed, the long lines make it very unreadable,
and there are relatively natural places where the line can be broken.
Also split a line just below the second one in a similar way.
The f-string on 1018 doesn't need to be an f-string.
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
This is done either by switching to single quoted f-strings, triple
double quoted f-strings when needed, or simply single-quoted strings.
The renderer HTML is exactly identical before/after this commit.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
Within single-quoted f-strings, and within triple double quoted
strings, escaping all the double quotes is completely useless and
makes the code more difficult to read. Get rid of all this useless
escaping.
The renderer HTML is exactly identical before/after this commit.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
The python installer package isn't able to overwrite files of packges
that already exist, this causes problems when doing a rebuild or
update without a full clean.
To fix this we can use functionality from importlib to identify and
remove any conflicting python package files before installation.
We also need to use internals from python-installer, as we want to use
the same logic as pyinstaller uses internally for getting the scheme so
that we ensure we clean the correct package scheme (we want it to be the
same as the one we're installing)
Fixes:
Traceback (most recent call last):
File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 69, in <module>
main()
File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 61, in main
install(
File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/_core.py", line 109, in install
record = destination.write_file(
File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 207, in write_file
return self.write_to_fs(scheme, path_, stream, is_executable)
File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 167, in write_to_fs
raise FileExistsError(message)
FileExistsError: File already exists: /home/buildroot/buildroot/output/target/usr/lib/python3.10/site-packages/tinycss2/__init__.py
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
[yann.morin.1998@free.fr:
- extend commit log about the use of the installer internals (the
symbols prefixed with '_')
- check path.files against explicitly None
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The python installer cli isn't able to overwrite files of packages
that already exist, this causes problems when doing a rebuild or
update without a full clean.
Since we need to add functionality to our pyinstaller.py script to fix
this issue we must also use pyinstaller.py for host python packages.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit 7652817c93 updated the
documentation but forgot to update support/dependencies
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
It includes a simple test for the full cpio image, and a test of the
dracut image. To validate that the dracut image is a subset of the full
image, 'pv' is added to the image, and the test verifies that pv is not
part of the image. Note that the real rootfs is not mounted at the
moment, so pv is never available in the running image.
Systemd and other init systems are currently untested.
Signed-off-by: Thierry Bultel <thierry.bultel@linatsea.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The CPIO filesystem generated by the test_python_s3transfer test is
too large, and doesn't fit as an initramfs in the 256MB of RAM
available in the versatilepb machine. This causes a "Initramfs
unpacking failed: write error" when booting, and many files being
missing from the root filesystem, ultimately causing the test to fail.
It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635126
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The CPIO filesystem generated by the test_python_botocore test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.
It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635042
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The CPIO filesystem generated by the test_python_boto3 test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.
It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635041
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
In commit
9267b0f14d ("package/f2fs-tools: bump to
version 1.15.0"), f2fs-tools was bumped from 1.14.0 to 1.15.0.
It turns out that this version bump causes the output of dump.f2fs to
slightly change.
In version 1.14.0, it looked like this:
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 262144 (128 MB)
Info: MKFS version
"Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = 70c101c3
Info: checkpoint state = 181 : trimmed nat_bits unmount
In version 1.15.0, it looked like this:
Info: MKFS version
"Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: Segments per section = 1
Info: Sections per zone = 1
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = b89f8bb
Info: checkpoint state = 181 : trimmed nat_bits unmount
You will notice that the message "Info: total sectors = 262144 (128
MB)" is no longer present, and only "Info: total FS sectors =
262144 (128 MB)" is not present.
Except our test case was precisely looking for this "Info: total
sectors" string in the output, causing the test to fail.
We fix this by simply matching on "Info: total FS sectors" now.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2884634814
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Octave package test can occasionally fail due to timeout while testing
some octave modules. This commit slightly increase the timeout value
to reduce those failures.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit makes sure that the python-crossbar package pulls in the
right dependencies, based on the requirements-min.txt.
It does so by:
- Changing the 0002-Remove-idna-requirement patch by a more thorough
patch that drops all indirect dependencies from
requirements-min.txt, making it easier to have a 1:1 mapping
between lines in requirements-min.txt and Buildroot selects.
- Changing the
0003-crossbar-webservice-wap-use-markupsafe-instead-of-we patch to
update requirements-min.txt to indicate the new MarkupSafe
dependency. Here again, to have a 1:1 mapping between lines in
requirements-min.txt and Buildroot selects.
- Updating the Buildroot selects to match requirements-min.txt, with
relevant comments when it does not.
- Fixing up the Crossbar test case to no longer force autobahn to use
umsgpack. Instead, we now use the default of msgpack that is
expected by autobahn.
Fixes bug #14556, https://bugs.busybox.net/show_bug.cgi?id=14556.
Signed-off-by: Emile Cormier <emile.cormier.jr@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
python-scipy is moving to a meson build infrastructure, and requires
fortran cross-compilation. This commit extends the cross-compilation
configuration file generated for use by meson to add the "fortran",
"fortran_args" and "fortran_link_args" properties.
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
construct is a Python library for declarative serialization/
deserialization of structured binary data.
Signed-off-by: Martin Povišer <povik+lin@cutebit.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Test the scipy::io module, to demonstrate that runtime dependencies
are correct.
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Importing the scipy module in the Python interpreter running in Qemu
takes quite a while on slow machine, so a timeout extension to 30
seconds is needed to make sure the test has the time to run.
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This adds an optional linebreak at the vendor attribute in the CPE ID.
It should be noted this is purely for formatting/layout purposes
and does not actually insert any additional characters
(newline or otherwise) into the rendered text.
This means that even though the text renders across two lines,
copy-pasting will still yield one line of text.
example: https://sen-h.github.io/pkg-stats/c245575.html
see also: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/wbr
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Change the test into a characterization test for all warnings and errors
get-developers can return when parsing the DEVELOPERS files.
It will be helpful when changing the behavior of get-developers to bail
out on all syntax checking warnings.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test relies on the OLA Dummy plugin presenting a test device
and port. It starts the daemon, performs few configuration commands,
covers the Python bindings and also test the OLA web interface.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Jagan Teki <jagan@amarulasolutions.com>
[Arnout:
- Indent hash file with two spaces.
- Bump to 1.7.3 to fix build failure with recent GCC.
- Get from github instead of PyPI.
- Add host-meson and host-python-pythran dependencies.
- Properly propagate Config.in dependencies.
- Correct usage of BR2_INSTALL_LIBSTDCPP symbol.
- Remove F77, no longer used.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Guillaume:
- -lnpymath: npymath.ini localization problem
- -lnpyrandom resolution problem
- fix legal-info for latest version LICENSE.txt
- zlib is a scipy::io module runtime requirement
- update serie for scipy 1.8.1 (latest)
]
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
[Thomas:
- add runtime test
- drop dependency on OpenBLAS, which is not needed in a minimal
configuration
- remove PYTHON_SCIPY_NPY_PKG_CONFIG_PATH logic as it is no longer
needed
]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This attempts to re-implement the "sortable_hint" feature without
relying on words. The column headers and CVE expand/contract buttons
change color and cursor style on hover.
If Javascript is enabled:
Just like [PATCH 1/3] more rules are applied to the
generated stylesheet before content is loaded.
If Javascript is disabled:
The headers stay pearly white. :-)
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Sometimes a package can have a lot of CVEs.
Rather than have the CVE cell make a really tall row
(that means you have to scroll a bunch) this collapses the CVE
cell to a fixed size scrollable element with a
sticky button that lets you expand and collapse it.
If Javascript is enabled:
A stylesheet is generated and appended before content rendering,
amending the cells style to have a fixed height and overflow.
Also, the expand/contract button is unhidden.
This means the CVE cells are rendered in a collapsed state
instead of being rendered in an expanded state and then
slamming shut.
This avoids a "flash" and *helps* (vertically at least) manage CLS
(cumulative layout shift).
see: https://web.dev/cls/
If Javascript is disabled:
The cells stay fully open and the expand/contract button stays hidden.
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit c4e6d5c8be (core: implement per-package SDK and target)
introduced leading TABs in that otherwise-space-indented script.
Convert all to spaces, for homogeneity
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Back many years ago, we developed an Eclipse plugin that simplified
the usage of Buildroot toolchains. Enabling the BR2_ECLIPSE_REGISTER=y
was registering the Buildroot toolchain into a special file in your
HOME folder that the Eclipse plugin would recognize to allow to
directly use the Buildroot cross-compiler.
This Eclipse plugin has not been maintained for years. The last commit
in the repository dates back from September 2017. Since then Eclipse
has moved on, and the plugin is no longer compatible with current
versions of Eclipse.
Also, Eclipse is probably no longer that widely used in the embedded
Linux space, as other more modern IDEs have become more popular.
All in all, it's time to say good bye to this Eclipse integration.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This is a simple test importing pyalsa, showing alsa library version and
attempting to list cards.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It's time to finally switch over globally to the new spacing format
that we have agreed on for the hash file, with 2 spaces as a separator
between fields.
This commit was mechanically generated using:
find . -type f -name '*.hash' | xargs sed -i 's%^md5[ \t]*\([^ \t]*\)[ \t]*\(.*\)$%md5 \1 \2%'
find . -type f -name '*.hash' | xargs sed -i 's%^sha1[ \t]*\([^ \t]*\)[ \t]*\(.*\)$%sha1 \1 \2%'
find . -type f -name '*.hash' | xargs sed -i 's%^sha256[ \t]*\([^ \t]*\)[ \t]*\(.*\)$%sha256 \1 \2%'
find . -type f -name '*.hash' | xargs sed -i 's%^sha512[ \t]*\([^ \t]*\)[ \t]*\(.*\)$%sha512 \1 \2%'
This commit can easily be backported on the LTS branch by re-running
the same commands, if needed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 2e399893b4)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
sortGrid() has been rewritten to dynamically generate stylesheets with
explicit grid-row properties to re-order the rows, instead of removing
and reinserting the cells.
Performance *should* now be comperable to sorttable.js.
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Like all good problems, disparate pieces work together to create
a "synergistically" hairy mess.
The sortGrid() overhaul highlighted a flaw in pkg-stats allowing
for duplicate package class names across rows.
As an example,
boot/barebox/barebox.mk and boot/barebox/barebox/barebox.mk
both get the classname ._barebox and so sortGrid() sticks them on
the same line giving a table with a vestigal row sticking out
of the right side like some kind of appendage.
Also I neglected to add a "_" to the current version column's cells
pkgname class so instead of "._pkgname" we had ".pkgname" and so
the cells were not collected properly as part of the row.
These issues explain the formatting weirdness.
package classnames are now ".path_to_package_makefile" without suffix
(.mk) (so ._boot_barebox_barebox and ._boot_barebox_barebox_barebox
instead of ._barebox) in order to guarantee uniqueness.
and what was *accidentally*
class="centered current_version data .barebox" is now
class="centered current_version data ._boot_barebox_barebox"
just like *all* the other cells in the row. :p
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add four new tests for systemd (rw and ro in each case):
- use dbus-broker instead of the original dbus
- use the original dbus, with dbus-broker installed
The first two extend the existing IfUpDown test cases by just enabling
dbus-broker; the second ones extend this further, by explicitly enabling
the original dbus.
For one of the tests, we overload the test_run() function to test that
the dbus-broker daemon is indeed running as root. We need not replicate
that check in the other dbus-broker-only test, and it does not make
sense to test that in tests that have the original dbus enabled.
Presence of the original dbus and dbus-broker on the same system is
valid: the original dbus is used as the default system bus daemon. We do
not test switching between the two at runtime, though as this is really
too corner-case specific. We just test to ensure the original dbus
system bus daemon is not impacted by the presence of dbus-broker.
Note: the 'full' test-case enables all systemd options, and some of them
do pull the original dbus package, so we can't use that to test the
integration of dbus-broker; instead, we extend the ifupdown case, which
does not enable the original dbus.
The default external toolchain for cortex-A9 is the old ARM toolchain
which has kernel headers 4.10 Since dbus-broker needs toolchain headers
>= 4.17, it can't be selected with this toolchain. Switch the systemd
tests to the Bootlin toolchains instead. We switch all of them to make
things easier. Note that we will need to take care in the future that
the headers version used in the bootlin toolchain doesn't get bigger
than the kernel that is used. The kernel is currently 5.10, the headers
in the bleeding edge bootlin toolchain are 5.4.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Norbert Lange <nolange79@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The 'rtl8723ds' name, when queried from release-monitoring.org at
https://release-monitoring.org/api/projects/?pattern=rtl8723ds returns
one project, with one "stable_versions" array, which is empty. This
was not expected by the pkg-stats code, causing an exception:
Task exception was never retrieved
future: <Task finished name='Task-764' coro=<check_package_latest_version_get() done, defined at /home/thomas/projets/buildroot/./support/scripts/pkg-stats:558> exception=IndexError('list index out of range')>
Traceback (most recent call last):
File "/home/thomas/projets/buildroot/./support/scripts/pkg-stats", line 566, in check_package_latest_version_get
if await check_package_get_latest_version_by_guess(session, pkg):
File "/home/thomas/projets/buildroot/./support/scripts/pkg-stats", line 544, in check_package_get_latest_version_by_guess
projects[0]['stable_versions'][0],
IndexError: list index out of range
This commit therefore improves the checks done on the results received
from release-monitoring.org to avoid this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Pretty straightforward. Adds my name to copyright notice.
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
CSS classes are generated for each package name for sorting purposes,
However some package names start with a number and this is not allowed.
(see https://www.w3.org/TR/CSS2/syndata.html#value-def-identifier)
Fix is to prepend a character to every class name such as "_".
so every ".package" is now "._package".
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This fixes the .version-needs-update class being overridden by .correct class.
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Somewhere between binutils 2.35 and 2.37, some functionality was
added in readelf to parse more DWARF information. Unfortunately, as
reported in binutils bug
28981 ("https://sourceware.org/bugzilla/show_bug.cgi?id=28981"), this
feature causes a number of fairly scary warnings to be displayed when
running readelf on binaries built with Clang, such as the pre-built
rustc and rustdoc binaries part of the host-rust-bin package. It
looks like this:
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x23
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Bogus end-of-siblings marker detected at offset 2f in .debug_info section
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Bogus end-of-siblings marker detected at offset 10b in .debug_info section
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Bogus end-of-siblings marker detected at offset 10c in .debug_info section
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Further warnings about bogus end-of-sibling markers suppressed
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x23
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x23
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x23
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x23
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: Unrecognized form: 0x22
readelf: /home/thomas/projets/buildroot/output/host/bin/rustc: Warning: DIE at offset 0x1da refers to abbreviation number 5827 which does not exist
These warnings are caused by the readelf calls done by the
support/scripts/check-host-rpath script. The annoying thing is that
once host-rust-bin has been installed in $(HOST_DIR), this warning
appears after the installation of every single host package, because
support/scripts/check-host-rpath rescans all binaries every time.
To avoid showing those scary warnings, this commit sends the error
output of readelf to /dev/null.
Of course, it would be nicer to only filter out those warnings, but
filtering the error output without merging the error output into the
standard output is tricky, so let's keep things simple. If there is
really an error, readelf will abort.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since systemd requires a relatively new kernel, we switched to a
self-built 4.19.204 kernel. However, since then, the downloaded kernel
used by the tests has been updated to 5.10, which is certainly recent
enough to support systemd. Switch to this one. This reduces the test
time significantly.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The --testcases option of run-tests says how many test cases to build in
parallel. It automatically derives a jlevel from it by dividing the
number of cores + 1 by the number of parallel testcases. However, this
will typically result in a fractional number. Make doesn't like
fractional numbers as argument to -j.
Convert the number to integer (rounding down).
* br2_jlevel is an int, as multiprocessing.cpu_count() is an int, so it
will be always >=2 (cpu_count() raises an error if it can't determine
the number of CPU, so it will always return at least 1);
* args.testcases is an int, and is checked to be >=1
So br2_jlevel + args.testcases is guaranteed to always be bigger
than or equal to args.testcases, and the division thus bigger than 1.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
- ensure division provide at least 1
- drop the test below
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The first three lines of all systemd runtime tests are identical, and
they already call into a common function. Therefore, move those lines
into the common function as well.
We need to pass an additional argument for the rootfs type. This changes
the signature, which could create confustion with
InitSystemBase.check_init() that has a different signature. Therefore,
rename the function to check_systemd(). That also allows us to call
self.check_init() directly instead of going through super().
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
As for the Buildroot testsuite, multiply every emulator timeout by 10
to avoid sporadic failures in elastic runners.
qemu_arm_vexpress_tz_defconfig tested locally with sucess.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1970084046
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, glibc depends on !BR2_STATIC_LIBS in all the toolchain
variants.
However, for some architectures, glibc is the only supported libc. In
commit 3b3105328e ("Config.in: only
allow BR2_STATIC_LIBS on supported libc/arch"), we implemented a fix
to avoid configurations were BR2_STATIC_LIBS=y with an architecture
already supported by glibc, because these configurations are
impossible. This commit 3b3105328e
prevents from selecting BR2_STATIC_LIBS=y when the C library used for
the internal toolchain backend is glibc.
However, it introduces a discrepency between how this topic is handled
for internal and external toolchains:
- For internal toolchains, we prevent BR2_STATIC_LIBS=y if glibc is
chosen.
- For external toolchains, we allow BR2_STATIC_LIBS=y in all cases,
and it's each glibc toolchain that has !BR2_STATIC_LIBS
This commit addresses this discrepency by preventing BR2_STATIC_LIBS=y
if glibc is chosen in all cases.
Thanks to this, we can remove the !BR2_STATIC_LIBS dependency on both
the glibc package, and all glibc external toolchains.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=14256
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: update to master, fix the gen-bootlin-toolchains script, add
a comment in the static/shared choice to indicate that static is
supported only with uclibc or musl]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
cmp from diffutils is part of the host system requirements, so check
for it. It is used in package/pkg-generic.mk.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit is based on earlier work from Łukasz Stelmach
<l.stelmach@samsung.com> to add support for different page sizes on
ARM64.
In his initial submission, Łukasz took an approach similar to this
one, i.e make it ARM64-specific. Following the feedback on the mailing
list, his second version [1] tried to generalize the logic to
configure the page size between architectures. But the general
consensus during the review process was that there wasn't much to
generalize in the end.
So, this new iteration is back to a simpler approach:
* We have new options in Config.in.arm to configure the page
size. Only 4 KB and 64 KB are supported, because our testing in
Qemu and real hardware has not allowed to get a successful setup
for 16 KB pages. We can always re-add support for 16 KB later if
that is resolved.
* The logic to define the ARCH_TOOLCHAIN_WRAPPER_OPTS options is
moved from the ARC-specific file to arch/arch.mk, and extended to
cover ARM64.
* The appropriate logic in uclibc.mk and linux.mk is added to tweak
the relevant configuration options.
* A test case is added in the runtime test infrastructure to test
building and booting under Qemu a 64 KB configuration, with all 3 C
libraries.
For the regular configuration of 4 KB pages, this commit makes one
functional change: on ARM64, -Wl,-z,max-page-size=4096 is now passed in
the compiler flags of the wrapper.
[1] https://patchwork.ozlabs.org/project/buildroot/list/?series=275452
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When compiling Qt6 applications for the target, CMake needs to have
two variables defined to find the host installation of Qt. These two
variables are unconditionally defined, regardless of whether Qt6 is
enabled in the configuration or not, as they do no harm when Qt6 is
not present/used.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
[Thomas: add test case, add missing dependencies]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When we use the statistics output to generate a CVE/CPE customer
report showing whether a product is affected by CVEs, we are primarily
interested in whether they are relevant to the target
system. Currently we cannot see if the package is configured for the
build (infra==host) and/or the target system (infra==target).
Therefore this commit extends the pkg-stats script to leverage the
information available in "make show-info" output to tweak the list of
package infrastructures for each package. Thanks to this commit, the
script now has a more consistent behavior:
* When pkg-stats is run without -c, i.e without a defined Buildroot
configuration, it continues to operate as it did, i.e it lists all
package infrastructures supported by the package (such as autotools
host+target, or kconfig target, etc.)
* When pkg-stats is run with -c, i.e with a defined Buildroot
configuration which defines the list of packages that should be
considered, then for each package it now lists only the package
infrastructures used by the package in that current
configuration. For example if you have a package with a host and
target variant, but only the host variant is used in your
configuration, now the pkg-stats output will only say that the host
variant of this package is used;
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[Thomas: pretty much rework the entire implementation and how the
result is presented.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It's been ages (5 years at the next release) that we've not installed
host packages in $(HOST_DIR)/usr, but we still have a few packages that
reference it or install things in there.
Drop all of those in one fell swoop.
The run-time test still succeeds, and the following defconfig, which
should exercise all touched packages [*], does build:
BR2_x86_i686=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_GAWK=y
BR2_PACKAGE_GETTEXT=y
BR2_PACKAGE_ABOOTIMG=y
BR2_PACKAGE_DBUS_PYTHON=y
BR2_PACKAGE_OLA=y
BR2_PACKAGE_JIMTCL=y
BR2_PACKAGE_LUA=y
# BR2_PACKAGE_LUA_32BITS is not set
BR2_PACKAGE_ARGPARSE=y
BR2_PACKAGE_PERL=y
BR2_PACKAGE_PHP=y
BR2_PACKAGE_PHP_APCU=y
BR2_PACKAGE_PHP_LUA=y
BR2_PACKAGE_PHP_PAM=y
BR2_PACKAGE_PHP_PECL_DBUS=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
BR2_PACKAGE_PYTHON_PLY=y
BR2_PACKAGE_PYTHON_PYBIND=y
BR2_PACKAGE_LIBVA=y
BR2_PACKAGE_BIND=y
BR2_PACKAGE_BIND_SERVER=y
BR2_PACKAGE_BIND_TOOLS=y
BR2_PACKAGE_APPARMOR=y
BR2_PACKAGE_APPARMOR_BINUTILS=y
BR2_PACKAGE_APPARMOR_UTILS=y
BR2_PACKAGE_APPARMOR_UTILS_EXTRA=y
BR2_PACKAGE_APPARMOR_PROFILES=y
BR2_PACKAGE_REFPOLICY=y
BR2_PACKAGE_URANDOM_SCRIPTS=y
BR2_PACKAGE_BASH=y
# embiggen-disk to exercise go
BR2_PACKAGE_EMBIGGEN_DISK=y
BR2_TARGET_GRUB2=y
BR2_TARGET_GRUB2_I386_PC=y
BR2_TARGET_GRUB2_I386_EFI=y
[*] exceptions:
- zfs was not tested: it needs a kernel to be built;
- compiler-rt was not tsted: it needs llvm to be built, that takes
ages, and other packages already reference the correct location for
llvm-config, so it was assumed that is OK.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Cc: Anisse Astier <anisse@astier.eu>
Cc: Antoine Tenart <atenart@kernel.org>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Cc: Christian Stewart <christian@paral.in>
Cc: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Guillaume William Brs <guillaume.bressaix@gmail.com>
Cc: Hervé Codina <herve.codina@bootlin.com>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Cc: Julien Boibessot <julien.boibessot@armadeus.com>
Cc: Julien Olivain <ju.o@free.fr>
Cc: Matt Weber <matthew.weber@collins.com>
Cc: Nicolas Carrier <nicolas.carrier@orolia.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vadim Kochan <vadim4j@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
---
Changes v1 -> v2:
- fix new instance that have crept in (Romain)
Commit f1bcb2a45c introduced a number of
flake8 errors. Fix these by:
- adding noqa to the multi-line string containing tabs;
- replacing other tabs with spaces;
- removing space after opening parenthesis;
- splitting the long lines.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds four test cases for Grub:
- Grub i386 legacy BIOS
- Grub i386 UEFI
- Grub x86-64 UEFI
- Grub AArch64 UEFI
There is some overlap with the ISO9660 filesystem test cases, some of
which use Grub, but we found it relevant to have separate test cases
for Grub, which were useful to test Grub in non-ISO9660 situations.
The Grub ARM UEFI case is not tested, as it requires Grub to be
chain-loaded by U-Boot. Implementing this test case is left as an
exercise for the reader.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- use EDK2 to build the OVMF blurbs from source, instead of the
binary blobs
- add host-dosfstools
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This migrates pkg-stats.html from html tables to CSS grid, allowing
the use of newer, simpler javascript that is short enough to be
inlined, instead of relying on externally hosted javascript.
Javascript sorting function was rewritten from scratch in ~55 lines,
short enough to be inlined directly in the html.
Tables were redone in CSS grid, but with care taken to mimic existing
"look and feel" of prevous implementation, albeit with slightly
better responsive behavior and default styling characteristics.
Column labels are now "sticky" and stay stuck to the top of the
viewport as you scroll down the page.
Also, css was rewritten in fewer lines and table elements were changed
to divs (for grid support).
Other small misc fixes include quoted hrefs and document language
declarations to make the w3c html validator happy.
Signed-off-by: Sen Hastings <sen@phobosdpl.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Previously, we used support/scripts/pycompile.py to generate the pyc
files for the python libraries.
While the script worked, it did not follow the PEP 3147 layout
requirements for py+pyc deployments.
Now, use the package's own compileall.py script. This will follow
PEP 3147 guidelines. It also supports "legacy" pyc only deployments as
described here:
https://peps.python.org/pep-3147/#case-4-legacy-pyc-files-and-source-less-imports
With this change, we no longer need to hack support for side-by-side pyc
files because files will be deployed as appropriate.
This also has the added benefit of not requiring python3 on the host to
build host-python3.
Fixes: #14911
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- build-tested in a python-less environment
- build+run-tested with the runtime-test infra
]
Tested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Our current python3 builds only tests the pyc-only case, so add two new
tests, one for py-only and one for py+pyc. For orthogonality, rename the
current test.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Now that we only have python3, we will never have to test a
python2-based build, so we can drop python2 compatibility
cruft.
In python3, print already is a function, we don't need to
import it from the future.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a runtime test in order to detect undesired changes in behavior of
the get-developers script.
The test uses a .patch file generated against the buildroot tree as a
fixture to check how get-developers operates when called to check it.
The test also overrides the DEVELOPERS file in order to be fully
reproducible and a -d option is added to get-developers in order to
allow this. Since get-developers only looks to already committed
files to compare against patch files, the fixture uses a package that
is very unlikely to be removed from buildroot tree: binutils.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: extracted from a larger patch from Ricardo, submitted at
https://patchwork.ozlabs.org/project/buildroot/patch/20220528014832.289907-1-ricardo.martincoski@gmail.com/]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Now that we have a working mechanism to validate the DEVELOPERS file
with the get-developers -v option, let's use it.
This brings back proper validation of the DEVELOPERS file, as
get-developers without argument no longer did any validation following
commit 45aabcddc5 utils/get-developers: really make it callable from
elsewhere than the toplevel directory".
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a new rudimentary test inspired by the examples from jmespath's
README file ([1]).
[1]: https://github.com/jmespath/jmespath.py/blob/develop/README.rst
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
[Thomas: add entry in DEVELOPERS file]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Many actual tests require network usage, so just import the module to
check that at least that part works.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
zerofree is a utility which scans the free blocks in an ext2 filesystem
and fills any non-zero blocks with zeroes.
https://frippery.org/uml/
The ext2fs/ext2fs.h header guards the inclusion of <sys/types.h> behind
HAVE_SYS_TYPES_H, which is an autotools-defined macro that is only
supposed to be defined by the package itself, i.e. e2fsprogs, and that
should not leak into installed headers. However, e2fsprogs does leak it,
so we work it around, liek gentoo does.
Tested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- fix MMU dependency for comment; reword comment
- fix multi-line assignment of ZEROFREE_CFLAGS
- do not add comment trailing after assignment
- extend commit log to explain why we need the workaround
- use TARGET_CONFIGURE_OPTS, drop explicit CC=
- install to explicit destination file
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a simple test to verify that msr-tools are working.
The test needs to build a custom x86_64 kernel with support for CPUID and
MSR.
As the TSC_AUX MSR is emulated on qemu we can use it to test that a value
written with wrmsr can indeed be read back with rdmsr.
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a simple compress-uncompress test to verify that pixz is working.
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Buildroot documentation section 9.2.1.6 "Additional kernel extensions"
indicates support for kernel extensions defined in external buildroot
trees but unfortunately, there didn't seem to be any support in
br2-external script.
This patch copies 'init' code support to include external kernel
extensions defined in 'linux' dir at the br2-external root directory as
explained in documentation.
Signed-off-by: Nicolas POIROT <ni.poirot@laposte.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This patch adds a test case that
1) Builds the complete LLVM and CLANG set of host tools
2) Cross-compiles the compiler-rt runtime using CLANG
3) Builds a cross-compiled application using CLANG and the libfuzzer
compiler-rt library.
4) Executes the fuzz application (part of the libfuzzer package) on
target and checks expected output for a heap-buffer-overflow.
Note: The libfuzzer package is just a tutorial example of how to use
the toolkit provided by llvm (Thus not adding it as a full
Buildroot package).
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Romain Naour <romain.naour@smile.fr>
[Arnout: add Matt to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Some packages such as libclc need to override cmake toolchain
variables, to avoid errors caused by trying to set overriden
variables ensure that they are not defined before being set.
This prevents difficult to debug silent dropping of overriden
variables.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
As we're about to remove the nds32 architecture support from
Buildroot, drop the toolchain-external-andes-nds32 external toolchain
package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This Bootlin toolchain has been available for a while, but was not
supported until now.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
These toolchains have been marked obsolete by toolchains.bootlin.com
as they are replaced by the mips64-n32 toolchains. See commit:
121e78806b
Will allow to fix:
http://autobuild.buildroot.net/results/d13b35ba5a0f68f72e6592bdd9218b625a3c6554/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The "arches" dict lists the architectures for which Bootlin toolchains
are available, along with the corresponding Buildroot conditions for
those architecture variants.
However, there is nothing that checks that such architectures really
have Bootlin toolchains available. Even if no toolchain is available,
the architecture is considered as support, and will be listed in the
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS option, making menuconfig
believe that some Bootlin toolchain is available for the selected
architecture variant.
This is currently the case with the "mips64" architecture (as
identified by the Bootlin toolchains project). Such toolchains have
been made obsolete in toolchains.bootlin.com and replaced by the
mips64-n32 toolchains. But "mips64" is still listed in "arches",
causing this architecture to be considered as having Bootlin
toolchains available.
To avoid this to happen in the future, this commit adds a check that
verifies that an architecture listed in "arches" really has at least
one toolchain available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The microblazeel, microblazebe and openrisc toolchains need to select
this option as they don't have any gdb/gdbserver.
Helps in fixing:
http://autobuild.buildroot.net/results/6315ef7b66ee4ae8f870c92186bc674d65f62f2c/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The Bootlin i686 toolchain was already made available only on BR2_i386
with !BR2_x86_i486 && !BR2_x86_i586 && !BR2_x86_1000. However, this
was not sufficient as a few other architecture variants of BR2_i386
are "lower" than i686, and they need to be excluded as well.
Allows to fix:
http://autobuild.buildroot.net/results/56ac1a8fa5b34a9ca10eef98ae9fb090b8c762c4/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The current description of the Bootlin ARM toolchains allows them to
be selected for ARM big endian configurations, which obviously doesn't
work as these Bootlin ARM toolchains are little endian only.
We fix this by adding BR2_arm in the list of conditions for those
toolchains.
Will allow to fix:
http://autobuild.buildroot.net/results/7befbb686bb972016ba4e742976dcdb3fed1be11/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit allows to get a proper description of the dependencies for
the RISC-V 64-bit toolchain, that includes the BR2_USE_MMU dependency.
Fixes:
http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We now support both MMU-enabled and MMU-less RISC-V 64-bit
configurations. However, the Bootlin toolchain for RISC-V 64-bit only
supports MMU-enabled configurations, but the current logic in
toolchain/toolchain-external/toolchain-external-bootlin/ does not take
this into account, and allows selecting the Booltin toolchain for
MMU-less RISC-V 64-bit configurations.
To fix this, the gen-bootlin-toolchains script is modified to add the
BR2_USE_MMU dependency to the description of the RISC-V 64-bit
toolchain.
However, the BR2_USE_MMU dependency was also added for glibc and musl
toolchains unconditionally, so to avoid duplicating the dependency, we
now only add it only if not already present in the list of
dependencies for this toolchain.
This will allow to fix:
http://autobuild.buildroot.net/results/d6aee9b275b1ec399aea59758ac8f69fdc5691fc/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The Config.in options created for each toolchain were properly taking
into account the !BR2_STATIC_LIBS dependency of glibc
toolchains. However, this dependency was not taken into account into
the main BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS
option. Consequently, if an architecture is only supported by glibc,
but BR2_STATIC_LIBS is enabled, the main "Bootlin toolchain" option
was visible... but with no selectable toolchain.
We fix this by making sure that
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS is only true for all
architectures supported, taking into account the fact that some
architectures can only be supported if !BR2_STATIC_LIBS, when the only
available C library is glibc.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some constraints on a setup ended up with a plus sign in the path
for historical reasons and would then fail to match on the comparison
of the host/lib dir match. So, the =~ for bash can be augmented
with a double quote expansion to preserve the literal value of
the characters in the variable.
Example Path: /home/vagrant/test+buildroot/per-package
Signed-off-by: Charles Hardin <ckhardin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS option has been
replaced by BR2_TARGET_ROOTFS_OCI_CMD in commit [1].
Since BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS contains
only one item, we can safely replace by
BR2_TARGET_ROOTFS_OCI_CMD in the defconfig fragment
used by the test_oci test case.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2491321058
[1] 08d65d81d8
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin@orange.com>
Cc: Sergio Prado <sergio.prado@e-labworks.com>
Cc: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
For the new patch, adding block size options (commit 555f8dfd),
Yann E. MORIN requested updated testcases that specifically ensure
the extreme blocksizes (4K and 1024K) don't cause issues.
This patch splits the current test case in 2, testing with both
block sizes and ensuring the block size was applied in the same
fashion as for the specified compression.
Signed-off-by: Linus Kaschulla <linus@cosmos-ink.net>
[yann.morin.1998@free.fr: keep exisitng test with default size]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
While building the kernel tools, libelf header is missing:
output/TestZfsGlibc/build/linux-5.15.35/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
10 | #include <gelf.h>
Select BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF to build host-libelf.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014008
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The commit [1] introcuded TestZfsBase as a common function
between all Zfs tests. But TestZfsBase test is executed
as a test itself.
Rename test_run() to base_test_run() to avoid this issue.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429014006
[1] 593e8cb71f
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
As reported by [1], the lxc test is broken since lxc >= 4.0.11.
A patch was added to lxc 4.0.11 to use the new mount api for devpts
setup [2] but the fall back code doesn't work when this new mount
API is not supported. This API was added in kernel 5.6.
(kernel 5.5)
DEBUG conf - conf.c:lxc_setup_devpts_child:1682 - No new devpts instance will be
mounted since no pts devices are required
lxc-start lxc_iperf3 DEBUG conf - conf.c:lxc_setup_dev_console:1966 - Cleared
all (0) mounts from "/dev/console"
lxc-start lxc_iperf3 ERROR mount_utils - mount_utils.c:mount_at:661 - No such
file or directory - Failed to mount "/proc/self/fd/44" to "/proc/self/fd/43"
lxc-start lxc_iperf3 ERROR conf - conf.c:lxc_setup_dev_console:1988 - No such
file or directory - Failed to mount "10(/dev/pts/0)" on "43"
lxc-start lxc_iperf3 ERROR conf - conf.c:lxc_setup_console:2143 - No such file
or directory - Failed to setup console
(kernel 5.6)
lxc-start lxc_iperf3 TRACE mount_utils - mount_utils.c:can_use_mount_api:582 -
Kernel supports mount api
lxc-start lxc_iperf3 TRACE mount_utils - mount_utils.c:move_detached_mount:328
- Attach detached mount 45 to filesystem at 43
lxc-start lxc_iperf3 TRACE conf - conf.c:lxc_setup_dev_console:1990 - Setup
console "/dev/pts/0"
Bump the kernel to the current LTS 5.15.38 version that fully support the
mount API needed by lxc.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2429013708
[1] http://lists.busybox.net/pipermail/buildroot/2022-January/635251.html
[2] be606e16fd
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add TestZfsBase that contains the common parts of the test.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
GNU Octave is a high-level language, primarily intended for numerical
computations. It provides a convenient command line interface for
solving linear and nonlinear problems numerically, and for performing
other numerical experiments using a language that is mostly compatible
with Matlab. It may also be used as a batch-oriented language. Octave
has extensive tools for solving common numerical linear algebra
problems, finding the roots of nonlinear equations, integrating
ordinary functions, manipulating polynomials, and integrating ordinary
differential and differential-algebraic equations. It is easily
extensible and customizable via user-defined functions written in
Octave's own language, or using dynamically loaded modules written in
C++, C, Fortran, or other languages.
https://www.octave.org/
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Removed 0001-Correct-a-flaw-in-the-Python-3-version-checking.patch
because is already merged.
Select libcurl as required dependency, because keylocation now
supports https. OpenSSL was already a dependency, so libcurl will be
built with https support.
Add upstream patch to support uClibc.
We update the test cases to use the latest LTS kernel, 5.15.x.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
pkg-stats currently uses the services from support/scripts/cpedb.py to
match the CPE identifiers of packages with the official CPE database.
Unfortunately, the cpedb.py code uses regular ElementTree parsing,
which involves loading the full XML tree into memory. This causes the
pkg-stats process to consume a huge amount of memory:
thomas 1310458 85.2 21.4 3708952 3450164 pts/5 R+ 16:04 0:33 | | \_ python3 ./support/scripts/pkg-stats
So, 3.7 GB of VSZ and 3.4 GB of RSS are used by the pkg-stats
process. This is causing the OOM killer to kick-in on machines with
relatively low memory.
This commit reimplements the XML parsing needed to do the CPE matching
directly in pkg-stats, using the XmlParser functionality of
ElementTree, also called "streaming parsing". Thanks to this, we never
load the entire XML tree in RAM, but only stream it through the
parser, and construct a very simple list of all CPE identifiers. The
max memory consumption of pkg-stats is now:
thomas 1317511 74.2 0.9 381104 152224 pts/5 R+ 16:08 0:17 | | \_ python3 ./support/scripts/pkg-stats
So, 381 MB of VSZ and 152 MB of RSS, which is obviously much better.
The JSON output of pkg-stats for the full package set, before and after
this commit, is exactly identical.
Now, one will probably wonder why this isn't directly changed in
cpedb.py. The reason is simple: cpedb.py is also used by
support/scripts/missing-cpe, which (for now) heavily relies on having
in memory the ElementTree objects, to re-generate a snippet of XML
that allows us to submit to NIST new CPE entries.
So, future work could include one of those two options:
(1) Re-integrate cpedb.py into missing-cpe directly, and live with
two different ways of processing the CPE database.
(2) Rewrite the missing-cpe logic to also be compatible with a
streaming parsing, which would allow this logic to be again
shared between pkg-stats and missing-cpe.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- add missing import of requests
- import CPEDB_URL from cpedb, instead of duplicating it
- fix flake8 errors
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some upstream sites are very slow to respond, and the default timeout
of 300 seconds of the aiohttp.ClientSession() is too long. Let's
reduce it to 15 seconds.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This is useful when debugging/developing the pkg-stats script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This is useful when debugging/developing the pkg-stats script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This commit adds support for a new type of graph, showing the timeline
of a build. It shows, with one line per package, when each of this
package steps started/ended, and therefore allows to see the
sequencing of the package builds.
For a fully serialized build like we have today, this is not super
useful (except to show that everything is serialized), but it becomes
much more useful in the context of top-level parallel build.
We chose to order the graph by the time-of-configure, as it is the
closest to the actual cascade-style of a true dependency graph, which is
tiny bit more complex to achieve properly. The actual result still looks
pretty good.
The graph-build make target is extended to also generate this new
timeline graph.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- sort by start-of-configure time
- re-use existing colorsets (default or alternate)
- fix python2isms
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The check_package_get_latest_version_by_distro() function analyzes the
data returned by release-monitoring.org. For two of our
packages (bento4 and qextserialport), release-monitoring.org returns
something that is a bit odd: it returns an entry with a
"stable_versions" field that contains an empty array. Our code was
ready to have or not have a "stable_versions" entry, but when it is
present, we assumed it was not an empty array. These two packages, for
some reason, break this assumption.
In order to solve this problem, this commit is more careful, and uses
the stable_versions field only if it exists and it has at least one
entry. The code is also reworked as a sequence of "if...elif...else"
to be more readable.
This fixes the following exception when running pkg-stats on the full
package set:
Task exception was never retrieved
future: <Task finished name='Task-10772' coro=<check_package_latest_version_get() done, defined at ./support/scripts/pkg-stats:532> exception=IndexError('list index out of range')>
Traceback (most recent call last):
File "./support/scripts/pkg-stats", line 535, in check_package_latest_version_get
if await check_package_get_latest_version_by_distro(session, pkg):
File "./support/scripts/pkg-stats", line 489, in check_package_get_latest_version_by_distro
version = data['stable_versions'][0] if 'stable_versions' in data else data['version'] if 'version' in data else None
IndexError: list index out of range
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: non-sequence tests as True]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
TestRust and TestRustBin has been introduced at the time when there was
no cargo package infrastructure or any package using rust compiler
(Buildroot 2018.02).
Since then the ripgrep package has been introduced, initially using
the generic package infrastructure and converted later to the cargo
package infrastructure.
Due a recent change in rust/cargo removing the cargo config file [1]
the test TestRust and TestRustBin now fail to compile since they build
an hello-world crate outside of the cargo package infrastructure
without the correct environment for cross-compiling.
Replace the 'hello-world' crate by ripgrep package and check if it
can run properly in Qemu.
Fixes tests.package.test_rust.TestRustBin:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202545
But doesn't fixes tests.package.test_rust.TestRust due another bug:
https://gitlab.com/buildroot.org/buildroot/-/jobs/2116202544
[1] b6378631c2
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fixes https://gitlab.com/buildroot.org/buildroot/-/jobs/2088684091
python sample_python_pyyaml_dec.py
Traceback (most recent call last):
File "/root/sample_python_pyyaml_dec.py", line 5, in <module>
data = yaml.load(serialized)
TypeError: load() missing 1 required positional argument: 'Loader'
yaml.load() requires a loader argument since the move to version 6.0:
https://github.com/yaml/pyyaml/pull/561
The test does not need the extra functionality of load(), so instead move to
the recommended safe_load().
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit 471ecea5ee (core/show-info: 'name' only applies to packages)
removed the 'name' field for rootfs (really, for non-package) entries,
thus breaking the pkg-stats processing.
We fix that by excluding any entry that has no 'name', on the assumption
that if it has no name, it is not a package.
Reported-by: Xogium on IRC
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This adds pep517(needed for flit-core to build itself) and flit python
package types.
We need to add an installer script and pass it appropriate options for
installing pep517 wheels generated by python-pypa-build during the
build stage. Unfortunately it seems pep517 does not support builds
without using the wheel format.
We also need to add a patch fixing the version parser in flit-core.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
[Arnout:
- fix indentation in pkg-python.mk (tabs, not spaces);
- use the new _CMD variables instead of duplicating the entire _CMDS
definitions;
- no need to filter dependencies (they're not self-referencing);
- _NEEDS_HOST_PYTHON no longer exists;
- host-python-pypa-build gets added to DEPENDENCIES automatically.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit add a simple test checking the reported distro name and
id are Buildroot (as reported by /etc/os-release).
Signed-off-by: Julien Olivain <ju.o@free.fr>
[Arnout: drop python2 variant]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The -z option for head was only added in coreutils 8.25, but some older
enterprise-grade distributions (e.g. the oldest still maintained RHEL 7)
only have nothing more recent than coreutils 8.22.
We fix that by using sed to remove everything that starts with the first
NULL byte, \x00.
Signed-off-by: Clayton Shotwell <clayton.shotwell@collins.com>
[yann.morin.1998@free.fr: hex is \xHH, not \xH, reword commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Python2 for the target is about to get removed, so drop the tests using it.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
support/testing/tests/package/test_lua_cffi.py:14:1: W391 blank line at end of file
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Allow developers to run check-package for init scripts, that call
shellcheck, without having to install the tool.
Since the docker have a fixed version of the tool, there will be no
difference between runs in different machines.
One can call:
$ utils/docker-run utils/check-package package/package/S*
$ utils/docker-run shellcheck package/package/S*
This change also allows to eventually run check-package for init scripts
in the GitLab CI.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
... so we can catch regressions on check-package.
Update to the new docker image that was pushed after the previous
commit.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
... so the unit tests for check-package can run in the GitLab CI.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
lua-sdl2 is not available on Lua 5.4, so update its test to use Lua 5.3
instead.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Some software decides based on uid/gid whether a user is a system or
normal (human) user, with different behaviour for those flavors (example
journald [2]).
So adding logic to create system-users is necessary, we take the now
common ranges from [1].
This extends the mkusers script to allow -2 for uid/gid, this argument
will take an identifier from the user range. All identifiers used up to
now should have been from the system range, so -1 is now interpreted as
a system user/group.
Note that after this commit, all the UIDs and GIDs that are created
automatically (with -1) will change. That means if there is peristent
data on an existing system that was created by such an automatic user,
it will suddenly belong to a different user. However, this could already
happen before: if a USERS line is added to a package, then other UIDs
may change as well.
Add system/user ranges as variables, and the argument for user/system
uid variable as well. Thus some magic constants could be removed, some
further occurences of -1 were replaced with equivalent logic. For
consistency, the existing MIN/MAX_UID/GID variables are renamed to
FIRST/LAST_USER_UID/GID.
Update the documentation with the new automatic ranges.
[1] - https://systemd.io/UIDS-GIDS/
[2] - https://www.freedesktop.org/software/systemd/man/journald.conf.html
Signed-off-by: Norbert Lange <nolange79@gmail.com>
[Arnout: use -1 for system users; refactor the changes a bit]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Removed a few variables, as they were only used to communicate
between the meson package and pkg-meson.mk and are not needed
anymore.
Moved cross-compilation.conf.in out of meson package.
Creating the cross-compilation.conf files for packages is now
using the original template.
To avoid duplicate code, the common sed pattern is stored in
a make variable.
Use explicit Buildroot variables for compiler tools,
and some fixes. (TARGET_LDFLAGS and TARGET_CXXFLAGS
were mixed up with PKG_TARGET_CFLAGS)
Signed-off-by: Norbert Lange <nolange79@gmail.com>
[Arnout: keep PKG_MESON_INSTALL_CROSS_CONF in
TOOLCHAIN_TARGET_FINALIZE_HOOKS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 323ae1e681)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit add a simple test doing symmetric encryption/decryption
to check this python interface with the gpg binary is working fine.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The x86-64-v4 toolchain assumes availability of AVX512, as per the
definition of the x86-64-v4 "standard".
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Following the merge of
d6ce2a1681 ("arch/Config.in.x86: add
option for -march=x86-64") and
eeace1cc13 ("arch/Config.in.x86: add support for
x86-64-v2, x86-64-v3, x86-64-v4"), bootlin.toolchains.com now provides
toolchains targetting the x86-64, x86-64-v2, x86-64-v3 and x86-64-v4
architecture variants.
This commits modifies gen-bootlin-toolchains to support these
toolchains. It should be noted that the description for the x86-64-v3
and x86-64-v4 toolchains are for now the same, as Buildroot doesn't
yet have the options to describe the extra features that x86-64-v4
expects to find on the hardware platform.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This new test ensures that libraries and binaries generated
using Parrot Alchemy build system are correct.
Indeed, the test uses libshdata-stress.
This binary depends on libshdata.
libshdata depends on libfutils and libfutils depends on ulog.
All of these binaries and libraries are built using Alchemy.
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit 1ba85b7f87 (support/download: add explicit no-hash support)
introduced the 'none' hash type, in an attempt to make hash files
mandatory, but not failing on archives localy generated, like those
for git or svn repositories, especially for those packages where a
version choice was present, which would allow for either remote
archives for which we'd have a hash or VCS trees for which we could
not have a hash for the localy generated archive.
Indeed, back in the time, we did not have a mean to generate
reproducible archives, so having a hash file without a hash for
thosel ocally generated archives would trigger an error in the
hash-checking machinery.
But now, low-and-behold, we do know how to generate those archives,
and we have a mechanism to explicitly exclude some archives from being
hash-checked (e.g. when the version string itself can be user-provided).
As such, the 'none' hash type no longer has any raison d'être, we do not
use it in-tree, and its use in a br2-external tree is most probably
inexistent (as is the use of hash files alotgether most probably).
So we simply drop the support for that.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
[Thomas: drop support in checkpackagelib, as reported by Ricardo.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
support/scripts/pkg-stats:1171:8: E713 test for membership should be 'not in'
support/scripts/pkg-stats:1175:8: E713 test for membership should be 'not in'
support/scripts/pkg-stats:1179:8: E713 test for membership should be 'not in'
3 E713 test for membership should be 'not in'
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1955772278
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When debugging pkg-stats, it's quite useful to be able to disable some
features that are quite long (checking upstream URL, checking latest
version, checking CVE). This commit adds a --disable option, which can
take a comma-separated list of features to disable, such as:
./support/scripts/pkg-stats --disable url,upstream
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The .affects() method of the CVE class in support/scripts/cve.py can
return 3 values: CVE_AFFECTS, CVE_DOESNT_AFFECT and CVE_UNKNOWN.
We of course properly account for CVEs where .affects() return
CVE_AFFECTS, but the ones for which CVE_UNKNOWN is returned are
currently ignored, and therefore treated as if they did not affect the
package.
However CVE_UNKNOWN in fact indicates that the v_start/v_end fields of
the CPE entry could not be parsed by
distutils.version.LooseVersion(). Instead of ignoring such cases, this
commit adds support for the concept of "unsure CVEs", which will be
listed next to CVEs known to affect the package, so that we are aware
of them and can investigate the version issue.
Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In a follow-up commit, we are about to bump python-cryptography to a
new version, which has the interesting charateristic of using Rust
code. This means python-cryptography will now only be available on
platforms supported by Rust, which for now excludes uclibc-based
configurations (none of the Rust Tier1/Tier2 platforms use uClibc,
there is some uClibc support in Tier3 platforms but they have not been
added to Buildroot for now).
So in preparation for this bump, we switch the few test cases of
Python packages that directly or indirectly use python-cryptography to
use a glibc toolchain. Another impacted test case is the
docker-compose test case, but it already uses a glibc toolchain;
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In most pure Rust packages, the Cargo.toml manifest is at the root
directory, which is why we could call "cargo vendor" without
specifying the path of the manifest.
However, other packages, such as python-cryptography, which have parts
implemented in Rust, have their Cargo.toml located in a specific
subdirectory.
This commit extends the cargo-post-process download script to
understand a BR_CARGO_MANIFEST_PATH environment variable, which allows
a package to pass the location of the Cargo.toml file. If not passed,
"Cargo.toml" is used, preserving the existing behavior for other
packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This breaks the post_process_unpack() function in
support/download/helpers, which had a sequence of pipe, with "head"
that can abort early and cause the pipe to fail.
Fixes intermitent:
make[1]: *** [package/pkg-generic.mk:190: /builds/tpetazzoni/buildroot/test-output/TestDockerCompose/build/containerd-1.5.8/.stamp_downloaded] Error 141
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In order to be package agnostic, the install phase is now using cargo
instead of install. TARGET_CONFIGURE_OPTS is now also set when running
cargo in order to support cross compiling C code within cargo.
This commit also adds support/download/cargo-post-process to perform
the vendoring on Cargo packages.
The <pkg>_LICENSE variable of cargo packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
[Thomas: add support for host-cargo-package and vendoring]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit introduces the download post-process script
support/download/go-post-process, and hooks it into the Go package
infrastructure.
The -modcacherw flag is added to ensure that the Go cache is
read/write, and can be deleted properly upon "make clean".
The <pkg>_LICENSE variable of golang packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
For now, the download post-process logic uses mk_tar_gz, which repacks
a tarball compressed with gzip. So we can only accept as input a
tarball also compressed with gzip. To enforce that, this commit
changes post_process_unpack() to use tar xzf. This makes sure that if
a tarball compressed with something else than gzip gets used, it will
bail out and we will notice.
Support for other compression schemes can be added later on.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The commit [1] added a sed command used to retreive a pattern
to keep only defconfigs whose name start with the pattern.
"<foo>-defconfigs-<pattern>"
The sed command doesn't work as expected if <foo> contains a
single hyphen [2]:
"qemu-6.2.0-defconfigs-qemu"
Update the sed command to ignore completely the part before
"-defconfigs-".
[1] 65d2f04c01
[2] http://lists.busybox.net/pipermail/buildroot/2022-January/632507.html
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The way that python-pybind can be used is fairly complicated, so a
runtime test for it is convenient. In addition, this test validates that
the headers actually work at runtime.
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
[Arnout:
- Retain python3 only.
- python-pybind is a target package, not host.
- Select python-pybind instead of depend.
- Simplify python-pybind-example package.
- Check in python-pybind-example build if pybind11.get_include()
produces output.
- Don't use python3 -m pybind11 --includes: it includes the main python
includes, which are for the host, not for the target.
- Use TestPythonPackageBase instead of open-coding something imported
with host python.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The original patch for commit cff428fe31 ("download/git: support Git
LFS") included a call to "git lfs install" but this was a problem as it
could modify ~/.gitconfig outside the dl/ tree. When this was
updated it was thought that the modification to gitconfig was
unnecessary because the LFS fetch and checkout steps are performed
manually.
Unfortunately, this is not correct and the LFS checkout fails with:
Cannot checkout LFS objects, Git LFS is not installed.
Add the call to "git lfs install", with the --local option so that only
the repository's .git/config is modified and not the user's global
~/.gitconfig.
This is also required for submodules as the parent repository's config
is not inherited.
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: John Keeping <john@metanate.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Create a test to check Mender client at runtime.
The aim of this test is:
- to check the correct execution of simple Mender commands,
in a minimal environment;
- to validate there is no missing dependencies for runtime.
This test is not a board integration test for Mender,
including well-configured bootloader, partitioning, ...
Check:
- the daemon is started;
- the current 'artifact name' (name of the image or update) of the active
partition is read, without error.
For that, we need to fake (see the 'overlay' directory):
- some bootloader environment variables;
- the name of an update.
Signed-off-by: Mikael Bourhis-Cloarec <mikael.bourhis@smile.fr>
[Romain: remove single hyphen command (Mender 3.0.0)]
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
download post process scripts will often need to unpack the source
code tarball, do some operation, and then repack it. In order to help
with this, post-process-helpers provide an unpack() function and a
repack() function.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
In order to support package managers such as Cargo (Rust) or Go, we
want to run some custom logic after the main download, but before
packing the tarball and checking the hash.
To implement this, this commit introduces a concept of download
post-processing: if -p <something> is passed to the dl-wrapper, then
support/download/<something>-post-process will be called.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- double-quote variable expansion when calling post-process script
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, relocate-sdk.sh must be run _after_ relocating the SDK. There
are cases where it is useful to already prepare the SDK _before_
relocating. For example, it allows to prepare a tarball that the user
has to extract to a specific, pre-defined location and nothing more than
that, which is simpler for the user than requiring the script to be run.
In addition, it hides the build directory that was used by the SDK
builder (somewhat).
Add an optional argument to relocate-sdk.sh that gives the target
directory.
Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Git Large File Storage replaces large files with text pointers in the
Git repository while storing the contents on a remote server. If a
repository is using this extension, then git-lfs must be used to
checkout the large files before the source archive is generated.
Signed-off-by: John Keeping <john@metanate.com>
[vfazio:
- add git-lfs to DL_TOOLS_DEPENDENCIES
- fixup for 5a0d681394
("infra/pkg-download: make the DOWNLOAD macro fully parameterised")
]
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[Arnout:
- don't "git lfs install";
- recurse into submodules.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
support/testing/tests/download/sshd.py:50:28: E261 at least two spaces before inline comment
1 E261 at least two spaces before inline comment
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add download test infrastructure which starts an OpenSSH server using
the sshd binary installed on the Buildroot host. This server can then be
used to test the expected usage of the SCP and SFTP download methods.
The test creates new SSH keys for the server and client, so that the
server can be run as a non-root user.
A new test module has been added called `tests.download.sshd` which
contains helper methods to create the SSH keys and a class called
`OpenSSHDaemon` which handles the sshd server component.
The tests download example packages in the br2-external project `ssh`.
They check the following conditions for both SCP and SFTP download
methods:
- Correct hash.
- Incorrect hash.
- No hash file.
The SSH download test infrastructure is based on test_git.py.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
[Arnout:
- remove spurious end-of-line backslash;
- remove unnecessary executable bit;
- skip test instead of failing if sshd, ssh-keygen, scp or sftp are not
found;
- decode the output of subprocess;
- use subprocess.check_output instead of subprocess.get_output;
- use subprocess.check_call instead of manually checking return code;
- don't set always-overridden SSHD_PORT_NUMBER in .mk file;
- explicitly set sshd options on commandline instead of relying on host
/etc/sshd/sshd_config;
- let sshd listen only on localhost;
- user internal sftp server;
- disable BACKUP_SITE, no network is supposed to be accessed;
- remove the -bad and -nohash versions;
- rename {sftp,scp}-good to plain {sftp,scp};
- move the sftp and scp packages into a single "ssh" external.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add Secure File Transfer Program (SFTP) support using a simple wrapper.
SFTP is a common protocol used to transfer files securely between
enterprises, but it is not currently supported in Buildroot because all
of the packages are usually available via HTTP, git or some other
download method.
SFTP is similar to FTP but it preforms all operations over an encrypted
SSH transport using a specific protocol. This is unlike ftps, which is
traditional FTP over an SSL/TLS connection.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Michael Drake <michael.drake@codethink.co.uk>
[Arnout:
- update documentation with sftp everywhere scp is mentioned;
- rename "verbose" variable to "quiet";
- print the sftp command, similar to wget and scp helpers.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Install the openssh-server package into the test container. This
package, as well as its dependency openssh-client, is required to test
SCP and SFTP download methods on the localhost, as if these tools were
already installed on the user's machine.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
It wasn't immediately obvious to me what the two Buildroot base test
classes were for, so add docstrings to explain the differences between
BRConfigTest and BRTest.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This package was initially requested by José Pekkarinen, so he is
assigned as the maintainer for it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
commit b3c66481e1 replaced RISC-V LP64
bootlin toolchains by RISC-V LP64D. The config symbols
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE and
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_STABLE were marked as legacy.
Those changes were not reflected in the autobuild toolchain configs in
support/config-fragments/autobuild/bootlin-riscv64-{glibc,musl}.config
When testing a package with the command:
./utils/test-pkg --all --package somepackage
bootlin-riscv64-{glibc,musl} toolchain are always skipped. The build
logfile contains:
[...]
Value requested for BR2_TOOLCHAIN_EXTERNAL_BOOTLIN not in final .config
Requested value: BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
Actual value:
Value requested for BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE not in final .config
Requested value: BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE=y
Actual value: # BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE is not set
This commit update the autobuild config fragments for RISC-V 64bit
toolchains so they can be used by test-pkg.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
-E flag instructs patch to remove empty files. However, in some cases
empty files are essential. If they are missing, build could be broken
or other bad things can happen.
Note that empty files are still removed when their headers are properly
formattedo: timestamp set to 1970-01-01T00:00:00Z, destination set to
/dev/null.
Signed-off-by: Andrey Nechypurenko <andreynech@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, we have two functions that build a comma-separated list
of items; one is double-quoting the items, while the other is
single-quoting them. Their naming is not very consistent.
Besides, in a followup change, we will need to build a comma-separated
list of items that are already double-quoted.
Introduce a macro that does just build a comma-separated list, and
use that in the two other macros; rename the existing macro so the
naming is consistent.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Following the releases of 2021.11 Bootlin toolchains, this commit
represents the result of re-running the gen-bootlin-toolchains script.
The only part that isn't auto-generated are the contents of
Config.in.legacy, which account for the replacement of the RISC-V LP64
toolchain by RISC-V LP64D toolchains.
The complete set of runtime test cases was verified on Gitlab CI:
https://gitlab.com/tpetazzoni/buildroot/-/pipelines/437767674
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
toolchains.bootlin.com no longer provides a LP64 RISC-V 64-bit
toolchain, but a more useful LP64D RISC-V 64-bit toolchain. Of course,
the old tarballs remain available, but no new versions of the LP64
toolchain will be produced.
This commit reflects this change in the gen-bootlin-toolchains script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Already supported:
- Pushing a branch called "<foo>-defconfigs" tests all defconfigs.
- Pushing a branch called "<foo>-defconfig-<defconfig-name>" will
test one particular defconfig
This commit adds support for:
- Pushing a branch called "<foo>-defconfigs-<pattern>" which will
test all defconfigs whose name start with the pattern. For example
"<foo>-defconfigs-qemu_" will test all Qemu defconfigs
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
support/testing/tests/package/test_php_lua.py:35:1: E302 expected 2 blank lines, found 1
Add the missing line before class definition.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The php-pam package provides a PHP PAM (Pluggable Authentication
Modules) integration.
https://pecl.php.net/package/PAM
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The php-lua package provides a PHP extension that embeds the lua
interpreter and offers an OO-API to lua variables and functions.
https://pecl.php.net/package/lua
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Two patches are present and were retrieved from the following
upstream pull request in order to support PHP8:
https://github.com/laruence/php-lua/pull/47
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
APCu is an in-memory key-value store for PHP.
Keys are of type string and values can be any PHP variables.
APCu only supports userland caching of variables
https://pecl.php.net/package/APCU
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It is possible that some users of buildroot have put it in a repository
and call into it from another Makefile such as:
.DEFAULT:
$(MAKE) O=$(abspath $(O)) -C buildroot $(@)
This technique works well except that Make tells us that it changes into
the buildroot directory:
make[1]: Entering directory 'buildroot'
Because this line doesn't have an equals within it, python raises a
ValueError exception within pkg-stats.
This patch has python tell the invoked make not to print directories
Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Like for the github helper, add some tests to test the download of
Gitlab's generated tarball.
[1] f83826c90d
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Back in 2013, a github download helper has been introduced to cope with
changes in github download-URL's [1][2].
Since then a testing infrastructure has been introduced in Buildroot
but no tests has been added to check if the github download helper is
still working.
It was reported recently [3] that the github helper doesn't work anymore
using tags. Buildroot is not the only project having the issue, see
Github feedback discussions [4].
Add tests for direct archive download (archives uploaded by maintainers),
download from a git tag and git hash using the github helper.
Make sure that Buildroot doesn't use BR2_BACKUP_SITE
(http://sources.buildroot.net).
[1] https://bugs.busybox.net/show_bug.cgi?id=6302
[2] c7c7d0697c
[3] https://bugs.busybox.net/show_bug.cgi?id=14396
[4] https://github.com/github/feedback/discussions/8149
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When calling 'printvars', the 'suitable-host-package' macro is printed
(a macro is just a variable like the others, after all, just with some
parameters). Because it is printed as a variable, it is missing its
parameters, but it still tries to evaluate the $(shell) construct.
This causes spurious warning:
make[1]: support/dependencies/check-host-.sh: Command not found
Only try and call the script if there is actually a tool to check for.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Now that our pipelines are using the Docker image from the Gitlab
registry, there is no longer any reason to push the image to the
Docker hub.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
On a properly setup machine, it is totally useless to use sudo to run
docker; it is very bad practice. Instead, users really should add
themselves to the docker group.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The heuristic to extract the various variables of interest is pretty
crude: we filter on variables ending with certain suffixes (like
'%_VERSION' to get the version strings).
However, in doing so, we may dump variables that are not actual package
versions (especially with br2-external trees), and those may contain one
or more equal sign. And anyway, an actual package version string may
very well contain an equal sign too.
But the current situation is that the output of 'printvars' is split on
all equal signs, which will not fit in the 2-tuple we assign the result,
thus causing an exception.
Fix that by limiting to a single split.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As explained by Jörg [1], iteration with pairs() does not result in the
same order since luajit 2.1.
From [2]
"Table iteration with pairs() does not result in the same order?
The order of table iteration is explicitly undefined by the Lua
language standard. Different Lua implementations or versions may use
different orders for otherwise identical tables. Different ways of
constructing a table may result in different orders, too. Due to
improved VM security, LuaJIT 2.1 may even use a different order on
separate VM invocations or when string keys are newly interned.
If your program relies on a deterministic order, it has a bug.
Rewrite it, so it doesn't rely on the key order.
Or sort the table keys, if you must."
Note: The "luvi -v" return 255 even on success.
[1] http://lists.busybox.net/pipermail/buildroot/2021-November/627938.html
[2] https://luajit.org/faq.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the commit replacing moonjit by luajit [1] luvi doesn't work without
rng support enabled.
Switch to armv5 to use virtio-rng-pci on the qemu command line [2].
[1] 9450b53c8e
[2] https://git.buildroot.net/buildroot/tree/support/testing/infra/emulator.py?h=2021.08.1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently when a tag is added to the Buildroot git tree, the gitlab-ci
create a pipeline with several hundred of jobs (~750) to build all
defconfigs and execute the Buildroot testsuite.
However, there is only a limited number of gitlab-ci runner (9 runners)
and some jobs reach the timeout limit (24h) while waiting for a runner
[1]. Indeed, the Buildroot project doesn't use the Gitlab's shared
runners.
In addition to the pipeline created when a new tag is added to the
git repository, two pipelines are created each weeks to execute the
Buildroot testsuite (on monday [2]) and build all defconfigs (on
Thursday [3]).
At some point there are too many jobs waiting in gitlab due board
defconfigs builds. Indded a board defconfig requires a lot of time
(~30min) compared to other jobs in order to build a toolchain and a
kernel linux along with a basic rootfs. There is currently 262
defconfigs.
This is even worse when several pipelines are trigged at the same
time (new git tag and scheduled pipeline trigger).
In order to reduce the number of long jobs, don't build board
defconfigs with pipelines trigged on tag, keeping only the runtime
tests and the Qemu's defconfigs.
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/1758966541
[2] https://gitlab.com/buildroot.org/buildroot/-/pipelines/404035190
[3] https://gitlab.com/buildroot.org/buildroot/-/pipelines/401685550
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d815599e37)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes:
support/testing/tests/package/sample_python_unittest_xml_reporting.py:4:1: E302 expected 2 blank lines, found 1
support/testing/tests/package/sample_python_unittest_xml_reporting.py:8:1: E305 expected 2 blank lines after class or function definition, found 1
1 E302 expected 2 blank lines, found 1
1 E305 expected 2 blank lines after class or function definition, found 1
Signed-off-by: Pierre-Jean Texier <texier.pj2@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The test_jffs2 test fail for the same reason as test_ubi test with qemu >= 2.9
due to a qemu 2.8 bug. See commit d8447c38f5.
Divide the erase block size by two.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1687590514
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Based on initial work from Nicolas Carrier
<nicolas.carrier@orolia.com>, with the following additions:
- Updated to a newer version
- Added proper license file handling
- Added runtime test case
- Restricted to Python 3.x
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The php-pecl-dbus package provides a PHP extension for interaction
with D-Bus busses.
https://github.com/derickr/pecl-dbus
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The current Gitlab CI mechanism allows to trigger all tests in a CI
pipeline by pushing a branch named <something>-runtime-tests, or to
trigger a single test in a CI pipeline by pushing a branch name
<something>-tests.<name of test>.
However, there are cases where it is useful to run a suite of tests,
for example to run all tests in tests.init.test_busybox.
This commit makes that possible by extending the current semantic of
<something>-tests.<name of test> to not expect a complete test name,
but instead to accept all tests that starts with the given pattern.
This allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox.TestInitSystemBusyboxRo
like it was the case before. But it now also allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox
to run all Busybox tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a rudimentary test inspired from the "Using boto3" section in
the package README ([1]).
Note that it doesn't try to do anything with the instanciated
resource, as this would require a network connection when the test
runs.
[1]: https://github.com/boto/boto3
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a rudimentary test inspired from the "Using botocore" section in
the package README ([1]).
Note that it doesn't try to use the instantiated client, as this would
require a network connection when the test runs.
[1]: https://github.com/boto/botocore
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
On some developers machines, the default timeout (5 seconds) is not
enough for the test to succeed.
Increase it to 20 seconds, to let more time for the rsa keys to be
generated.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
A simple test that runs nmap twice to create the files scanme-1.xml and
scanme2.xml, then runs pyndiff on both files.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The pkg-stats script queries release-monitoring.org to find the latest
upstream versions of our packages. However, up until recently,
release-monitoring.org had no notion of stable
vs. development/release-candidate versions, so for some packages the
"latest" version was in fact a development/release-candidate version
that we didn't want to package in Buildroot.
However, in recent time, release-monitoring.org has gained support for
differentiating stable vs. development releases of upstream
projects. See for example
https://release-monitoring.org/project/10024/ for the glib library,
which has a number of versions marked "Pre-release".
The JSON blurb returned by release-monitoring.org has 3 relevant
fields:
- "version", which we are using currently, which is a string
containing the reference of the latest version, including
pre-release.
- "versions", which is an array of strings listing all versions,
pre-release or not.
- "stable_versions", which is an array of string listing only
non-pre-release versions. It is ordered newest first to oldest
last.
So, this commit changes from using 'version' to using
'stable_versions[0]'.
As an example, before this change, pkg-stats reports that nfs-utils
needs to be bumped to 2.5.5rc3, while after this patch, it reports
that nfs-utils is already at 2.5.4, and that this is the latest stable
version (modulo an issue where Buildroot has 2.5.4 and
release-monitoring.org has 2-5-4, this will be addressed separately).
Note that part of this change was already done in commit f7b0e0860, but
it was incomplete.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The pkg-stats scripts tries to match packages against
release-monitoring.org in two ways:
- First by using the "Buildroot" distribution registered on
release-monitoring.org, in which we have added a lot of mappings
between Buildroot package names and release-monitoring.org package
names. If there is a match using this distribution, the package
status is RM_API_STATUS_FOUND_BY_DISTRO, which means that the
resulting HTML has a "found by distro" statement.
- Then, if the first solution didn't work, by using the pattern
matching, as done in the check_package_get_latest_version_by_guess()
function.
However, there is a bug in this later case: it sets the package status
to RM_API_STATUS_FOUND_BY_DISTRO as well, while it should have been
RM_API_STATUS_FOUND_BY_PATTERN. Due to this bug, in the resulting HTML
file from a pkg-stats run, all packages are marked as "found by
distro" even the ones that are "found by guess".
This commit fixes that by setting the correct package status.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
For example with libpng: 1.6.37 instead of 1.7.0beta89
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: coalesce into a single line]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
A recent update of flake8 in CI introduced a new check E741. It
basically checks that variables are at least 3 characters long. Up to
now, however, we have used shorter names in some places - all of them
turn out to be "l" for a line of text.
Replace all those "l" variables with "line".
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/scripts/boot-qemu-image.py:47:21: E741 ambiguous variable name 'l'
support/scripts/check-dotconfig.py:20:38: E741 ambiguous variable name 'l'
support/scripts/size-stats:76:13: E741 ambiguous variable name 'l'
support/testing/tests/core/test_bad_arch.py:17:32: E741 ambiguous variable name 'l'
support/testing/tests/package/test_python_treq.py:10:30: E741 ambiguous variable name 'l'
support/testing/tests/toolchain/test_external.py:30:42: E741 ambiguous variable name 'l'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The dbus-next package uses the Python type annotation for dbus types. This is
not compatible with the python typing assumption that flake8 makes.
Exclude F821 from this line.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/testing/tests/package/sample_python_dbus_next.py:17:36: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:48: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:56: F821 undefined name 's'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the ubi/ubifs test has been introduced, it's not possible to
boot the same ubi image twice [1]:
"TODO: if you boot Qemu twice on the same UBI image, it fails to
attach the image the second time, with "ubi0 error:
ubi_read_volume_table: the layout volume was not found"."
For some reason, the kernel corrupt the ubi image if the ubifs
rootfs is mounted with write access. Use a custom config file
to mount the rootfs readonly (vol_type=static). Doing so requires
to add the flash size (vol_size=64MiB).
At least it allows to boot several times the same ubi image.
[1] bf4a6490e4
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The current ubi/ubifs test (test_ubi.py) rely on a Qemu bug present in
2.8.0 that was fixed in Qemu 2.9.0 [1]. The ubi/ubifs settings is
updated to run with Qemu >= 2.9.0 using the new multiple chip handling.
If needed, the old behavior can be enabled using the pflash01 property
"old-multiple-chip-handling" [2].
The issue was not detected until now since we are sill using an old
qemu (2.8 from Debian stretch) for testing in gitlab (using the
Buildroot Docker image used by gitlab-ci.yml).
First the logical eraseblock size (LEB) must be updated to the value
0x3ff80 reported by the kernel when using qemu >= 2.9.0.
UBIFS (ubi0:0): Mounting in unauthenticated mode
UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: LEB size mismatch: 524160 in superblock, 262016 real
UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: bad superblock, error 1
But the system is still failing to boot:
UBIFS error (ubi0:0 pid 1): ubifs_scan: garbage
UBIFS error (ubi0:0 pid 1): ubifs_recover_master_node: failed to recover master node
ubifs is reading garbage since Qemu >= 2.9.0 report a sector
length per device divided by the number of devices (see commit [1]).
The kernel detect two flash devices (dmesg):
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
Divide the physical eraseblock (PEB) size by two.
Tested with qemu 2.9.0, 5.1.0.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1543100932
[1] https://git.qemu.org/?p=qemu.git;a=commitdiff;h=feb0b1aa11f14ee71660aba46b46387d1f923c9e
[2] http://lists.busybox.net/pipermail/buildroot/2021-September/622069.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Adding the Image format on the Qemu command line avoid this warning:
"WARNING: Image format was not specified for 'output/TestUbi/images/rootfs.ubi' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions."
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit e6ee07f41a (package/python-flask-expects-json: new package)
added a non-functional test case that, as noticed by Edgar, fails with:
AssertionError: '%{http_code}' != '200'
That's because the % sign is self-escaped, à-la C, in the first part
of the command, probably to avoid its being %-formatted. But only the
second part of the command is %-formatted, so we do not need to
self-escape % in the first part.
Additionally, since eb3ee3078a (support/testing/infra/emulator.py:
prevent the commands from wrapping), we no longer need to play tricks
with commands that are too long to fit on the first line of the shell
prompt.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The lua-augeas package provides a Lua binding for augeas
https://github.com/ncopa/lua-augeas
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
dtbocfg, which stands for Device Tree Blob Overlay Configuration
File System, was developed to serve as a userspace API of Device
Tree Overlay.
https://github.com/ikwzm/dtbocfg
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 516b837002)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The list of tests is as follows:
TestGdbHostOnlyDefault: build just minimal host-gdb, default version
TestGdbHostOnlyAllFeatures: build host-gdb, default version, with all
features enabled (TUI, Python, simulator)
TestGdbserverOnly: build just target gdbserver, default version
TestGdbFullTarget: build just target gdb, default version
TestGdbHostOnly9x: build minimal host-gdb, 9.x version
TestGdbHostGdbserver9x: build minimal host-gdb 9.x + gdbserver
TestGdbHostGdbTarget9x: build minimal host-gdb 9.x + full gdb
TestGdbHostOnly11x: build minimal host-gdb, 11.x version
TestGdbHostGdbserver11x: build minimal host-gdb 11.x + gdbserver
TestGdbHostGdbTarget11x: build minimal host-gdb 11.x + gdb
TestGdbArc: build minimal host-gdb + gdb + gdbserver, for the special
ARC architecture version
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Traditional VT-10x terminals (and their emulators) [0] have a "magic
margins" feature that enables the last character position to be updated
without scrolling the screen: whenever a character is printed on the
last column, the cursor stays over the character, instead of moving to
the next line.
The Busybox shell, ash, attempts to defeat this feature by printing
CR,LF right after echoing a character to the last column.[1] This
doesn't play well with emulator.py. The run() method of the Emulator
class captures the output of the emulated system and assumes the first
line it reads is the echo of the command, and all subsequent lines are
the command's output. If the line made by the command + shell prompt is
longer than 80 characters, then it is echoed as two or more lines, and
all but the first one are mistaken for the command's output.
We fix this by telling the emulated system that we are using an
ultra-wide terminal with 29999 columns. Larger values would be ignored
and replaced by the default, namely 80 columns.[2]
[0] https://vt100.net/docs/vt100-ug/chapter3.html - DECAWM
[1] https://git.busybox.net/busybox/tree/libbb/lineedit.c?h=1_34_0#n412
[2] https://git.busybox.net/busybox/tree/libbb/xfuncs.c?h=1_34_0#n258
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Co-authored-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Compiling on Ubuntu 20.04 generates this:
./util.c: In function ‘file_write_dep’
./util.c:54:18: warning: ‘..config.tmp’ directive writing 12 bytes into a region of size between 1 and 4097 [-Wformat-overflow=]
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~
./util.c:54:2: note: ‘sprintf’ output between 13 and 4109 bytes into a destination of size 4097
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
and similar warnings on confdata.c, lines 778, 989, 995, 1000, 1007,
1040, 1046 and 1054. Avoid the warnings by enlarging the destination
buffer of fprintf().
Normally, we want changes to kconfig to be reflected by patches in
support/kconfig/patches. This makes it easier to resync with upstream
kconfig. However, in this case, everything that is changed here is
already changed completely (and differently) upstream, so there is no
added value in keeping the patch.
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
Add dedicated Grub2 builtin config for the EFI compatible cases.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.
However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:
Traceback (most recent call last):
[...]
emulator.login()
File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.
We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.
[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- don't change encoding, use codec_errors
- rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When Grub2 is build it is configured only for one boot set-up, BIOS Legacy,
EFI 32 bit or EFI 64 bit. It can not deal with several boot set-up on the
same image.
This patch allows to build Grub2 for different configurations simultaneously.
To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI
BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more
reasonable to just extend the grub2 package into 3 packages.
We can no longer use autotools-package as a consequence of this multi-build, and
we have to resort to generic-package and a partial duplication of
the autotools-infra. Grub2 was already using custom option like --prefix or
--exec-prefix so this won't add much more weirdness.
We use a GRUB2_TUPLES list to describe all the configurations selected.
For each boot case described in the GRUB2_TUPLES list, it configures and
builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.
We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the
BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- keep sub-options properly indented
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
It's requirement has been removed in previous commit.
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
`which' has been discontinued after 2.21 release in 2015 due this (git
repository is empty [1]) and version shipped in Debian produces warning
[2]:
/usr/bin/which: this version of `which' is deprecated; use `command -v' in scripts instead.
`command is POSIX [3] and supported on all common shells (bash, zsh,
dash, busybox sh, mksh).
Patch tested on dash as the default shell.
[1] https://git.savannah.gnu.org/cgit/which.git
[2] 3a8dd10b45
[3] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS, as this commit is based on initial work
from Nicolas.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds a new package called python-flask-expects-json, which
also to validate the JSON blurbs submitted to a Flask web
application. A runtime test is added as well, making sure that the
package minimally works with an example Flask application.
The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS file, as Nicolas is also interested in
this package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds a test for python-flask package. As we are about to
add python-flask-expects-json together with a test, it made sense to
also add a test for python-flask itself.
As far as the DEVELOPERS file is concerned, the test files are added
both to the existing maintainer of package/python-flask, as well as to
myself.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit bumps the python-smmap2 and python-gitdb2 packages in
lockstep, as the new version of gitdb2 requires a newer version of
smmap2, but the current version of gitdb2 cannot work with the newer
version of smmap2 (sigh).
Also, upstream the projects have been renamed: gitdb2 is now named
gitdb on PyPi (see https://pypi.org/project/gitdb2/) and smmap2 is now
named smmap (https://pypi.org/project/smmap2/). However, to avoid
needless churn, we don't rename the Buildroot packages, but that
rename is visible in the name of the tarballs being downloaded.
Also, since version 4.0.0, smmap supports only Python 3.x, so we add a
dependency on Python 3.x and drop the test case of gitdb2 on Python
2.x.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Starting with Qemu 6.1.0, gcc 7.5 is needed to build.
Since we build host-qemu package for qemu defconfig, we have to
upgrade to (at least) Debian buster that provide gcc 8 as host compiler.
While testing this upgrate, the test_edk2 failed since it actually
requires Qemu >= 4.1.0 to support arm SBSA reference machine [1].
Debian Buster only provide Qemu 3.1.
Finally, upgrade to Debian bullseye but it requires some linux
kernel version bump in several defconfigs since host gcc is based
on gcc-10 [2].
[1] https://git.qemu.org/?p=qemu.git;a=commit;h=64580903c2b3aee08d74d64e6248a313b246cb69
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38
Signed-off-by: Romain Naour <romain.naour@gmail.com>
[Arnout: don't install python]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).
Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import
Tested with python3 -m py_compile.
[1] https://www.python.org/doc/sunset-python-2/
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
To avoid spending some time to build the x86_64 toolchain (~20min),
switch to corei7 cpu (Nahalem) and use the prebuilt Bootlin toolchain.
We have to use the "stable" Bootlin toolchain to use the same kernel version
for the toolchain kernel headers and the running kernel.
With the "bleeding-edge" toolchain we have the "kernel too old" issue
(running kernel 4.19 vs kernel headers 5.4)
Runtime tested locally.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Switch from the Buildroot internal toolchain for armv5 to
the prebuilt Bootlin external toolchain.
The test doesn't require to build a toolchain, there was
no prebuilt glibc toolchain recent enough at the time this
test has been introduced.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This test already use builtin kernel provided by the testsuite infra:
self.emulator.boot(arch="armv7",
kernel="builtin",
options=["-initrd", img])
But a second kernel is build from the its defconfig. This second kernel
is not used by the test.
The TestRust (using BR2_PACKAGE_HOST_RUST=y) is really long to build,
save some cpu time by removing the kernel build.
This unused kernel (based on 4.11.3 release) doesn't even build with
host gcc >= 10.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.19.79 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
The issue was fixed in 4.19.114 [2]
Bump to the latest 4.19.x version.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 5.5.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
But we can't just update this test to the next linux kernel LTS 5.10.y since
the minimum gcc version has been updated to gcc 4.9 since 5.8 kernel [2]
and the Sourcery CodeBench ARM 2014.05 is used (gcc 4.8 based).
Enable arm cortex A9 and VFP support to switch to the ARM arm prebuilt
toolchain (the Bootlin toolchain could be used).
While at it use the prebuilt buildin kernel for the vexpress target
recently updated to 5.10.7.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1564202094
[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6ec4476ac82512f09c94aff5972654b70f3772b2
[3] 3cf2782906
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This patch adds runtime testing of the OCI archive created by the
sloci scripting. It launches a containerd instance, imports, and
runs the OCI container.
The existing QEMU AARCH64 kernel config was extended to enable common
options used by a container runtime (cgroup and overlayfs).
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
[Arnout: adapt file name which is arm64 now; add to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Version 1.46.3 had a regression, which meant the file that would store
the filesystem image had to pre-exist, or mkfs.ext2 would fail to
generate the filesystem:
mkfs.ext4: No such file or directory while trying to determine
filesystem size
The regression was fixed upstream, and is now part of the 1.46.4
release, so bump to that release.
Fixes: #14196
Additionally, as noticed by Romain, the defaults settings for generating
"small" filesystems have changed: the inode size has been increased fom
128 to 256 bytes in 1.46.4 [0]. This causes the number of inodes to
diverge slightly from the requested number; instead of 8 more inodes,
there are now 8 fewer than requested.
Adapt our test accordingly.
[0] a23b50cdb5
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reported-by: Kevin Tomary <kevin.tomary@hotmail.com>
Reported-by: Leon de Rooij <leon@exquisip.nl>
Reported-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Michael Walle <michael@walle.cc>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This tests build a bogus package that installs a binary built for the
host architecture into $(TARGET_DIR), which should cause a build
failure, at least as long as the host architecture isn't ARM.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
[yann.morin.1998@free.fr: drop uneeded subprocess import to fix flake8]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some tests will need to grep through the build log to verify that some
features are working are expected. In order to allow them to open the
build log, we provide a new function called log_file_path(), which
returns the path to the log file if available.
We also use this function in open_log_file().
Note that open_log_file() cannot be used directly to grep through the
log file at the end of a build: because it opens in "a+" mode, it
greps starting from the end of the file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The gitlab-ci support in test-pkg allows to parallelize the test-pkg
work into several gitlab jobs. It's much faster than local serialized
testing.
To trigger this, a developer will have to add, in the latest commit of
their branch, a token on its own line, followed by a configuration
fragment, e.g.:
test-pkg config:
SOME_OPTION=y
# OTHER_OPTION is not set
SOME_VARIABLE="some value"
This configuration fragment is used as input to test-pkg.
To be able to generate one job per test to run, we need the list of
tests in the parent pipeline, and the individual .config files (one per
test) in the child pipeline. We use the newly-introduced --prepare-only
mode to test-pkg, and collect all the generated .config files as
artefacts; those are inherited in the child pipeline via the
"needs::pipeline" and "needs::job" directives. This is a bit tricky,
and is best described by the Gitlab-CI documentation [0].
We also list those .config files to generate the actual list of jobs to
run in the child pipeline.
Notes:
- if the user provides an empty fragment, this is considered an error:
indeed, without a fragment (and the package name), there is no way
to know what to test;
- if that fragment yields an empty list of tests, then there is
nothing to test either, so that is also considered an error.
[0] https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
- split the change to test-pkg to its own patch
- generate the actual yml snippet in support/scripts/generate-gitlab-ci-yml,
listing the .config files created by test-pkg
- some code-style-candies...
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This version bump is needed to pass the ATF test with
hardening option enabled (-fstack-protector-strong)
With the version v2.2, ATF fail due to undefined references:
./build/juno/release/bl2u/arm_tzc400.o: In function `arm_tzc400_setup':
arm_tzc400.c:(.text.arm_tzc400_setup+0x10): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0x18): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xb8): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xcc): undefined reference to `__stack_chk_fail'
Since commit ccac9a5bbb, Buildroot no
longer forces ENABLE_STACK_PROTECTOR. However, we rely on the ATF build
system to handle it correctly, and this wasn't the case in v2.2.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1524842591
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
gcc 10.x is now used by default but the kernel 4.19 used by
test_docker_compose doesn't build with it.
Bump the kernel to 4.19.204 release that contains a lot of
fixes for newer gcc.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
check_network() must check the error code of the command
used to check the network configuration with the value
passed as argument "exitCode".
But this argument is ignored since this commit [1].
Revert the last change of check_network().
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1522848308https://gitlab.com/kubu93/buildroot/-/jobs/1522848306
[1] afc1ed4d51
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The python2 support has been removed since the python-idna bump to version 3.2 [1]
[1] 0c7e30b43a
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d5127a4de7)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In a followup commit, the make command used to log and display the last
lines on error will be used in another job.
Factorize it by introducing .run_make template.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Python-RSA is a pure-Python RSA implementation.
Signed-off-by: Raphaël Mélotte <raphael.melotte@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Parallelizes locale generation based on `BR2_JLEVEL` setting.
Locale generation always runs during the finalize stage and can consume
a significant amount of time. Parallelizing it greatly reduces that time
on multi-core machines.
To parallelize it, we first invoke `localedef` for every locale in
parallel with the `--no-archive` option. This creates the intermediate
locale data instead of writing to the finally archive directly.
Then, we invoke `localedef` again once to create the archive from the
intermediate compiled locale data files.
We have to do it this way because `localedef` does not do any locking
when writing to the archive file, so calling it without `--no-archive`
concurrently could result in a corrupt archive file or an archive file
that is missing some locales.
While we're at it, make two additional improvements:
- Remove locale-archive before adding to it. Otherwise, repeated
applications of target-finalize will keep on growing the file.
- Sort the locales when creating locale-archive so its contents are
reproducible.
We use `find` to collect the installed locales rather than LOCALES. This
makes it possible for something else (skeleton, overlay, custom package)
to create and install additional locales and still have them added to
locale-archive.
Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
[Arnout:
- Remove -j$(PARALLEL_JOBS), it's already part of $(MAKE)
- Remove HOST_DIR, TARGET_DIR, STAGING_DIR, they're already exported
- Extend commit message
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This test script tests polkit with and without systemd.
The Systemd test does the following:
- The brtest user attempts to restart the systemd-timesyncd service and is
denied.
- A systemd-timesyncd-restart.rules file provided by polkit-rules-test
is copied from /root/ to /etc/polkit-1/rules.d
- The brtest user attempts to restart the systemd-timesyncd service and should
now succeed.
The initd test does the following:
- The brtest user attempts to run the test application "hello-polkit" with the
command "pkexec hello-polkit" and is denied.
- A hello-polkit.rules file provided by polkit-rules-test is copied from /root/
to /etc/polkit-1/rules.d
- The brtest user attempts to re-run the test hello-polkit binary with
"pkexec hello-polkit" and succeeds.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 2eaa6d0f36 (boot/uboot: fix uboot building host tools on x86
architecture) added use of $(PKG_CONFIG_HOST_BINARY), but forgot to add
the corresponding build-ordr dependency.
Add this missing depenency now.
Additionally, the associated test had an explicit host pkgconf enbled in
its configuration. This is superfluous now that uboot properly depends
on host-pkgconf, so drop that from the test.
Note: it hapenned to work, because host-pkgconf, when explicitly enabled
in the configuration, and without per-package directories, would build
before uboot and thus be available. This would fail with PPD, though,
and thus would break for TLPB.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The make all command run the tools/makefile on the process.
This makefile use "pkg-config" command to support static link.
The issue is the use of pkg-config configured for crosscompiling
to build binaries tools for host architecture.
To fix it, I add pkg-config environment variable to configure it for host.
Add a test to avoid future regress on the build of U-boot.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- fix mixed space-TAB indentation
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit 3cf2782906 (support/testing/infra/emulator.py: update pre-built
kernels) bumped the default kernels used by the testing infra.
However, the newer armv7 kernel (at least) no longer has support for
lz4-compressed squashfs filesystems.
This breaks the squashfs test:
Filesystem uses "lz4" compression. This is not supported
List of all partitions:
1f00 131072 mtdblock0
(driver?)
1f01 32768 mtdblock1
(driver?)
b300 2048 mmcblk0
driver: mmcblk
No filesystem could mount root, tried:
squashfs
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,0)
Updating the kernel again is a little bit cumbersome, while fixing the
actual test is relatively trivial, so this is what we do: we switch
over to lzo, which is supported by the new kernel:
# zcat /proc/config.gz |grep SQUA
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_ZLIB=y
# CONFIG_SQUASHFS_LZ4 is not set
CONFIG_SQUASHFS_LZO=y
# CONFIG_SQUASHFS_XZ is not set
While at it, also drop the superfluous line disabling gzip compression:
it is part of a choice, so enabling one (lzo here) forcibly disables the
others (of which gzip).
Fixes: 3cf2782906
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Will avoid the following warning:
WARNING: Image format was not specified for
'/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and
probing guessed raw. Automatically detecting the format is
dangerous for raw images, write operations on block 0 will be
restricted. Specify the 'raw' format explicitly to remove the
restrictions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
All the tests that are using if=sd as a Qemu options are changed to
use infra.img_round_power2() instead of simply extending the size of
the image to the next MB boundary, which is not longer sufficient with
Qemu >= 5.1.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: drop now-useless imports]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since Qemu 5.1, SD card images must have a size that are a power of
two. While some filesystem (such as ext2/3/4) allow to specify the
expected size of the filesystem, others such as SquashFS do not have
this capability.
We were already extending the size of such images to the next 1 MB
boundary using "truncate -s %1M", but that is no longer sufficient. So
instead, we introduce a helper function that extends the size of an
image to the next power of two.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- use f.trunctate() rather than subprocess.call([truncate,...])
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since Qemu 5.1, the SD card size must be a power of two, so the
default size for ext2/3/4 filesystem images of 60 MB is not
suitable. Since 16 MB is used for the Ext4 test, let's use the same
value for the other tests as well (ext2, ext2r1, ext3). Without this
change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >=
5.1.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The BRTest() class implements an assertRunOk() method that does the
very common work of running a command inside the emulator, and
checking that it is successful.
This commit changes all locations where this .assertRunOk() method can
be used, instead of open-coding the same logic.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Bump U-Boot version in TestATFAllwinner. Updating U-Boot version to
2021.04 requires the following two changes.
First, after switching to binman, u-boot.itb is no more generated for
64-bit sunxi boards. Combined u-boot-sunxi-with-spl.bin image should
be used instead. This image contains SPL, U-Boot, and FIT image,
where FIT image contains other binaries such as BL31 and SCP.
Second, new U-Boot enables support for System Control Processor (SCP)
firmware. SCP firmware is included by default into FIT image in the
combined u-boot-sunxi-with-spl.bin binary. When SCP is not available
or not needed, it should be explicitly disabled by pointing to an
empty file. Support for Allwinner SCP firmware is not yet available
neither in Buildroot nor in mainline kernel. So disable it for now
using custom U-Boot build options.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Switch to mainline TF-A that provides basic support for H5 and A64.
Note that Allwinner platform layer in TF-A does not provide support
for GCC stack protection, so make sure to disable this TF-A feature.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
As this version brings support for kernel up to 5.12, we update the
test cases to use the 5.12 kernel.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This patch implements a simple test in which a dummy file system image
is created, then `bmaptool create` and `bmaptool copy` are used to copy
it to another file.
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: several reworks, add myself to DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The pre-built vexpress kernel used by the testing infrastructure is a
4.0.0 kernel, which is getting old to be used with reasonably recent
toolchains.
This commit updates the pre-built kernels for both the versatile and
vexpress machines to 5.10.7 (they have already been put online).
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The br-arm-internal-glibc.config is generally used as a configuration
to test the bleeding edge versions of components. However, it has been
lagging behind somewhat, so let's bring it up-to-date:
- Binutils 2.36.x
- GCC 11.x
Let the fun begin in the autobuilders!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This module is only partially compatible with lzlib (which is no longer
maintained).
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: amend commit log about limited compatibility]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
- If a package doesn't have any versioning, ignore and state that
- If a package is virtual, CVE=ignore and CPE state virtual
- For any of these NA cases, don't provide search link and color box
green
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
has_valid_infra() is incorrectly named; it probably should be named
is_actual_package(), and has_valid_infra() would be changed to
actually represent having an actual infra.
This resolves packages reporting as having no valid package infra and
cleans up reporting cases of CPE and CVEs where there isn't a valid version
or package definition outside Buildroot
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently a verified CPE reports the following if versions are not found
cpe:2.3🅰️qemu:qemu:5.2.0:*:*:*:*:*:*:*
CPE identifier unknown in CPE database (Search)
This patch clarifies the report to state the 'version' is unknown instead
of the 'identifier'.
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, patches with renames are refused, as they reqire patch 2.7
or newer. So far, we did not require that version because it was too
recent to be widely available.
But patch 2.7 has been released in 2012, almost 9 years ago now; it is
old enough that we can start relying on it.
Add a check that patch is GNU patch 2.7 or newer, and so drop the common
check for patch, and drop the check about renames in apply-patches.sh.
Signed-off-by: Ryota Kinukawa <pojiro.jp@gmail.com>
[yann.morin.1998@free.fr:
- drop common check
- shorten variable names
- drop now-incorrect comment about busybox w/desktop
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since [1], PIC/PIE is enabled by default but the TestRelroPartial
test expect implicitely PIC/PIE being disabled.
Disable PIC/PIE from the config fragment provided by
TestRelroPartial.
[1] 810ba387be
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661757
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
From [1]
"Xorg does not implement real dynamic linking and requires that its
modules get loaded in the right order."
From /var/log/Xorg.0.0.log:
(II) LoadModule: "modesetting"
(II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
(EE) Failed to load /usr/lib/xorg/modules/drivers/modesetting_drv.so: /usr/lib/xorg/modules/drivers/modesetting_drv.so: undefined symbol: shadowRemove
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661899
[1] https://forums.gentoo.org/viewtopic-p-8245578.html#8245578
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the mesa3d bump to version 21.0.3 [1], the
BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST option is not supported anymore
since the mesa DRI swrast driver has been removed upstream
So, switch to Gallium swrast.
[1]15a2f9b819806d38a7d8172a20f80130b1d60e63
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Boot a QEMU sbsa-ref machine with ATF, EDK2, GRUB2 and a minimal
kernel. This is a simple but effective test of a compliant setup.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When doing analysis it is helpful to be able to view what CVE have
been patched / diagnosed to not apply to Buildroot. This exposes
that list to the reporting and prevents a step where you have to
dig into the .mk's of a pkg to check for sure what has been
ignored.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: only set background if there are ignored CVEs]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
For cases of a CPE having a unknown version or when there hasn't
been a CPE verified, proposed a search criteria to help the
user research an update.
(libcurl has NIST dict entries but not this version)
cpe:2.3🅰️haxx:libcurl:7.76.1:*:*:*:*:*:*:*
CPE identifier unknown in CPE database (Search)
(jitterentropy-library package doesn't have any NIST dict entries)
no verified CPE identifier (Search)
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: fix flake8 issues]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This script queries the list of CPE IDs for the packages of the
current configuration (based on the "make show-info" output), and:
- for CPE IDs that do not have any matching entry in the CPE
database, it emits a warning
- for CPE IDs that do have a matching entry, but not with the same
version, it generates a snippet of XML that can be used to propose
an updated version to NIST.
Ref: NIST has a group email (cpe_dictionary@nist.gov) used to
recieve these version update and new entry xml files. They do
process the XML and provide feedback. In some cases they will
propose back something different where the vendor or version is
slightly different.
Limitations
- Currently any use of non-number version identifiers isn't
supported by NIST as they use ranges to determine impact
of a CVE
- Any Linux version from a non-upstream is also not supported
without manually adjusting the information as the custom
kernel will more then likely not match the upstream version
used in the dictionary
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr:
- codestyles as spotted by Arnout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The python2 support has been removed since the python-colorzero bump version to 2.0.
[1] 73bf3292e1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The python2 support has been removed since the python-colorzero bump version to 2.0.
Remove the gpiozero test with python2
[1] 73bf3292e1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 76b4f9e9b6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Create a new user 'sudotest' to validate that sudo really works (i.e.
properly has setuid).
Creating the user and adding it to sudoers is done at runtime, otherwise
we'd need to add extra files to the config which complicates things a
little bit.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.
Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Test that the TAICLOCK and TCP servers are working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that s6-rc service database compilation is working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that a few basis utilities are working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that directory scanning and supervision is working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: properly indent, and use textwrap to dedent again.]
Test that the interpreter can run a basic command.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Commit 54d3d94b6e broke the 'hg' download
method, in a similar way as it broke the 'git' download method (later fixed
with commit b70ce56651), by introducing extra
output on stdout in a case where the output is redirected.
In the case of 'hg', the 'hg archive' step uses shell redirection rather
than directly letting hg write the output file, since commit
76b51f90c0.
As a result, the extra print added by the _hg function is prepended to the
actual archive, causing an invalid archive.
Fix by using the _plain_hg function instead. The disadvantage is that the
command for 'hg archive' is no longer printed.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
ijson < 2.5 (as available in Debian 10) use the slow python backend by
default instead of the most efficient one available like modern ijson
versions, significantly slowing down cve checking. E.G.:
time ./support/scripts/pkg-stats --nvd-path ~/.nvd -p avahi --html foobar.html
Goes from
174,44s user 2,11s system 99% cpu 2:58,04 total
To
93,53s user 2,00s system 98% cpu 1:36,65 total
E.G. almost 2x as fast.
As a workaround, detect when the python backend is used and try to use a
more efficient one instead. Use the yajl2_cffi backend as recommended by
upstream, as it is most likely to work, and print a warning (and continue)
if we fail to load it.
The detection is slightly complicated by the fact that ijson.backends used
to be a reference to a backend module, but is nowadays a string (without the
ijson.backends prefix).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
OpenZFS is an advanced file system and volume manager which was originally
developed for Solaris and is now maintained by the OpenZFS community. This
repository contains the code for running OpenZFS on Linux and FreeBSD.
http://zfsonlinux.org/
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
[me:
- fix test case on how to use a pre-built toolchain
- reorder the test case config
- add test case with glibc
- drop superflous test timeout override
- only select libtirpc when C library lacks native RPC
- drop unused ZFS_MODULES variable
- drop ZFS_CPE_ID_PREFIX and ZFS_AUTORECONF_OPTS which are defaults
- drop NLS options, already set in a generic manner
- drop incomplete/improper sysvinit support
- some cosmetics
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bb10b0dfe6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a --transform expression is provided, it is by default also applied
to the target of a symlink.
When we create tarballs (from git or svn checkouts), we use a --transform
expression to replace the leading ./ with the package name and version.
This causes issues when a package contains symlinks that points to
./something, as the leading './' is also replaced.
Fix that by using the 'S' transformation scope flag, as described in the
tar manual:
https://www.gnu.org/software/tar/manual/html_node/transform.html#transform
In addition, several transformation scope flags are supported, that
control to what files transformations apply. These are:
‘r’ Apply transformation to regular archive members.
‘R’ Do not apply transformation to regular archive members.
‘s’ Apply transformation to symbolic link targets.
‘S’ Do not apply transformation to symbolic link targets.
‘h’ Apply transformation to hard link targets.
‘H’ Do not apply transformation to hard link targets.
Default is ‘rsh’ [...].
Fixes: #13616
This has been checked to not change any of the existing hash for any of
our git-downloaded package (some are host-only, hence the few fixups):
---8<---
$ m="$( git grep -l -E -- -br[[:digit:]]+.tar.gz boot package/ \
|awk -F/ '{print $(NF-1)}' \
|sed -r -e 's/(imx-mkimage|netsurf-buildsystem|prelink-cross|qoriq-rcw|vboot-utils)/host-\1/g' \
-e 's/$/-source/'
)"
$ make defconfig; make clean; BR2_DL_DIR=$(pwd)/trash-me make ${m}
---8<---
Note: it is unclear what the 'H' flag does nor how it works, because the
concept of "target of a hardlink" is not obvious; probably it has to do
with how tar internally detects and stores hardlinks. Since we do not
yet have any issue with hardlinks, just ignore the problem for now, and
postpone until we have an actual issue with a real test-case.
Signed-off-by: Jean-pierre Cartal <jpcartal@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- re-indent commit log
- add scriptlet to test existing hashes
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit 54d3d94b6e ("support/download: print
command used for download") broke the git and svn download helpers, because
these helpers have invocations of the _git/_svn commands where the exact
output matters.
For example for git, this would result in:
date: invalid date ‘GIT_DIR=.../dl/libyuv/git/.git git log -1 --pretty=format:%ci \n2019-04-12 17:48:45 +0000’
Detected a corrupted git cache.
Removing it and starting afresh.
Fix by splitting the _git function in two: _git and _plain_git.
The former echoes the command, and then calls the latter.
Most invocations use _git as before, but those cases where the output should
not be disturbed, directly call _plain_git.
For symmetry, all download helpers are aligned, even though only the git and
svn helpers were broken.
Fixes: #13631
Fixes:
http://autobuild.buildroot.org/results/c2f/c2fcd4aa6660e3c2f9c6f85646ca7dfe0db56040/
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: add bug report and autobuild failure]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Even though that most download commands actually print some output, like
progress indication or other messages, the actual command used is not. This
makes it hard to analyze a build log when you are not fully familiar with
the typical output of said log.
Update the download helpers to do just that, respecting any quiet/verbose
flag so that a silent make (make -s) does not get more verbose.
Note: getting rid of the duplication of the command in the script is not
straightforward without breaking support for arguments with spaces.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: use printf, not echo]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Most 'verbose' variable inside the download helpers actually mean 'quiet'.
I.e. they are assigned in case quiet operation is requested, and empty in
case of non-quiet operation. Using the name 'verbose' for such a variable is
confusing, especially when you want to test the variable on emptiness or
non-emptiness (in a subsequent commit).
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
In Makefiles, variables are split, filtered, and otherwise mangled on
a space as a separator. In a shell, they will also be split on TABs.
We split and filter and iterate on variables in a lot of places, and
most importantly, spaces in PATH is very seldom tested, if at all, so
a lot of packages will not be working properly in such a situation.
For example, the config.guess contains constructs that are not resilient
to a space in PATH:
PATH=$PATH:/.attbin ; export PATH
Also, our fakedate will iterate over PATH:
for P in `echo $PATH | tr ':' ' '`; do
Those are only two cases, but the first means basically all
autotools-based packages are susceptible to subtle breakage.
Furthermore, Buildroot itself does not support that the top-level or
output directories are in a path with spaces anyway.
So, instead of chasing all cases that might be potentially broken,
let's just detect the case and bail out, like we already do when PATH
contains a \n, or when it contains the current working directory.
Reported-by: Dan Raymond <draymond@foxvalley.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since c043ecb20c (support/download: change format of archives
generated from svn), the svn backend uses the generic helper to
create reproducible archives.
That helper really does its job as expected, but the svn backend
is flawed in two ways:
- the first, most obvious breakage happens with versions older
than 1.9, as they do not support the '--show-item' option
for the 'info' action;
- the second is more involved, in that svn will by default
expand the old, legacy, deprecated, cumbersome CVS-style
keywords, in the form of revision marks like '$Date$' in a
C-style comment in a source file. These replacements are
done on checkout as well as on export, and they use local
settings, like the local locale and timezone.
This means that two people with different settings, will get
different sources when the svn-checkout or svn-export the same
revision from the same tree...
Needless to say that this is not very reproducible...
While the first is easily solved, the second is more involved.
We need to ensure that what source is used initially to compute
the hash, will also be the source that are used to check the hash.
There are basically two solutions:
1. we ensure the same environment, by forcing the timezone and
the locale to arbitrary values
2. we disable keyword expansion
For the first solution, this still leaves the possibility that we
miss some environment settings that have an impact on the keyword
expansion. It would mean that Yann's settings be used, as he did
introduce the hash for the only svn-downloaded package we have,
avrdude, settings which are:
TZ=Europe/Paris
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="fr_FR.utf8"
LC_NUMERIC="fr_FR.utf8"
The second option means that the generated archives change. That
means we'd have to bump the archive version for svn downloads, and
that we update the hashes for all the svn-downloaded packages.
We chose to go with the second option, because this is what really
makes more sense, rather than hard-coding arbitrary values in the
environment. And we also have only one svn-downloaded package,
avrdude.
And thus, we're reaching the trigger for this change: avrdude is
impacted by the CVS-keyword expansion issue:
https://svn.savannah.gnu.org/viewvc/avrdude/trunk/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js?revision=1396&view=markup
which would give two different files when checked out on different
machines:
diff -durN foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js
--- foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
+++ bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
@@ -1,6 +1,6 @@
/**
* @preserve jquery.layout 1.3.0 - Release Candidate 30.51
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
* $Rev: 303005 $
*
* Copyright (c) 2012
@@ -4718,7 +4718,7 @@
/**
* jquery.layout.state 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5074,7 +5074,7 @@
/**
* jquery.layout.buttons 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5356,7 +5356,7 @@
/**
* jquery.layout.browserZoom 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2012
* Kevin Dalman (http://allpro.net)
So we also update the hash for avrdude.
Fixes:
http://autobuild.buildroot.org/results/e3b/e3b0508047f32008ebfa83c5255ec5994b6af120/ (time issue)
http://autobuild.buildroot.org/results/48e/48e78e84b425e79cdb98c16ab40247a0fa7e9676/ (keyword expansion issue)
Reported-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Reviewed-by: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
FOO_CPE_ID_VALID really ought to be an internal implementaion detail.
Packages that really want to trigger their CPE defintitions really
should set one of the actual variables to a meaningful value.
There are two CPE-related variables that we could chose to set to
replace FOO_CPE_ID_VALID: FOO_CPE_ID_VENDOR and FOO_CPE_ID_PRODUCT.
Between those two, _VENDOR more often diverges from the default than
_PRODUCT does, so that's what we use.
---8<------8<------8<------8<------8<---
#!/bin/bash
# Replace FOO_CPE_ID_VALID = YES with FOO_CPE_ID_VENDOR = foo_project
for i in $(git grep -l -E '[^)]_CPE_ID_VALID = YES' package support); do
pkg="$(basename "${i%/*}")"
sed -r -i -e "s/_CPE_ID_VALID = YES/_CPE_ID_VENDOR = ${pkg}_project/" "${i}"
done
---8<------8<------8<------8<------8<---
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: update cpe-test comment to reflect pkg3 change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some download backends, like svn, will provide timestamps with a
sub-second precision, e.g.
$ svn info --show-item last-changed-date [...]
2021-02-19T20:22:34.889717Z
However, the PAX headers do not accept sub-second precision, leading to
failure to download from subversion:
tar: Time stamp is out of allowed range
tar: Exiting with failure status due to previous errors
make[1]: *** [package/pkg-generic.mk:148: [...]/build/subversion-1886712/.stamp_downloaded] Error 1
Fix that by massaging the timestamp to drop the sub-second part. We
do that in the generic helper, rather than the svn backend, so that
all callers to the generic helper benefit from this, as this is more
an internal details of the tarball limitations, than of the backends
themselves.
Reported-by: Roosen Henri <Henri.Roosen@ginzinger.com>
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- add Henri as reporter
- move it out of the svn backend, and to the generic helper
- reword the commit log accordingly
- use an explicit time format rather than -Iseconds
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
As reported on IRC by sephthir, the gitlab test of the defconfig
qemu_sparc_ss10_defconfig doesn't error out while the system
is not working properly.
This is because we explicitly wait for the timeout as an expected
condition, but do not check for it. Indeed, pexpect.expect() returns
the index of the matching condition in the list of expected conditions,
but we just ignore the return code, so we are not able to differentiate
between a successful login (or prompt) from a timeout.
By default, pexepect.expect() raises the pexpect.TIMEOUT exception on a
timeout, and we are already prepared to catch and handle that exception.
But because pexpect.TIMEOUT is passed as an expected condition, the
exception is not raised.
Remove pexpect.TIMEOUT from the list of expected conditions, so that the
exception is properly raised again, and so that we can catch it.
The qemu_sparc_ss10_defconfig is already fixed by
4d16e6f532.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
[yann.morin.1998@free.fr: reword commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 610e67b1fc)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Add the list of <pkg>_IGNORE_CVES to the json output to show that we have a
known cause (available patch or the CVE is not valid for our package
configuration) that a affected CVE is not reported.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
pickle is no longer used since 09a71e6a75
Fixes:
support/scripts/cpedb.py:7:1: F401 'pickle' imported but unused
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, the CPE XML database is parsed into a Python dict, which is
then pickled into a local file, to speed up the processing of further
invocations.
However, it turns out that since the initial implementation, we have
switched the XML parsing from the out of tree xmltodict module to the
standard ElementTree one, which has made the parsing much faster. The
pickle caching only saves 6 seconds, on something that takes more than
13 minutes total.
In addition, this pickle caching consumes a significant amount of RAM,
causing the Python process to be OOM-killed on a server with 4 GB of
RAM.
So let's just drop this caching entirely.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/musl, use a pre-built Bootlin
toolchain.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/glibc, use a pre-built Bootlin
toolchain.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Most of the toolchains now use gcc 9.x and kernel headers 5.9, instead
of gcc 8.x and kernel headers 5.4.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Instead of using an external toolchain built specifically for the
autobuilders to test Xtensa/uclibc, use a pre-built Bootlin toolchain.
To be noted: that fragment was in fact already using a Bootlin
bleeding-edge toolchain, because BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is
missing from the fragment:
$ cat support/config-fragments/autobuild/br-xtensa-full.config >.config
$ make olddefconfig
$ grep BOOTLIN .config
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_XTENSA_LX60_UCLIBC_BLEEDING_EDGE=y
# BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_XTENSA_LX60_UCLIBC_STABLE is not set
The original fragment was supposed to use a stable toolchain, so we
switch to explictly use a stable Bootlin toolchain.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- add blurb about missing BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
