The NVD files that are used to build the list of CVEs affecting
Buildroot packages are quite large (a few hundreds MB of json),
and cause the pkg-stats scripts to have a huge memory footprint
(a few GB with Python 2.7).
However, because we only need to iterate on CVE items one by one,
we can process them in streaming (ie decoding one CVE at a time
from the JSON representation). Because the json module from the
python standard library does not support such a mode of operation,
we switch to the third-party package ijson, which is compatible
with both Python 2 and Python3.
To run the script with these modifications, one should install
the ijson python package. This can be done with pip:
`pip install ijson`. On Debian based distributions, this can
also be done with the apt package manager:
`apt install python-ijson`.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit d255b67972 fixed the handling of
the a package local m4/ directory which might be missing. But this
only works if it is the very first argument. But for this package this
is not possible because we already occupy this with the extra include
directory for autoconf-archive. Bring back the hook to create the m4/
directory to fix this.
Fixes:
http://autobuild.buildroot.net/results/dc907421a343b8523b14fc9a846e0caf7abe630c/
Signed-off-by: Michael Walle <michael@walle.cc>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Remove the lib/ssl/src/deps directory before configuring the package.
Otherwise, during the compilation of the ssl app, it may fails by
looking for logger.hrl in the wrong location (bootstrap/lib/kernel
instead of lib/kernel).
Fixes:
http://autobuild.buildroot.net/results/97606fcd11eaf0822b58a9532c5325601d43eaac/
Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com>
Tested-by: Frank Vanbever <frank.vanbever@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Create the staging symlink the same way as the host symlink. This means
using a make dependency rather than recreating it every time.
In coreutils versions below 8.27, re-creation of symbolic links was not
atomic. This means that there is a period in time where the existing link is
removed, before the new one is created. In coreutils 8.27 this was fixed,
see [1]. Note that CentOS 7 ships with coreutils 8.22.
In the following scenario, this is a problem:
- an application is compiled using the sysroot prepared by Buildroot and
links against Xenomai userspace libraries, but its build process is steered
from outside of Buildroot
- to know the correct flags, the application makefile uses the 'xeno-config'
file to request them, and passes DESTDIR=/buildroot/output/staging
- the xeno-config responds with flags based on the path
'/buildroot/output/staging/...'
- while the application build is ongoing, a 'make' happens in Buildroot,
causing the 'staging' symlink to be recreated (even though it already
existed)
- when exactly at this time, the application calls the compiler with -I
flags pointing to output/staging, the build fails with:
-I/buildroot/output/staging/usr/include/xenomai/mercury: Error: ^ is not a directory
-I/buildroot/output/staging/usr/include/xenomai: Error: ^ is not a directory
-I/buildroot/output/staging/usr/include/xenomai/xenomai: Error: ^ is not a directory
-I/buildroot/output/staging/usr/include/xenomai/psos: Error: ^ is not a directory
Failed: ** ^ *
Work around this problem by only creating the staging symlink once, similar
to how the host symlink (if any) is created.
See also commit d0f4f95e39 which changed the
way these symlinks are made. The reasoning in this commit is to move away
from the 'dirs' target.
[1] 376967889e
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The libxml2 package has two patches that fix the two CVEs affecting
libxml2 in version 2.9.10, so let's use LIBXML2_IGNORE_CVES to ensure
these CVEs are no longer reported by pkg-stats.
Cc: Titouan Christophe <titouan.christophe@railnova.eu>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
It seems like throughout the series that the CVE pkg-stats support
went through, the support for ignoring CVEs in the per-package
<pkg>_IGNORE_CVES variable was forgotten.
Let's re-introduce this, which is now very simple thanks to the CVE
class, its .identifier() propertly and the .is_cve_ignored() method of
the Package class
Cc: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
libupnpp uses pkg-config since version 0.15.1 and
3dc44417e8
so remove unneeded static workaround
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This is needed so that building the owfs Python module uses the gcc
from owfs per-package directory, and not the one from the python
per-package directory.
Fixes:
http://autobuild.buildroot.net/results/0d582dda367507991a4c38141db36b0fa8e47e67/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
With per-package directory support, Python external modules are
causing a problem: the _sysconfigdata.py module installed by the
Python interpreter contains a number of paths that are relative to the
current package per-package directory, i.e python or python3. For
example:
'BLDSHARED': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-gcc -shared',
'CC': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-gcc',
'CXX': '/home/thomas/projets/buildroot/output/per-package/python/host/bin/arm-linux-g++',
etc.
These paths are problematic, because it means that the wrong compiler
gets used when building external Python modules: instead of using the
compiler from the external Python module per-package host directory,
it uses the one from the 'python' or 'python3' per-package host
directory. Due to this, any native dependency needed by the external
Python module is not found, even though it is properly present in the
current package per-package directory.
Of course, the problem occurs with both target Python modules and host
Python modules.
To fix this, we simply rewrite those paths in _sysconfigdata.py before
building a Python package.
Interestingly, until now, the _sysconfidata.py that was used during
the build was the one from $(TARGET_DIR), which is a bit unusual: it
is more common to use files from $(STAGING_DIR) during the build
process. So this commit changes the PYTHON_PATH and PYTHON3_PATH
variables so that they point to $(STAGING_DIR), which makes the
_sysconfigdata.py fixup in $(STAGING_DIR) effective.
Fixes:
http://autobuild.buildroot.net/results/a24b0555fd4261b50dc3986635c30717d9cbe764/ (python-psycopg2)
http://autobuild.buildroot.net/results/080fa893e1b0e7a8c8a31ac1c98eb8871b97264d/ (python-alsaaudio)
http://autobuild.buildroot.net/results/79bc070f98d6d9d8ef78df12b248cdc7d0e405c3/ (python-lxml)
and many more Python packages that use native code with a native library
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When APR_INCLUDEDIR and APU_INCLUDEDIR point to the same directory,
Apache builds properly. However, with per-package directory support,
they point to different directories, and APU_INCLUDEDIR contains both
the APR headers and the APU headers.
Due to this, the Apache Makefile logic to generate its exports.c file
leads to duplicate definitions, because the APR headers are considered
twice: once from APR_INCLUDEDIR, once from APU_INCLUDEDIR.
We fix this by introducing a patch to the Apache build system.
In addition, apr provides a special libtool script that gets used by
apr-util and apache. apr-util already had a fixup for this, but apache
did not, which was causing the gcc from apr-util per-package
directories be used during the apache build, causing build failures.
To fix this, we adjust this libtool script to point to the correct
tools in apache's per-package directories.
There are no autobuilder failures for this, because Apache needs
apr-util, and apr-util currently fails to build when
BR2_PER_PACKAGE_DIRECTORIES=y.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
With per-package directories support enabled, the build of apr-util
fails, for two reasons:
- The rules.mk file is generated by the 'apr' package, and then
copied into the 'apr-util' source directory. This is done by the
'apr-util' build process. Unfortunately, this rules.mk file has a
number of hardcoded paths: to the compiler and to the libtool
script.
Due to this, the compiler from the 'apr' per-package directory gets
used. But this compiler uses the 'apr' package sysroot, which does
not have all the dependencies of the 'apr-util' package, causing
the build to fail because <expat.h> is not found.
- Similarly, the libtool script itself has some hardcoded paths,
which make it use the compiler/linker from the 'apr' per-package
directory, so it does not find the expat library.
We fix both issues by doing the necessary replacement in both rules.mk
and libtool.
Fixes:
http://autobuild.buildroot.net/results/2a67b5d58f79348e20a972125e4797eff5585716/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes CVE-2019-20388: xmlSchemaPreRun in xmlschemas.c in libxml2 2.9.10
allows an xmlSchemaValidateStream memory leak.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes the following security issues (12.15.0):
- CVE-2019-15606: HTTP header values do not have trailing OWS trimmed
- CVE-2019-15605: HTTP request smuggling using malformed Transfer-Encoding
header
- CVE-2019-15604: Remotely trigger an assertion on a TLS server with a
malformed certificate string
For more details, see the advisory:
https://nodejs.org/en/blog/vulnerability/february-2020-security-releases/
On top of this, 12.16.0 brings a number of changes and bugfixes.
Update the license hash for an addition of the (MIT) licensing terms for the
uvwsai module:
+
+- uvwasi, located at deps/uvwasi, is licensed as follows:
+ """
+ MIT License
+
+ Copyright (c) 2019 Colin Ihrig and Contributors
+
+ Permission is hereby granted, free of charge, to any person obtaining a copy
+ of this software and associated documentation files (the "Software"), to deal
+ in the Software without restriction, including without limitation the rights
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ copies of the Software, and to permit persons to whom the Software is
+ furnished to do so, subject to the following conditions:
+
+ The above copyright notice and this permission notice shall be included in all
+ copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ SOFTWARE.
+ """
While we are at it, adjust the white space in the .hash function to match
the new agreements.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 64c42c5e2c removed the hook for
creating the m4local directory with the assumption that it would be
created because the first include is treated in a special way if it
doesn't exists
However, this assumption was wrong as m4local is the second include, the
first one is m4 (which already exists in the archive). So put back the
hook. The other solutions would be to patch:
- Makefile.{am,in} to remove m4local
- configure.ac and Makefile.{am,in} to add m4local before m4
However, both solutions don't seem to be upstreamable
Fixes:
- http://autobuild.buildroot.org/results/e40313c6ec193d6156e26eff62303545fba09413
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The package instrumentation step 'step_pkg_size' is populating the files:
output/build/packages-file-list.txt
output/build/packages-file-list-staging.txt
output/build/packages-file-list-host.txt
by comparing the list of files before and after installation of a package,
with some clever tricks to detect changes to existing files etc.
As an optimization, instead of gathering this list before and after each
package, where the 'after-state' of one package is the same as the
'before-state' of the next package, only the 'after-state' is used and
is shared between packages.
This works fine, except at the end of the build, as explained next.
In the target-finalize step, many files will be touched. For example, files
like /etc/hosts, /etc/os-release, but also all object files that are
stripped, and all files touched by post-build scripts or created by rootfs
overlays. This means that the 'after-state' of the last package does not
reflect the actual situation after target-finalize is run.
For a single complete build this poses no problem. But, if one incrementally
rebuilds a package after the initial build, e.g. with 'make foo-rebuild',
then all changes that happened in target-finalize at the end of the initial
build (the 'after-state' of the last package built) will be detected as
changes caused by the rebuild of package foo. As a result, all these files
will incorrectly be treated as 'owned' by package foo.
Correct this situation by capturing a new state at the end of
target-finalize, so that the 'before-state' of an incremental build will be
correct.
Note: the reasoning above talks about packages-file-list.txt and
target-finalize, but also applies to
packages-file-list-staging.txt/staging-finalize and
packages-file-list-host.txt/host-finalize.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reorder imports using the isort utility to fix a warning from pylint3:
wrong-import-order: standard import "import multiprocessing" should be
placed before "import nose2"
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit 12c0f68caf (package/nfs-utils: bump version to 2.4.3) added an
extra empty line, causing check-package to whine:
package/nfs-utils/nfs-utils.mk:27: consecutive empty lines
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The current Buildroot defconfigs for qemu_x86 and qemu_x86_64
instantiate a console on tty1, which appears on QEMU's
graphical window. Add a console on the serial port (ttyS0) to
be used later for gitlab testing.
This change is need since the script used for gitlab testing
needs to use a serial output with pexpect.
This change is similar to the one made for raspberrypi [1] to
handle HDMI and serial console:
This requires three changes:
1. have two 'console=' entries in the kernel command line: tty1,
then ttyS0;
2. change BR2_TARGET_GENERIC_GETTY_PORT to "console", so it starts
a getty on the last console= passed to the kernel, ttyS0;
3. add a new getty on tty1 to the generated inittab.
Step 2 is actually obtained by removing BR2_TARGET_GENERIC_GETTY_PORT
entirely from the defconfigs, since "console" is the default value.
Step 3 requires a post-build script since the Buildroot makefiles can
configure only one console.
Note: instead of simply adding a new getty on ttyS0 (which would
work) this patch actually changes BR2_TARGET_GENERIC_GETTY_PORT to
instantiate a console on UART, then adds back tty1 via
post-build.sh. This is done only to avoid the "GENERIC_SERIAL" comment
where we instantiate a console on QEMU graphical window, then
instantiate a really-serial console on another line.
The result is these two inittab lines:
console::respawn:/sbin/getty -L console 0 vt100 # GENERIC_SERIAL
tty1::respawn:/sbin/getty -L tty1 0 vt100 # QEMU graphical window
[1] 20878a1017
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The commit [1] added host-qemu package for each qemu defconfig
for gitlab runtime testing.
[1] 29e1cb8884
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Joel Stanley <joel@jms.id.au>
Acked-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This defconfig was generated by savedefconfig but we usually
use a manually modified defconfig to add some comments for
Kconfig symbols.
No content change intended.
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Joel Stanley <joel@jms.id.au>
Acked-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Other changes:
- Update License hash which properly adds the OpenSSL exception.
Tested with Debian 8:
br-arm-full [1/6]: OK
br-arm-cortex-a9-glibc [2/6]: OK
br-arm-cortex-m4-full [3/6]: SKIPPED
br-x86-64-musl [4/6]: OK
br-arm-full-static [5/6]: SKIPPED
sourcery-arm [6/6]: OK
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Other changes:
- Remove upstream patches
- Update COPYING.LIB hash as upstream updated the file to match the new LGPL
2.1 license from upstream. See:
f0d44cc446
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
[Peter: change libssh2 to libssh as pointed out by Vincent Fazio]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Bump to version 2.4.3 of nfs-utils. All patches have been upstreamed, so
drop them all. It now needs rpcgen built by host-nfs-utils, to do this
let's pass its path to --with-rpcgen= instead of 'internal'.
Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Tested-by: Petr Vorel <petr.vorel@gmail.com>
[Peter: drop AUTORECONF, explicitly depend on host-nfs-utils]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes version parsing for release-monitoring.org support
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
According to PEP8 empty sequences should be checked as booleans.
Fixes the following PEP8 warning:
Do not use `len(SEQUENCE)` to determine if a sequence is empty
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit de591c5c3a (package/wireguard-linux-compat: new package) split up
the wireguard package in wireguard-tools and wireguard-linux-compat, but
forgot to update the conditional in linux.mk, so the kernel config fixups
needed for wireguard are no longer applied.
Update the conditional to use the BR2_PACKAGE_WIREGUARD_LINUX_COMPAT symbol
instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
License hash change is due to date update.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
