Commit Graph

48896 Commits

Author SHA1 Message Date
Michael Vetter
61703b82cd package/jasper: Apply fix for CVE-2018-19542
Add 0002-check-null-in-jp2_decode.patch:

Patch was proposed upstream[1] but upstream is very inactive.
Linux distributions use the same fix to patch their packages.

1: https://github.com/mdadams/jasper/pull/200
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 13:39:42 +01:00
Michael Vetter
fddee3cf74 package/jasper: Apply fix for CVE-2018-19541
Add 0001-verify-data-range-CVE-2018-19541.patch:
We need to verify the data is in the expected range. Otherwise we get
problems later.

Patch was proposed upstream[1] but upstream is very inactive. Linux
distributions use the same fix to patch their packages.

1: https://github.com/mdadams/jasper/pull/211
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 13:39:33 +01:00
Carlos Santos
3b1d9f74c7 board/pc/post-image-efi.sh: fail on any error
As already done in post-build.sh, to prevent hiding errors.

Signed-off-by: Carlos Santos <unixmania@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:47:47 +01:00
Michael Vetter
cf36b7c3ac DEVELOPERS: add Michael Vetter for libstrophe
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:47:18 +01:00
Alexander Lukichev
a191ae4b90 DEVELOPERS: change email address for Alexander Lukichev
Signed-off-by: Alexander Lukichev <alexander.lukichev@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:43:11 +01:00
Alexander Lukichev
071166c558 package/openpgm: drop release- prefix to match release-monitoring.org
Autobuild has a "version bump" checker that sends message to a package
maintainer if its declared version does not match the latest one on
release-monitoring.org.  In case of openpgm the version _is_ the latest one,
but a mismatch is caused by including a "release-" prefix into tags on
upstream and excluding them on other websites when referring to the
package's version.

This also fixes sha256 value for the downloaded archive.

Signed-off-by: Alexander Lukichev <alexander.lukichev@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:42:28 +01:00
Romain Naour
c0719217e1 package/gcc: bump to version 7.5.0
Remove upstream patch [1]
1002-xtensa-backport-fix-for-PR-target-90922.patch

[1] 0110ab63c0

See:
https://gcc.gnu.org/ml/gcc/2019-11/msg00099.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:16:11 +01:00
Peter Korsgaard
b5fd68a454 {linux, linux-headers}: bump 5.4 series to 5.4.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 12:14:10 +01:00
Marcus Folkesson
9e000606c3 {linux, linux-headers}: add version 5.4
Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 11:47:43 +01:00
Peter Korsgaard
7b24bd59c8 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 09:39:41 +01:00
Peter Korsgaard
14ec220df3 Kickoff 2020.02 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-02 08:57:02 +01:00
Peter Korsgaard
a4ff0817c5 docs/website/news.html: add 2019.11 announcement link
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 23:05:50 +01:00
Peter Korsgaard
836b84a774 Update for 2019.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 22:39:47 +01:00
Peter Korsgaard
0f5ac40374 package/{pkg-generic, python, python3}: rename .py file exclusion variable to not conflict
Fixes:
http://autobuild.buildroot.net/results/3b6/3b6280b0b7a9634b747db2865b21c6266007c725/

The PYTHON_KEEP_PY_FILES global variable conflicts with the per-package
<pkg>_KEEP_PY_FILES variable for the python package, causing make to
complain:

package/zlib/zlib.mk:7: *** Recursive variable 'PYTHON_KEEP_PY_FILES' references itself (eventually).  Stop.

As a workaround, rename the global variable to KEEP_PYTHON_PY_FILES so it
cannot conflict with the per-package variable.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 21:55:37 +01:00
Mark Corbin
ca440a773b package/musl-fts: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:16:08 +01:00
Mark Corbin
abd4976515 package/mp4v2: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(405)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:15:46 +01:00
Mark Corbin
56fff9ef97 package/matchbox-startup-monitor: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:15:12 +01:00
Mark Corbin
484b687f79 package/matchbox-panel: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:15:05 +01:00
Mark Corbin
45f5c12c4b package/matchbox-lib: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:14:56 +01:00
Mark Corbin
381fb14bbd package/matchbox-keyboard: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:14:49 +01:00
Mark Corbin
680d20cd47 package/matchbox-fakekey: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:14:43 +01:00
Mark Corbin
5fb87bbd35 package/matchbox-desktop: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:14:35 +01:00
Mark Corbin
99eb6cfb24 package/matchbox-common: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:14:06 +01:00
Mark Corbin
dcc2a29f41 package/linux-fusion: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:13:55 +01:00
Mark Corbin
b27c5c1c0b package/luasql-sqlite3: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(406)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:13:36 +01:00
Mark Corbin
0d9c7347f0 package/luasyslog: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(Err)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:13:20 +01:00
Michael Vetter
154bbfd53f package/jasper: bump to 2.0.16
Changes:
  * Fix assertion failure JPC_NOMINALGAIN (CVE-2016-9396) (#50)
  * Fix build on Windows 10 (#162)
  * Improve README
  * Fix build with CMake 2.x
  * Add missing dereference operators (#178, #157)
  * Check data in jas_image (CVE-2018-19539) (#196)
Signed-off-by: Michael Vetter <jubalh@iodoru.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:11:45 +01:00
Pascal de Bruijn
89ebe8b4a1 package/openvmtools: only try to start vmtoolsd on vmware
adds ConditionVirtualization=vmware to vmtoolsd.service

Signed-off-by: Pascal de Bruijn <p.debruijn@unilogic.nl>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:10:25 +01:00
Pascal de Bruijn
164963b808 package/openvmtools: use correct variable in udev rule RUN+=
DEVPATH is not a valid substitution in a RUN+=, devpath is:
https://mirrors.edge.kernel.org/pub/linux/utils/kernel/hotplug/udev/udev.html

So use that to get rid of the following warning:

Invalid value "/bin/sh -c 'echo 180 >/sys$DEVPATH/device/timeout'" for RUN
(char 27: invalid substitution type), ignoring, but please fix

Source: https://github.com/vmware/open-vm-tools/pull/376

Signed-off-by: Pascal de Bruijn <p.debruijn@unilogic.nl>
Acked-by: Jérémy Rosen <jeremy.rosen@smile.fr>
[Peter: extend description]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:06:25 +01:00
Pascal de Bruijn
7ad22f41b5 package/openvmtools: udev rules files should not be executable
/usr/lib/udev/rules.d/99-vmware-scsi-udev.rules is marked executable,
causing systemd to complain.

Source: https://github.com/vmware/open-vm-tools/pull/376

Signed-off-by: Pascal de Bruijn <p.debruijn@unilogic.nl>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 19:01:14 +01:00
Michael Vetter
3a508f8564 package/prosody: bump to bugfix version 0.11.3
This is a bugfix release for the stable 0.11 branch. It is recommended
for all users of 0.11.x to upgrade.

Important note for those upgrading: Previous releases did not
automatically expire messages from group chat (MUC) archives, so if
mod_muc_mam was loaded and enabled for a MUC, archives would grow
indefinitely. This is not what most deployments want, therefore
automatic expiry is now implemented and enabled with a default 7 day
retention. You can configure this with the muc_log_expires_after
configuration option, which can be set to "never" to restore the old
behaviour and preserve any existing logs older than 7 days.

For details see:
https://prosody.im/doc/release/0.11.3

Signed-off-by: Michael Vetter <jubalh@iodoru.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:58:34 +01:00
Mark Corbin
3fa37a41d8 package/mxsldr: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:58:15 +01:00
Mark Corbin
f06ae71f64 package/opentyrian-data: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(405)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:57:47 +01:00
Mark Corbin
08f2c31c05 package/netperf: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(Err)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:57:07 +01:00
Mark Corbin
c04dab6f6f package/oprofile: add an upstream URL to Config.in
Add an upstream URL to the help text in Config.in. This
addresses the 'Missing' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:55:26 +01:00
Mark Corbin
c3c12bd9a3 package/ortp: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(404)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:52:57 +01:00
Mark Corbin
10a67d8860 package/opkg: update the upstream URL in Config.in
Update the upstream URL in the help text in Config.in. This
addresses the 'Invalid(405)' URL status in the package stats
web page output.

Signed-off-by: Mark Corbin <mark@dibsco.co.uk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 18:52:28 +01:00
Thomas Petazzoni
30bf7e3f22 package/network-manager: add missing dependency on libglib2
Commit 7643ab05de
("package/network-manager: drop obsolete dependencies") was a bit too
brutal in its dependency removal, as it forgot that removing dbus-glib
as a dependency would make libglib2 no longer part of the dependencies
of network-manager.

However, network-manager does require libglib2. From configure.ac:

PKG_CHECK_MODULES(GLIB, [gio-unix-2.0 >= 2.37.6 gmodule-2.0],
                  [AC_SUBST(LOG_DRIVER, '$(top_srcdir)/build-aux/tap-driver.sh')
                   AC_SUBST(AM_TESTS_FD_REDIRECT, '--tap')],
                  [PKG_CHECK_MODULES(GLIB, gio-unix-2.0 >= 2.40 gmodule-2.0)
                   AC_SUBST(LOG_DRIVER, '$(top_srcdir)/build-aux/test-driver')])

So this commit re-adds libglib2 as a dependency, and propagates the
appropriate "depends on". Nothing selects BR2_PACKAGE_NETWORK_MANAGER,
so we don't have to propagate these additional "depends on" anywhere.

Fixes:

  http://autobuild.buildroot.net/results/2025b1bd721bb5c5fa6638ccf389d2fd8fd10339/
  https://bugs.busybox.net/show_bug.cgi?id=12326

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Lubomir Rintel <lkundrak@v3.sk>
Cc: Christian Stewart <christian@paral.in>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 11:42:58 +01:00
Bernd Kuhls
af712ca7ce {linux, linux-headers}: bump 4.{14, 19}.x series
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-01 11:25:50 +01:00
Adam Duskett
01d3360eb6 package/libglib2: bump to 2.62.3
Changes include:
 - Rename 0002-add-option-to-build-tests.patch to
   0002-allow-explicit-disabling-of-tests.patch to work with 2.62.3.

   Upstream now has an option called "installed_tests," which can be used to
   prevent the unit tests from being built. However, the check only works for
   cross-compiling and not host builds, which would mean that the tests still
   build when compiling the host variant.

   This conditional causes an error on older systems such as Debian8 or Centos7
   because the command `objcopy --add-symbol` is used when building the test
   "test_resources2," which is not available with the older version of objcopy
   provided by those distributions.

   Removing the conditionals and checking for just the installed_tests_enabled
   option allows for a much simpler patch.

 - Change -Dtests=false to -Dinstalled_tests=false in libglib2.mk as per the
   above comment.

           br-arm-full [1/6]: OK
br-arm-cortex-a9-glibc [2/6]: OK
 br-arm-cortex-m4-full [3/6]: SKIPPED
        br-x86-64-musl [4/6]: OK
    br-arm-full-static [5/6]: OK
          sourcery-arm [6/6]: OK

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-30 23:07:33 +01:00
Christian Stewart
aa4218f73f package/go: bump to 1.13.4
go1.13.4 (released 2019/10/31) with fixes to the net/http and syscall packages.

Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-30 22:41:43 +01:00
Thomas Petazzoni
d07c315f0c package/opencv3: ensure the python module works when BR2_PACKAGE_PYTHON{, 3}_PYC_ONLY=y
The OpenCV Python module does a fairly strange thing to read a few
configuration details: it uses Python's execfile() to execute two .py
files and access a few variables. However, execfile() only works with
.py files and not .pyc files.

When BR2_PACKAGE_PYTHON{,3}_PYC_ONLY=y, the .py files are all removed,
causing the OpenCV Python module to not work:

  File "usr/lib/python3.7/site-packages/cv2/__init__.py", line 89, in <module>
  File "usr/lib/python3.7/site-packages/cv2/__init__.py", line 58, in bootstrap
  File "usr/lib/python3.7/site-packages/cv2/__init__.py", line 56, in load_first_config
ImportError: OpenCV loader: missing configuration file: ['config.py']. Check OpenCV installation.

To fix this problem, this commit uses the newly introduced
<pkg>_KEEP_PY_FILES mechanism, to ensure the important config*.py
files are kept.

Fixes:

  https://bugs.busybox.net/show_bug.cgi?id=12171

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-30 09:47:47 +01:00
Thomas Petazzoni
56f3ed3fc1 package/{pkg-generic, python, python3}: add mechanism to exclude .py files from removal
When BR2_PACKAGE_PYTHON{,3}_PYC_ONLY=y, we force remove all .py files
from the system, as they have all been byte-compiled into their .pyc
variants.

However, it turns out that some packages (e.g: OpenCV) do some funky
things with a few .py files: they pass them through Python's
execfile() facility, which only works with .py files and not .pyc
files. It is used by OpenCV for example to read two small
configuration files.

In order to support such use cases, this commit introduces a very
simple mechanism by which packages can exclude some path patterns from
the .py removal: a per-package <pkg>_KEEP_PY_FILES variable that is
collected into a global PYTHON_KEEP_PY_FILES variable, then used by
the python/python3 target-finalize hooks.

This variable is intentionally not documented, this is really a hack
that we ideally would like to see go away, and we'd rather not see its
usage spread too much.

This is necessary to be able to fix bug #12171.

[Peter: check if PYTHON_KEEP_PY_FILES contains non-white space]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-30 09:46:27 +01:00
Bernd Kuhls
b20487151a {linux, linux-headers}: bump 4.{4, 9}.x / 5.3.x series
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-30 09:13:03 +01:00
Thomas Petazzoni
e8c8b41dc5 docs/manual: document the effect of per-package directory on variables
As suggested by Arnout Vandecappelle, this commit adjusts the
generic-package documentation to document the effect of per-package
directory support on HOST_DIR, STAGING_DIR and TARGET_DIR.

Suggested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 15:43:15 +01:00
Thomas Petazzoni
532574b259 docs/manual: add details about top-level parallel build support
[Peter: change version reference to Buildroot 2020.02]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 15:42:22 +01:00
Thomas Petazzoni
41ea767113 package/pkg-kconfig: handle KCONFIG_DEPENDENCIES with per-package directories
The pkg-kconfig infrastructure hijacks the regular chain of build
steps to insert its own step to prepare the configuration of kconfig
packages. This additional step may have dependencies of its own, such
as host-flex, host-bison or toolchain.

In the context of per-package directory support, those dependencies
must be copied to the per-package directory of the current package
prior to doing the config preparation. This commit implements this
logic by adding a call to prepare-per-package-directory at the right
spot.

Reported-by: Andreas Naumann <anaumann@ultratronik.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 15:29:04 +01:00
Thomas Petazzoni
b27fb7871c package/pkg-generic: make libtool .la files compatible with per-package directories
Libtool .la files unfortunately contain a number of absolute paths,
which now refer to per-package directories. Due to this, when building
package A, .la files may contain absolute paths referring to
directories in package B per-package sysroot. This causes some -L
flags referring to other sysroot from being added, which doesn't work
as the linker no longer realizes that such paths are within its
sysroot.

To fix this, we introduce a replacement step of .la files in the
configure step, to make sure all paths refer to this package
per-package directory.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 15:24:05 +01:00
Thomas Petazzoni
d9a4c0c7ea Makefile: allow top-level parallel build with BR2_PER_PACKAGE_DIRECTORIES=y
With per-package folder support, top-level parallel build becomes
safe, so we can enclose the .NOTPARALLEL statement in a
!BR2_PER_PACKAGE_DIRECTORIES condition.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 15:13:45 +01:00
Thomas Petazzoni
c4e6d5c8be core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.

There are two main benefits:

 - Packages will now see only the dependencies they explicitly list in
   their <pkg>_DEPENDENCIES variable, and the recursive dependencies
   thereof.

 - We can support top-level parallel build properly, because a package
   only "sees" its own host directory and target directory, isolated
   from the build of other packages that can happen in parallel.

It works as follows:

 - A new output/per-package/ directory is created, which will contain
   one sub-directory per package, and inside it, a "host" directory
   and a "target" directory:

   output/per-package/busybox/target
   output/per-package/busybox/host
   output/per-package/host-fakeroot/target
   output/per-package/host-fakeroot/host

   This output/per-package/ directory is PER_PACKAGE_DIR.

 - The global TARGET_DIR and HOST_DIR variable now automatically point
   to the per-package directory when PKG is defined. So whenever a
   package references $(HOST_DIR) or $(TARGET_DIR) in its build
   process, it effectively references the per-package host/target
   directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
   is handled as well.

 - Of course, packages have dependencies, so those dependencies must
   be installed in the per-package host and target directories. To do
   so, we simply rsync (using hard links to save space and time) the
   host and target directories of the direct dependencies of the
   package to the current package host and target directories.

   We only need to take care of direct dependencies (and not
   recursively all dependencies), because we accumulate into those
   per-package host and target directories the files installed by the
   dependencies. Note that this only works because we make the
   assumption that one package does *not* overwrite files installed by
   another package.

   This is done for "extract dependencies" at the beginning of the
   extract step, and for "normal dependencies" at the beginning of the
   configure step.

This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:

 - The filesystem image creation code cannot work.

 - We don't have a SDK to build code outside of Buildroot.

In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.

This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:

 - The check-host-rpath script, which verifies at the end of each
   package installation that it has the appropriate RPATH, is modified
   to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
   a correct RPAT.

 - The fix-rpath script, which mungles the RPATH mainly for the SDK
   preparation, is modified to rewrite the RPATH to not point to
   per-package directories. Indeed the patchelf --make-rpath-relative
   call only works if the RPATH points to the ROOTDIR passed as
   argument, and this ROOTDIR is the global host directory. Rewriting
   the RPATH to not point to per-package host directories prior to
   this is an easy solution to this issue.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-29 14:24:05 +01:00