Commit Graph

121 Commits

Author SHA1 Message Date
Yann E. MORIN
556a0a1104 Revert "make: support: use command -v' instead of which'"
This reverts commit ca6a2907c2.

Switching to using 'command -v' instead of 'which', opened a can of
worms that is hard to fix in a timely manner:

  - recursive call to 'make' from a post-build, post-iamge script, fails
    because of a redefinition of HOSTCC_NOCCACHE (a bug on its own that
    needs a separate fix anyway) [0];

  - 'make' believeing it can call "simple" commands with execve() et al.
    instead of passing them through a shell via system(), and thus
    failing to find 'command' in the PATH [1].

[0] https://lore.kernel.org/buildroot/20211001175329.GA1973888@lbrmn-mmayer.ric.broadcom.net/T/#m95c17eb8374e4e3dd6eee700d397aa12cca0739e
[1] https://lore.kernel.org/buildroot/20211001180304.GV1504958@scaer/T/#m3a8f36bd76ec7d8e5038a6c8932bb6ffe23ea268

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-01 20:09:58 +02:00
Yann E. MORIN
6cddc00399 Revert "support/dependencies: don't check for `which'"
This reverts commit a95c353a46.

Switching to using 'command -v' instead of 'which', opened a can of
worms that is hard to fix in a timely manner:

  - recursive call to 'make' from a post-build, post-iamge script, fails
    because of a redefinition of HOSTCC_NOCCACHE (a bug on its own that
    needs a separate fix anyway) [0];

  - 'make' believeing it can call "simple" commands with execve() et al.
    instead of passing them through a shell via system(), and thus
    failing to find 'command' in the PATH [1].

[0] https://lore.kernel.org/buildroot/20211001175329.GA1973888@lbrmn-mmayer.ric.broadcom.net/T/#m95c17eb8374e4e3dd6eee700d397aa12cca0739e
[1] https://lore.kernel.org/buildroot/20211001180304.GV1504958@scaer/T/#m3a8f36bd76ec7d8e5038a6c8932bb6ffe23ea268

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-01 20:09:55 +02:00
Petr Vorel
a95c353a46 support/dependencies: don't check for `which'
It's requirement has been removed in previous commit.

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-26 23:37:21 +02:00
Petr Vorel
ca6a2907c2 make: support: use command -v' instead of which'
`which' has been discontinued after 2.21 release in 2015 due this (git
repository is empty [1]) and version shipped in Debian produces warning
[2]:

/usr/bin/which: this version of `which' is deprecated; use `command -v' in scripts instead.

`command is POSIX [3] and supported on all common shells (bash, zsh,
dash, busybox sh, mksh).

Patch tested on dash as the default shell.

[1] https://git.savannah.gnu.org/cgit/which.git
[2] 3a8dd10b45
[3] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-26 23:37:21 +02:00
Ryota Kinukawa
f46e13f05f support/dependencies, scripts: accept patches with renames
Currently, patches with renames are refused, as they reqire patch 2.7
or newer. So far, we did not require that version because it was too
recent to be widely available.

But patch 2.7 has been released in 2012, almost 9 years ago now; it is
old enough that we can start relying on it.

Add a check that patch is GNU patch 2.7 or newer, and so drop the common
check for patch, and drop the check about renames in apply-patches.sh.

Signed-off-by: Ryota Kinukawa <pojiro.jp@gmail.com>
[yann.morin.1998@free.fr:
  - drop common check
  - shorten variable names
  - drop now-incorrect comment about busybox w/desktop
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 08:33:11 +02:00
Fabrice Fontaine
3ab8aefa87 support/dependencies: set cmake version min to 3.16
domoticz requires cmake 3.16 since version 2020.2 and
275effddf0

Fixes:
 - http://autobuild.buildroot.org/results/0caec85c70341036a039dbc337ad99196b6005a9

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-24 23:10:17 +02:00
Yann E. MORIN
e36974d9e8 support/dependencies: detect and bailout when PATH contains spaces/TABs
In Makefiles, variables are split, filtered, and otherwise mangled on
a space as a separator. In a shell, they will also be split on TABs.

We split and filter and iterate on variables in a lot of places, and
most importantly, spaces in PATH is very seldom tested, if at all, so
a lot of packages will not be working properly in such a situation.

For example, the config.guess contains constructs that are not resilient
to a space in PATH:

    PATH=$PATH:/.attbin ; export PATH

Also, our fakedate will iterate over PATH:

    for P in `echo $PATH | tr ':' ' '`; do

Those are only two cases, but the first means basically all
autotools-based packages are susceptible to subtle breakage.

Furthermore, Buildroot itself does not support that the top-level or
output directories are in a path with spaces anyway.

So, instead of chasing all cases that might be potentially broken,
let's just detect the case and bail out, like we already do when PATH
contains a \n, or when it contains the current working directory.

Reported-by: Dan Raymond <draymond@foxvalley.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-03-14 23:29:15 +01:00
Yann E. MORIN
ec50e407be support/dependencies: drop check for maximal tar version
So far, we checked that the tar present on the host was at most tar
1.29, because tar 1.30 changed the way it generates archives.

Having a maximum tar version requirement meant that we would eventually
always have to build our own host-tar, as distributions are updating
the version they use.

But now, we have found a way to generate reproducible archives starting
with tar 1.27 onward, so we no longer need the check for a maximum tar
version, so we can drop that requirement.

Note: this is semantically a revert of b8fa273d50 (check-host-tar.sh:
blacklist tar 1.30+), but keeping the new, mostly-linear code-path.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
2021-01-10 22:06:58 +01:00
Fabrice Fontaine
37f197f863 support/dependencies: set cmake version min to 3.15
quazip requires cmake 3.15 since version 1.0 and
89e7c201f0
818adc8224

The rationale for this requirement is that "default locations for the
install(TARGETS command based on the GNUInstallDirs package were only
added in 3.14" and "3.15 is not that much of a difference from 3.14 and
it introduced a lot of useful UI improvements.":
https://github.com/stachenov/quazip/issues/82

Fixes:
 - http://autobuild.buildroot.org/results/5d848a46109aef448ea1d1b857a500d9461dc2d9

Note: we also have some patches to allow some packages to build with
cmake-3.10, and this will not be tenable over the long run.

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[yann.morin.1998@free.fr: add the "note"]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-12-25 16:21:27 +01:00
Baruch Siach
1b1c049af2 support/dependencies: clarify intended use of host bison/flex
We should not rely on host installed bison/flex for target code. This
ensures better reproducibility of generated code.

http://lists.busybox.net/pipermail/buildroot/2020-November/296786.html

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-22 15:24:12 +01:00
Peter Korsgaard
42f61e759a support/scripts/apply-patches.sh: do not blindly remove *.orig files
apply-patches currently blindly removes *.orig / .*.orig files as GNU patch
by default writes these as backup files when patches only apply with fuzz.

This is unfortunate as package sources may contain files ending in .orig as
well, breaking the build.  Luckily GNU patch can be told to not write these
backup files using the --no-backup-if-mismatch option, so used that instead
of the .orig removal step.

--no-backup-if-mismatch is supported since GNU patch 2.3.8 (1997-06-17) and
busybox patch if built with CONFIG_DESKTOP, but E.G.  isn't supported by the
BSD patch, so add logic to dependencies.sh to error out if patch doesn't
support the flag.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-10-06 23:40:25 +02:00
Kamel Bouhara
e068f33700 support/dependencies: add BR2_NEEDS_HOST_GCC_PLUGIN_SUPPORT
Some packages requires support on the build machine to create gcc
plugins. This commit adds a blind option,
BR2_NEEDS_HOST_GCC_PLUGIN_SUPPORT, which such packages can
select. When this option is enabled, the logic in support/dependencies
verifies that everything needed on the build machine to build gcc
plugins is available.

Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-07-12 11:20:27 +02:00
Peter Korsgaard
22e833af5e Config.in: drop BR2_NEEDS_HOST_{JAVAC,JAR}
With classpath removed, no packages select these symbols any more - So drop
them and their corresponding logic in dependencies.sh / genrandconfig.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-03-03 23:55:48 +01:00
Yann E. MORIN
a95ea4b64d core/dependencies: check if we need to build our own host-coreutils
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-01-06 21:43:38 +01:00
Thomas Petazzoni
2e42840e2a package/pkg-utils.mk: rework implementation of extractor-system-dependency
Now that we have the EXTRACTOR_PKG_DEPENDENCY.* variables available,
we can use them to implement extractor-system-dependency: if for a
given archive type, the corresponding EXTRACTOR_PKG_DEPENDENCY.<type>
variable is empty, then it means we need the corresponding extractor
tool to be provided by the system.

Following this, EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS is no
longer used, so we can drop it from support/dependencies/.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2019-12-17 22:09:48 +01:00
James Hilliard
d4773b610a support/dependencies: set cmake version min to 3.10
This is required by wpewebkit and webkitgtk.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-28 23:01:47 +01:00
Yann E. MORIN
5ee815ab6a docs/manual: bump our gcc/g++ requirements to 4.8
Currently, we only require a gcc 4.4 version, which now is pretty old
(released in April 2009).  This requirement is not even tested nowadays,
with our oldest autobuilder having a 4.7 version only.

And even then, 4.7 is still old enough that it prevents us from
upgrading some packages. For example cmake 3.10+ requires C++11
constructs that were only added in gcc 4.8 (when C++11 support was
finally completed in gcc).

So, update our requirements for gcc to at least 4.8.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-10-27 10:00:29 +01:00
Yann E. MORIN
9c0c7846cd support/dependencies: don't check for python on the host
We no longer have anything that needs it during the build, so we don't
require it anymore.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-10-26 22:52:15 +02:00
Adrian Perez de Castro
e0c879509d support/dependencies/dependencies.sh: check for JSON:PP Perl module
The JSON::PP Perl module is used at build time by the webkitgtk and
wpewebkit packages.

Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-17 22:36:42 +02:00
Vadim Kochan
28878798cf support/dependencies: Add hint for ArchLinux when host needs IA32 libs
Add hint about which package needs to be installed to provide IA32 libs
support for the host when it is needed.

Signed-off-by: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-05-10 11:04:09 +02:00
Markus Mayer
09cb4ea933 support/dependencies/dependencies.sh: simplify an error message
There is no need to break the "\n" sequence using "%sn". We can just
escape it. Note: the escaping backslash needs to be escaped too,
because the shell will process the string before printf gets to see it.

Signed-off-by: Markus Mayer <mmayer@broadcom.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-04-10 07:45:30 +02:00
Romain Naour
b60729784a support/dependencies: add a check for python3
Since version 2.29, glibc requires python 3.4 or later to build the
GNU C Library [1].

We add a new check to verify the version of python3 interpreter
installed on the host.  If no suitable python3 interpreter is found,
define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package
dependencies when needed.

[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Adam Duskett <aduskett@gmail.com>
Reviewed-by: Adam Duskett <aduskett@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: drop not so useful comment in the .mk file, as suggested by
Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-15 22:26:40 +01:00
Fabrice Fontaine
2eb31511f5 support/dependencies: require at least CMake 3.8
gerbera package in version 1.3 unfortunately now requires CMake >= 3.8
for C++17 macros:
b5fd39f30f

So we need to bump our requirement from 3.1 to 3.8. If the host doesn't
have a CMake >= 3.8, Buildroot will build its own host-cmake package.

Also drop patch that relax cmake requirement on json-for-modern-cpp

Fixes:
 - http://autobuild.buildroot.org/results/6405647b47b132ff5d0d211b92d407322d52d507

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-03-01 10:10:30 +01:00
Bryce Ferguson
3f1b965bc3 infra: add force build flag for host dependencies
This commit adds a config option which will force buildroot to
build all host dependencies even if they are already present on the
host system. This may be a desirable option if different hosts are
used to build the same source. In this case, some packages will be
built on one host that are not built on another. This is problematic
if build source archives are cached afterwards for offline builds.

Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Bryce Ferguson <bryce.ferguson@rockwellcollins.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Peter: reword, drop exit 1, reshuffle]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-04 15:52:44 +01:00
Adam Duskett
6809f38351 dependencies.sh: Check for a host python version >= 2.7
Older distributions such as CentOS6 come with python2.6, which causes build
failures in packages such as host-libglib2 because they require python2.7 and
above.

host-libglib2 will produce the error message:
/bin/sh: python2.7: command not found

Python2.7 is a hard-coded value in configure.ac. If one changes the value to
just "python," the following stack trace is produced:

Traceback (most recent call last):
  File "./gdbus-2.0/codegen/gdbus-codegen.in", line 55, in <module>
    self.outfile.write(LICENSE_STR.format(config.VERSION))
ValueError    : sys.exit(codegen_main.codegen_main())
zero length field name in format

Instead of supporting an ancient version of Python that had its support ended
in October os 2013, it would be more pragmatic only to support Python2.7 and
above.

Luckily; CentOS6 has the centos-release-scl repository, which allows users to
install python2.7, and Debian 8 comes with Python2.7 already, making this patch
relatively low impact.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[Peter: only look at major.minor to handle x.y.z with z < 10]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-04 13:22:55 +01:00
Peter Korsgaard
13c43455a0 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-02 08:16:10 +01:00
Yann E. MORIN
2218dc85be support/dependencies: add a check for a suitable gzip
Recently, some hash mismatch have been reported, both by users as well
as autobuilder failures, about tarballs generated from git repositories.

This turned out to be caused by users having the 'gzip' command somehow
aliased to 'pigz' (which stand for: parallel implementation of gzip,
which takes advantage of multi-processor system to parallelise the
compression).

Unfortunately, the output of pigz-compressed archives differ from that
of gzip (even though they *are* valid gzip-compressed streams).

Add a dependency check that ensures that gzip is not pigz. If that is
the case, define a conditional dependency to host-gzip, that is used as
a download dependency for packages that will generate compressed files,
i.e. cvs, git, and svn.

Fixes:
    http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Marcin Niestrój <m.niestroj@grinn-global.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 15:14:58 +01:00
Yann E. MORIN
c3af086395 support/dependencies: treat BSD-tar like the other cases
Currently, when we detect that tar is BSD-tar, we fake an unsupported
version (major, minor) and rely on the version check to reject BSD-tar.

There is no reason to use such shenanigans, when we can simply reject it
from the onset.

Simplify the logic:
  - use positive logic in the condition
  - directly exit in error

Also, comment that case like the other cases are commented.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 15:14:50 +01:00
Peter Korsgaard
446a6f91b2 make: configure host-make with host- prefix
Fixes:
http://autobuild.buildroot.net/results/e29/e293aadc692d2ed337881ef2172ddf66a60bc05c/

And many more.

Install as 'host-make' rather than just 'make', as that otherwise confuses a
number of packages when they invoke recursive / sub-make.  The internal job
control logic of GNU make is version dependant, so mixing versions may lead
to issues like:

make[1]: Entering directory `/home/peko/autobuild/instance-0/output/build/boa-0.94.14rc21'
(cd src && make -w --jobserver-fds=5,6 -j)
make: unrecognized option '--jobserver-fds=5,6'

With this rename, only packages explicitly opting in for our host-make
(using the BR2_MAKE / BR2_MAKE_HOST_DEPENDENCY logic) will use it.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-20 08:17:43 +01:00
Romain Naour
f6db3df799 dependencies: host-make version check
The host make program is already checked by dependencies.sh but we
want to check the version number even if Buildroot is able to use
GNU make >= 3.81 but some packages may require a more recent version.

For example, since version 2.28 [1], glibc requires GNU Make >= 4.0.

For packages requiring make >= 4.0, the package makefile must use:
<PKG>_DEPENDENCIES = $(BR2_MAKE_HOST_DEPENDENCY) ...
<PKG>_MAKE = $(BR2_MAKE)

BR2_MAKE1 is also available if needed.

[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Thomas: remove extraction of "bugfix" part of the version, since it's
not used anywhere.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-08 23:36:36 +02:00
Yann E. MORIN
3ab56f8a63 support/dependencies: check for system-provided bison and flex
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-20 16:01:42 +02:00
David De Grave (Essensium/Mind)
d0505b849f support/dependencies/dependencies.sh: add details for Redhat/Fedora
Add the i686 package list to install when using pre-built 32 bits
binaries with a redhat/fedora host distribution (glibc.i686 and
zlib.i686).

Signed-off-by: David De Grave (Essensium/Mind) <david.degrave@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-07-02 22:43:56 +02:00
Fabrice Fontaine
dc5553d2ce dependencies.sh: add perl autodie for whois
Add dependency on perl autodie if whois package is selected

Fixes:
 - http://autobuild.buildroot.net/results/4d2ca14580cbc73f94e86ee834fac6d2541298ec

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-05 22:20:18 +02:00
Yann E. MORIN
72703d02b9 support/dependencies: check that PATH does not contain CWD
A person on IRC reported a build failure with the util-linux package,
looking like this:

for I in uname26 linux32 linux64        ; do \
	cd /home/aep/consulting/chargery/tracker/output/target/usr/bin && ln -sf setarch $I ; \
done
[...]
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error

The issue was an empty path in the PATH variable, which means "current
working directory", causing a "ln" binary built by util-linux for the
target to be used instead of the system-provided "ln".

We already check a number of things in the PATH and LD_LIBRARY_PATH
variables in support/dependencies/dependencies.sh, but we were not
checking that PATH did not contain an empty path.

This commit fixes that and takes this opportunity to simplify the test
code for PATH and LD_LIBRARY_PATH.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Thomas: improve commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-09 23:26:18 +02:00
Peter Korsgaard
cdac332d20 check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths
Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e

tar 1.27 subtly changed the tar format when a GNU long link entry is added
(which is done for path elements > 100 characters).  The code used to set
the permission mode of the link entry to 0:

  header = start_private_header ("././@LongLink", size, time (NULL));
  FILL (header->header.mtime, '0');
  FILL (header->header.mode, '0');
  FILL (header->header.uid, '0');
  FILL (header->header.gid, '0');
  FILL (header->header.devmajor, 0);
  FILL (header->header.devminor, 0);

This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with
negative and out-of-range integers), so the settings from
start_private_header() are used directly - Which are:

  TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),
		 header->header.mtime);
  MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode);
  UID_TO_CHARS (0, header->header.uid);
  GID_TO_CHARS (0, header->header.gid);

The end result is that tar >= 1.27 sets mode to 644.

The consequence of this is that we create different tar files when long path
names are encountered (which often happens when a package downloads a
specific sha1 from a git repo) depending on the host tar version used,
causing hash mismatches.

As a workaround, bump our minimum tar version to 1.27.  It would be nicer to
only do this if we have packages from bzr/git/hg enabled, but that is an
exercise for later.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-02 14:46:52 +02:00
Thomas Petazzoni
ea9669fffa core: kill DEPENDENCIES_HOST_PREREQ
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:52:06 +02:00
Thomas Petazzoni
0283d7ee39 package/pkg-generic: handle host-ccache as a regular dependency
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:

 - The extractor packages host-tar, host-xz and host-lzip

 - host-ccache itself

 - host-skeleton, because all packages depend on it

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:49:05 +02:00
Thomas Petazzoni
15815e43dd package/pkg-generic: handle host-lzip as an extract dependency
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:

 - host-lzip, because we would otherwise depend on ourself.

 - host-tar, because lzip itself is delivered as a tarball, so we need
   to have host-lzip depend on host-tar, and not host-tar depend on
   host-lzip

 - host-skeleton, because we need to have host-lzip depend on
   host-skeleton, and not the opposite.

We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.

In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:45:58 +02:00
Thomas Petazzoni
f13fdc45e8 package/pkg-generic: handle host-xz as an extract dependency
This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:

 - host-xz, because we would otherwise depend on ourself.

 - host-tar, because xz itself is delivered as a tarball, so we need
   to have host-xz depend on host-tar, and not host-tar depend on
   host-xz

 - host-skeleton, because we need to have host-xz depend on
   host-skeleton, and not the opposite.

In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-xz/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:44:04 +02:00
Thomas Petazzoni
e77013a3c4 package/pkg-generic: handle host-tar as an extract dependency
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.

In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.

Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:43:20 +02:00
Yann E. MORIN
b1aa2148b0 core: drop no-longer used C.UTF-8 locale option
Its use has been globbed into the more generic
BR2_NEEDS_HOST_UTF8_LOCALE option now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Julius Kriukas <julius@kriukas.lt>
Cc: Christian Stewart <kidovate@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:59:16 +01:00
Yann E. MORIN
9f8c7766e4 support/dependencies: unbreak check for UTF-8 locale
Although the UTF-8 locales in mainstream distributions all are suffixed
with just 'utf8', the nomenclature is a bit ambiguous with the way they
are to be specified with the various LC_* variables, suffixed there with
'UTF-8'.

Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_*
variables:
    http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02

    """
    If the locale value has the form:
        language[_territory][.codeset]

    it refers to an implementation-provided locale, where settings of
    language, territory, and codeset are implementation-defined.
    """

To avoid any confusion, use a regexp that is a bit more lax when
matching locales.

Also, quote the regexp, so that the '?' and '$' are not interpreted by
the shell.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:46:48 +01:00
Thomas Petazzoni
8cde7e6af9 support/dependencies/dependencies.sh: check for Python argparse module
The script support/scripts/check-uniq-files uses the argparse Python
module. In most recent Python versions (starting with 2.7), the
argparse module is part of the standard library, and we already check
for the availability of Python in
support/dependencies/dependencies.sh.

However, when running on an ancient distribution with Python 2.6, the
argparse module is not part of the Python standard library, but
available as an external module. Without this module, the build fails,
because check-uniq-files, which is used in target-finalize, fails to
run.

To avoid this failure, this commit adds a check in
support/dependencies/dependencies.sh to verify that the argparse
module is available.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-03 17:18:18 +01:00
Julius Kriukas
41569717c3 systemd: switch to C.UTF-8 locale when building
When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale
building systemd fails with an error:

[1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command.
FAILED: catalog/systemd.bg.catalog
/buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128)
package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed
make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1

This patch changes default host system locale from C to C.UTF-8 when
building systemd package to fix this issue. It also introduces
BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on
the host system. If locale is not available error message is show and build
process is stopped.

Signed-off-by: Julius Kriukas <julius@kriukas.lt>
[Thomas: use C.UTF-8 instead of en_US.UTF-8.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-02 22:44:34 +01:00
Peter Korsgaard
b8fa273d50 check-host-tar.sh: blacklist tar 1.30+
Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git.  This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.

For more details, see:
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html

To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:02:26 +01:00
Peter Korsgaard
7c09cb82b7 dependencies.mk: check for valid host-tar before other host dependencies
host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.

With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:01:47 +01:00
Gaël PORTAY
27d2229692 support/dependencies: add check for c++ multilib
Some packages build C++ 32bits host-tools and need the g++-multilib to
be installed on the build machine. As example, qt5webengine builds a C++
host-tool when target is 32bits.

Add the check for g++-multilib to the dependencies script; and update
the Dockerfile to install g++-multilib package.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-06 17:21:09 +01:00
Arnout Vandecappelle (Essensium/Mind)
390780d562 dependencies: correct dependencies for top-level parallel build
Commit 4932c8a7cc introduced the
core-dependencies target to make sure that the dependencies.sh script
runs before we attempt to compile any host tool, so that the absence of
a compiler is properly detected. However, this relied on the
left-to-right evaluation of dependencies. This will no longer be true
when we enable top-level parallel build.

Fix this by letting DEPENDENCIES_HOST_PREREQ depend on
core-dependencies.

Note that it is not possible to remove the
dependencies <- core-dependencies. Indeed, it is possible that
DEPENDENCIES_HOST_PREREQ is completely empty, and in that case we still
need to check core-dependencies.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:37:59 +01:00
Arnout Vandecappelle (Essensium/Mind)
42decc07ce core-dependencies: remove unneeded HOSTCC
dependencies.sh uses HOSTCC_NOCCACHE directly, and this variable is
exported from the top-level Makefile, so there is no need to pass
HOSTCC to it. HOSTCC is not used at all in dependencies.sh.

Thus, we also no longer need to apply the HOSTCC override for
core-dependencies. The core-depencies rule doesn't use HOSTCC or
HOSTCXX.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:37:54 +01:00
Alfredo Alvarez Fernandez
36d398ac30 dependencies: always use HOSTCC_NOCACHE for DEPENDENCIES_HOST_PREREQ
Currently, HOSTCC and HOSTCXX are set to their _NOCACHE variants in the
'dependencies' target. This is needed because at that time, ccache is
not built yet - host-ccache is one of the dependencies. However, because
this override is only specified for the 'dependencies' target (and
thereby gets inherited by its dependencies), the override is only
applied when the package is reached through the 'dependencies' target.
This is not the case when one of DEPENDENCIES_HOST_PREREQ is built
directly from the command line, e.g. when doing 'make host-ccache'. So
in that case, ccache will be built with ccache... which fails of
course.

To fix this, directly apply the override to the DEPENCIES_HOST_PREREQ
targets.

Note that this only fixes the issue for 'make host-ccache', NOT for
e.g. 'make host-ccache-configure'.

Signed-off-by: Alfredo Alvarez Fernandez <alfredo.alvarez_fernandez@nokia.com>
[Arnout: improve commit message]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-10-22 15:42:26 +02:00