Recently, some hash mismatch have been reported, both by users as well
as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow
aliased to 'pigz' (which stand for: parallel implementation of gzip,
which takes advantage of multi-processor system to parallelise the
compression).
Unfortunately, the output of pigz-compressed archives differ from that
of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is
the case, define a conditional dependency to host-gzip, that is used as
a download dependency for packages that will generate compressed files,
i.e. cvs, git, and svn.
Fixes:
http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Marcin Niestrój <m.niestroj@grinn-global.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, when we detect that tar is BSD-tar, we fake an unsupported
version (major, minor) and rely on the version check to reject BSD-tar.
There is no reason to use such shenanigans, when we can simply reject it
from the onset.
Simplify the logic:
- use positive logic in the condition
- directly exit in error
Also, comment that case like the other cases are commented.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
http://autobuild.buildroot.net/results/e29/e293aadc692d2ed337881ef2172ddf66a60bc05c/
And many more.
Install as 'host-make' rather than just 'make', as that otherwise confuses a
number of packages when they invoke recursive / sub-make. The internal job
control logic of GNU make is version dependant, so mixing versions may lead
to issues like:
make[1]: Entering directory `/home/peko/autobuild/instance-0/output/build/boa-0.94.14rc21'
(cd src && make -w --jobserver-fds=5,6 -j)
make: unrecognized option '--jobserver-fds=5,6'
With this rename, only packages explicitly opting in for our host-make
(using the BR2_MAKE / BR2_MAKE_HOST_DEPENDENCY logic) will use it.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The host make program is already checked by dependencies.sh but we
want to check the version number even if Buildroot is able to use
GNU make >= 3.81 but some packages may require a more recent version.
For example, since version 2.28 [1], glibc requires GNU Make >= 4.0.
For packages requiring make >= 4.0, the package makefile must use:
<PKG>_DEPENDENCIES = $(BR2_MAKE_HOST_DEPENDENCY) ...
<PKG>_MAKE = $(BR2_MAKE)
BR2_MAKE1 is also available if needed.
[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Thomas: remove extraction of "bugfix" part of the version, since it's
not used anywhere.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add the i686 package list to install when using pre-built 32 bits
binaries with a redhat/fedora host distribution (glibc.i686 and
zlib.i686).
Signed-off-by: David De Grave (Essensium/Mind) <david.degrave@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
A person on IRC reported a build failure with the util-linux package,
looking like this:
for I in uname26 linux32 linux64 ; do \
cd /home/aep/consulting/chargery/tracker/output/target/usr/bin && ln -sf setarch $I ; \
done
[...]
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
The issue was an empty path in the PATH variable, which means "current
working directory", causing a "ln" binary built by util-linux for the
target to be used instead of the system-provided "ln".
We already check a number of things in the PATH and LD_LIBRARY_PATH
variables in support/dependencies/dependencies.sh, but we were not
checking that PATH did not contain an empty path.
This commit fixes that and takes this opportunity to simplify the test
code for PATH and LD_LIBRARY_PATH.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Thomas: improve commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e
tar 1.27 subtly changed the tar format when a GNU long link entry is added
(which is done for path elements > 100 characters). The code used to set
the permission mode of the link entry to 0:
header = start_private_header ("././@LongLink", size, time (NULL));
FILL (header->header.mtime, '0');
FILL (header->header.mode, '0');
FILL (header->header.uid, '0');
FILL (header->header.gid, '0');
FILL (header->header.devmajor, 0);
FILL (header->header.devminor, 0);
This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with
negative and out-of-range integers), so the settings from
start_private_header() are used directly - Which are:
TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),
header->header.mtime);
MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode);
UID_TO_CHARS (0, header->header.uid);
GID_TO_CHARS (0, header->header.gid);
The end result is that tar >= 1.27 sets mode to 644.
The consequence of this is that we create different tar files when long path
names are encountered (which often happens when a package downloads a
specific sha1 from a git repo) depending on the host tar version used,
causing hash mismatches.
As a workaround, bump our minimum tar version to 1.27. It would be nicer to
only do this if we have packages from bzr/git/hg enabled, but that is an
exercise for later.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:
- The extractor packages host-tar, host-xz and host-lzip
- host-ccache itself
- host-skeleton, because all packages depend on it
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:
- host-lzip, because we would otherwise depend on ourself.
- host-tar, because lzip itself is delivered as a tarball, so we need
to have host-lzip depend on host-tar, and not host-tar depend on
host-lzip
- host-skeleton, because we need to have host-lzip depend on
host-skeleton, and not the opposite.
We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.
In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.
[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:
- host-xz, because we would otherwise depend on ourself.
- host-tar, because xz itself is delivered as a tarball, so we need
to have host-xz depend on host-tar, and not host-tar depend on
host-xz
- host-skeleton, because we need to have host-xz depend on
host-skeleton, and not the opposite.
In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.
[Peter: fix s/host-tar/host-xz/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.
In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.
Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Its use has been globbed into the more generic
BR2_NEEDS_HOST_UTF8_LOCALE option now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Julius Kriukas <julius@kriukas.lt>
Cc: Christian Stewart <kidovate@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Although the UTF-8 locales in mainstream distributions all are suffixed
with just 'utf8', the nomenclature is a bit ambiguous with the way they
are to be specified with the various LC_* variables, suffixed there with
'UTF-8'.
Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_*
variables:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02
"""
If the locale value has the form:
language[_territory][.codeset]
it refers to an implementation-provided locale, where settings of
language, territory, and codeset are implementation-defined.
"""
To avoid any confusion, use a regexp that is a bit more lax when
matching locales.
Also, quote the regexp, so that the '?' and '$' are not interpreted by
the shell.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The script support/scripts/check-uniq-files uses the argparse Python
module. In most recent Python versions (starting with 2.7), the
argparse module is part of the standard library, and we already check
for the availability of Python in
support/dependencies/dependencies.sh.
However, when running on an ancient distribution with Python 2.6, the
argparse module is not part of the Python standard library, but
available as an external module. Without this module, the build fails,
because check-uniq-files, which is used in target-finalize, fails to
run.
To avoid this failure, this commit adds a check in
support/dependencies/dependencies.sh to verify that the argparse
module is available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale
building systemd fails with an error:
[1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command.
FAILED: catalog/systemd.bg.catalog
/buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128)
package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed
make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1
This patch changes default host system locale from C to C.UTF-8 when
building systemd package to fix this issue. It also introduces
BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on
the host system. If locale is not available error message is show and build
process is stopped.
Signed-off-by: Julius Kriukas <julius@kriukas.lt>
[Thomas: use C.UTF-8 instead of en_US.UTF-8.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git. This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.
For more details, see:
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html
To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.
With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.
The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some packages build C++ 32bits host-tools and need the g++-multilib to
be installed on the build machine. As example, qt5webengine builds a C++
host-tool when target is 32bits.
Add the check for g++-multilib to the dependencies script; and update
the Dockerfile to install g++-multilib package.
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 4932c8a7cc introduced the
core-dependencies target to make sure that the dependencies.sh script
runs before we attempt to compile any host tool, so that the absence of
a compiler is properly detected. However, this relied on the
left-to-right evaluation of dependencies. This will no longer be true
when we enable top-level parallel build.
Fix this by letting DEPENDENCIES_HOST_PREREQ depend on
core-dependencies.
Note that it is not possible to remove the
dependencies <- core-dependencies. Indeed, it is possible that
DEPENDENCIES_HOST_PREREQ is completely empty, and in that case we still
need to check core-dependencies.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
dependencies.sh uses HOSTCC_NOCCACHE directly, and this variable is
exported from the top-level Makefile, so there is no need to pass
HOSTCC to it. HOSTCC is not used at all in dependencies.sh.
Thus, we also no longer need to apply the HOSTCC override for
core-dependencies. The core-depencies rule doesn't use HOSTCC or
HOSTCXX.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, HOSTCC and HOSTCXX are set to their _NOCACHE variants in the
'dependencies' target. This is needed because at that time, ccache is
not built yet - host-ccache is one of the dependencies. However, because
this override is only specified for the 'dependencies' target (and
thereby gets inherited by its dependencies), the override is only
applied when the package is reached through the 'dependencies' target.
This is not the case when one of DEPENDENCIES_HOST_PREREQ is built
directly from the command line, e.g. when doing 'make host-ccache'. So
in that case, ccache will be built with ccache... which fails of
course.
To fix this, directly apply the override to the DEPENCIES_HOST_PREREQ
targets.
Note that this only fixes the issue for 'make host-ccache', NOT for
e.g. 'make host-ccache-configure'.
Signed-off-by: Alfredo Alvarez Fernandez <alfredo.alvarez_fernandez@nokia.com>
[Arnout: improve commit message]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is useful on CentOS 7, whose "cmake" utility corresponds to version
2.8.12, which is too old for Buildroot.
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add the BR2_CMAKE_CANDIDATES variable, containing a list of candidates
to check and use as BR2_CMAKE, if possible.
This allows using "cmake3" on CentOS 7, whose default cmake corresponds
to version 2.8.12. Example:
$ make BR2_CMAKE_CANDIDATES="cmake cmake3"
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is useful on CentOS 7 whose "cmake" package provides cmake 2.8.12,
which is too old, but the "cmake3" package (from EPEL) provides version
3.6.3, which is satisfactory. Examples:
$ sh support/dependencies/check-host-cmake.sh 2.8 cmake cmake3
/usr/bin/cmake
$ sh support/dependencies/check-host-cmake.sh 3.1 cmake cmake3
/usr/bin/cmake3
$ sh support/dependencies/check-host-cmake.sh 3.8 cmake cmake3
(nothing)
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Pass the minimal version before the program name. In a later change the
script will become able to test a list of candidates.
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... otherwise it fails spectacularly as soon as PATH is referenced in a
package rule (i.e. very soon, fortunately):
>>> host-lzip 1.18 Downloading
/bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
/bin/bash: -c: line 1: syntax error: unexpected end of file
Fixes # 9886.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ciro Santilli <ciro.santilli@gmail.com>
[Thomas: fix typo in message, use tabs for indentation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This reverts commit 4422eca2d4.
We now have a workaround for the RPATH issue introduced in 3.7, so we
can use it again.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
cmake-3.7 has a bug in how it handles rpath, linking with libraries from
the host.
Until we completely understand the issue, just blacklist cmake-3.7.
The issue has been reported upstream:
http://public.kitware.com/pipermail/cmake/2017-February/064970.html
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit teaches the generic package handling code how to extract .tar.lz
archives. When lzip is not installed on the host, host-lzip gets built
automatically.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Don't special case $(XZCAT) when constructing DL_TOOLS_DEPENDENCIES. The next
commit will introduce another extractor that automatically builds when not
installed. Introduce EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS that lists
archive extensions for which the extractor is already checked in
support/dependencies/check-host-foo.mk. Use this in the newly introduced
extractor-dependency to populate DL_TOOLS_DEPENDENCIES.
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Thomas: add missing space after "firstword", as noticed by Thomas DS.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Usually, Buildroot does never initialize variables with empty content.
DEPENDENCIES_HOST_PREREQ was an unjustified exception.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Recently, the autoconf macros for libtool started using '/usr/bin/file'
to determine the type of library that is generated by the toolchain.
Packages that use this recent version of the libtool autoconf macros
will fail in a rather dramatic way when /usr/bin/file is not present
on the host: the package will still build but no shared library is
generated, which in turn may cause build failures in other packages
that link with it.
For example, libpng's configure determines that it is not possible to
build a shared library on MIPS64 because the expected output from 'file'
is not present. Therefore, only a static libpng.a is built. Later,
bandwithd links with -lpng but it doesn't use the pkg-config's
Private-Libs (because it's not linking statically) and it doesn't have
access to the NEEDED reference from the shared library. Therefore, it
doesn't link with zlib and fails with
pngrutil.c:(.text+0x55c): undefined reference to `inflate'
We cant use host-file because it is itself an autotools package and is
itself using libtool, so this would be a chicken-n-egg problem. Besides,
the libtool script really wants to call /usr/bin/file, so it would not
even find our host-file anyway.
So, just require that '/usr/bin/file' is present on the host.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we require a host with an UTF-8 locale as soone as we build a
Buildroot toolchain with support for locales. This means that we do
erquire such a locale when building a toolchain with either uClibc,
glibc or musl.
However, glibc and musl do not require such a locale to be present.
Use the new option to check if an UTF8 locale is needed on the host,
rather than deriving that from toolchain settings.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
ExtUtils::MakeMaker is one of the Perl Core modules usually packaged in
Perl package for a Debian/Ubuntu based system.
For a Fedora based system, each Perl Core modules have their own RPM
package. So install only Perl package is not enough.
Fixes:
>>> host-libxml-parser-perl 2.41 Configuring
[...]
perl `which perl` Makefile.PL
Can't locate ExtUtils/MakeMaker.pm in @INC (you may need to install the ExtUtils::MakeMaker module)
Add a new Perl module check in dependency.sh.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: François Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently all cmake packages depend on host-cmake. Unfortunately
host-cmake takes a long time to configure and build: almost 7 minutes
on a dual-core i5 with SSD. The time does not change even with ccache
enabled.
Indeed, building host-cmake is avoidable if it is already installed on
the build host: CMake is supposed to be quite portable, and the only
patch in Buildroot for the CMake package seems to only affect
target-cmake.
Thus we automatically skip building host-cmake and use the one on the
system if:
- cmake is available on the system and
- it is recent enough.
First, we leverage the existing infrastructure in
support/dependencies/dependencies.mk to find out whether there's a
suitable cmake executable on the system. Its path can be passed in the
BR2_CMAKE environment variable, otherwise it defaults to "cmake". If
it is enabled, found and suitable then we set BR2_CMAKE_HOST_DEPENDENCY
to empty; otherwise we set BR2_CMAKE_HOST_DEPENDENCY to 'host-cmake' and
override BR2_CMAKE with "$(HOST_DIR)/usr/bin/cmake" to revert to using
our own cmake (the old behaviour).
Then in pkg-cmake.mk we replace the hard-coded dependency on host-cmake
to using the BR2_CMAKE_HOST_DEPENDENCY variable, and we use $(BR2_CMAKE)
instead of $(HOST_DIR)/usr/bin/cmake.
Unlike what we do for host-tar and host-xzcat, for host-cmake we do
not add host-cmake to DEPENDENCIES_HOST_PREREQ. If we did, host-cmake
would be a dependency for _any_ package when it's not installed on the
host, even when no cmake package is selected.
Cmake versions older than 3.0 are affected by the bug described and
fixed in Buildroot in ef2c1970e4 ("cmake: add patch to fix Qt mkspecs
detection"). The bug was fixed in upstream CMake in version 3.0 [0].
Amongst all the cmake packages currently in Buildroot, the currently
highest version mentioned in cmake_minimum_required() is 3.1 (grantlee
and opencv3).
Thus we use 3.1 as the lowest required cmake for now, until a package is
bumped, or a new package added, with a higher required version.
[0] https://cmake.org/gitweb?p=cmake.git;h=e8b8b37ef6fef094940d3384df5a1d421b9fa568
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Davide Viti <zinosat@tiscali.it>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
[yann.morin.1998@free.fr:
- simplify logic in check-host-cmake.mk;
- set and use BR2_CMAKE_HOST_DEPENDENCY, drop USE_SYSTEM_CMAKE;
- bump to cmake 3.1 for grantlee and opencv;
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Thomas:
- Check for Thread::Queue, not Thread:Queue.
- Use 'printf' instead of 'echo -e', since printf is POSIX, but not
'echo -e'.]
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The introduction of <pkg>_STRIP_COMPONENTS broke the build of the
target tar package, because support/dependencies/check-host-tar.mk
defines TAR_STRIP_COMPONENTS to --strip-components. Which leads to
have the package infrastructure do:
$$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS)
which for the tar package evaluates to:
$$(TAR_STRIP_COMPONENTS)=$$(TAR_STRIP_COMPONENTS)
which evalutes to:
--strip-components=--strip-components
Which obviously doesn't work really well. And in fact the
TAR_STRIP_COMPONENTS definition in
support/dependencies/check-host-tar.mk is no longer necessary: it was
needed in the days where we were trying to support old tar versions
that did not support --strip-components. But nowadays, when such an
old tar version is encountered, we build our own host-tar which
supports --strip-components.
Fixes:
http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The screen is cluttered when we build for 32 bit target and 32 bit gcc
is missing.
~/buildroot$ make
[...]
/usr/bin/ld: cannot find crt1.o: No such file or directory
/usr/bin/ld: cannot find crti.o: No such file or directory
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find -lc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
[...]
Your Buildroot configuration needs a compiler capable of building 32 bits binaries.
The final note is enough, and adding 2>/dev/null to the gcc test
invocation is also more consistent with the rest of the script. The
patch makes the '/usr/bin/ld:' and 'collect2:' lines go away.
Signed-off-by: Jens Stimpfle <debian@jstimpfle.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When g++ is not installed, a misleading error message turns up because
of a bad combination of an unquoted shell variable and control flow.
~/buildroot$ make
You may have to install 'g++' on your build machine
/home/testuser/buildroot/support/dependencies/dependencies.sh: 136: [: -lt: unexpected operator
[Thomas:
- fixed commit log, as per the suggestion of Yann E. Morin.
- don't change existing empty new lines, suggested by Yann.
- use positive logic in the newly added test, suggested by Yann.]
Signed-off-by: Jens Stimpfle <debian@jstimpfle.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When we set LD_LIBRARY_PATH when building our host tools, we append any
pre-existing value to our custom path:
LD_LIBRARY_PATH="$(HOST_DIR)/usr/lib:$(LD_LIBRARY_PATH)"
But then if LD_LIBRARY_PATH was previously empty, we end up with an
LD_LIBRARY_PATH that ends with a colon.
Also, when we check that an existing LD_LIBRARY_PATH does not contain
CWD, we previously did not look for a zero-length prefix.
Since 'man ld.so' says of LD_LIBRARY_PATH:
A colon-separated list of directories in which to search for ELF
libraries at execution-time. Similar to the PATH environment
variable.
And POSIX states about PATH:
A zero-length prefix is a legacy feature that indicates the current
working directory.
And bash also recognises a zero-length prefix to search in CWD:
A zero-length (null) directory name in the value of PATH indicates
the current directory.
We may thus end up on a system where a zero-length prefix in
LD_LIBRARY_PATH is interpreted as CWD.
Do not append the previous LD_LIBRARY_PATH if it was empty, and check
for a zero-length prefix when checking dependencies.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
