In some cases, upstream just update their releases in-place, without
renaming them. When that package is updated in Buildroot, a new hash to
match the new upstream release is included in the corresponding .hash
file.
As a consequence, users who previously downloaded that package's tarball
with an older version of Buildroot, will get stuck with an old archive
for that package, and after updating their Buildroot copy, will be greeted
with a failed download, due to the local file not matching the new
hashes.
Also, an upstream would sometime serve us HTML garbage instead of the
actual tarball we requested, like SourceForge does from time for as-yet
unknown reasons.
So, to avoid this situation, check the hashes prior to doing the
download. If the hashes match, consider the locally cached file genuine,
and do not download it. However, if the locally cached file does not
match the known hashes we have for it, it is promptly removed, and a
download is re-attempted.
Note: this does not add any overhead compared to the previous situation,
because we were already checking hashes of locally cached files. It just
changes the order in which we do the checks. For the records, here is the
overhead of hashing a 231MiB file (qt-everywhere-opensource-src-4.8.6.tar.gz)
on a core-i5 @2.5GHz:
cache-cold cache-hot
sha1 1.914s 0.762s
sha256 2.109s 1.270s
But again, this overhead already existed before this patch.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of repeating the check in our download rules, delegate the check
of the hashes to the download wrapper.
This needs three different changes:
- add a new argument to the download wrapper, that is the full path to
the hash file; if the hash file does not exist, that does not change
the current behaviour, as the existence of the hash file is checked
for in the check-hash script;
- add a third argument to the check-hash script, to be the basename of
the file to check; this is required because we no longer check the
final file with the final filename, but an intermediate file with a
temporary filename;
- do the actual call to the check-hash script from within the download
wrapper.
This further paves the way to doing pre-download checks of the hashes
for the locally cached files.
Note: this patch removes the check for hashes for already downloaded
files, since the wrapper script exits early. The behaviour to check
localy cached files will be restored and enhanced in the following
patch.
[Thomas: fix minor typo in comment.]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of repeating the same test again and again in all our download
rules, just delegate the check for an already downloaded file to the
download wrapper.
This clears up the path for doing the hash checks on a cached file
before the download.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of relying on argument ordering, use actual options in the
download wrapper.
Download backends (bzr, cp, hg...) are left as-is, because it does not
make sense to complexify them, since they are almost very trivial shell
scripts, and adding option parsing would be really overkill.
This commit also renames the script to dl-wrapper so it looks better in
the traces, and it is not confused with another wrapper.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
For the host variant of packages, we normally only build the shared
libraries. However, ncurses uses non-standard options to select
between shared/static and therefore the host variant was building both
of them, even though the static libraries were unused.
By passing --without-normal, we disable the build of static
libraries. It saves a bit of disk space, and on my laptop,
host-ncurses takes 26 seconds to build instead of 40 seconds.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
As noted by Yann E. Morin, those dependencies are unneeded, since FLAT
can anyway only be used on m68k and Blackfin.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In preparation for the refactoring of the static/shared library
support, we add a BR2_BINFMT_SUPPORTS_SHARED hidden option that binary
formats supporting shared libraries should select.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Currently, the binary format choice is only shown for the Blackfin and
m68k architectures, since we assume that all other architectures are
using the ELF binary format. However, due to this, the BR2_BINFMT_ELF
symbol is in fact not set to 'y' for those architectures that use the
ELF format.
This will be causing problems for the refactoring of the static/shared
library support, as we will need to know if the binary format supports
shared libraries or not.
Therefore, we simply make the choice visible on all architectures,
even if it means that on many architectures no other choice than ELF
will be available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
The current binfmt selection in arch/Config.in allows to select FDPIC
on m68k, which is incorrect. This commit fixes that, and makes sure
FDPIC is the default on Blackfin, while FLAT is the default on m68k.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Since a while, the semantic of BR2_PREFER_STATIC_LIB has been changed
from "prefer static libraries when possible" to "use only static
libraries". The former semantic didn't make much sense, since the user
had absolutely no control/idea of which package would use static
libraries, and which packages would not. Therefore, for quite some
time, we have been starting to enforce that BR2_PREFER_STATIC_LIB
should really build everything with static libraries.
As a consequence, this patch renames BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS, and adjust the Config.in option accordingly.
This also helps preparing the addition of other options to select
shared, shared+static or just static.
Note that we have verified that this commit can be reproduced by
simply doing a global rename of BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS plus adding BR2_PREFER_STATIC_LIB to Config.in.legacy.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
glibc-2.20 includes some changes to the include/features.h file
introduced by this commit:
https://sourceware.org/git/?p=glibc.git;a=commit;h=ade40b10ff5fa59a318cf55b9d8414b758e8df78
Those changes make libsvgtiny fail because some warnings are thrown and
the build system is using the -Werror option. We disable this to be able
to build it, or otherwise we will see errors like this one:
GPERF: src/colors.gperf
COMPILE: build-Linux-Linux-release-lib-static/src_colors.c
In file included from
/br/output/host/usr/mipsel-buildroot-linux-gnu/sysroot/usr/include/string.h:25:0,
from src/colors.gperf:16:
/br/output/host/usr/mipsel-buildroot-linux-gnu/sysroot/usr/include/features.h:148:3:
error: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use
_DEFAULT_SOURCE" [-Werror=cpp]
and this one:
In file included from src/colors.gperf:18:0:
/home/ldap/vriera/work/mips-buildroots/mips32/output/build/libsvgtiny-12121/src/svgtiny_internal.h:71:0:
error: "strndup" redefined [-Werror]
Fixes:
http://autobuild.buildroot.net/results/3dd/3dd700405055750262738f867eb5aa08531f5781/
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Reviewed-by: Markos Chandras <Markos.Chandras@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
libuv has moved to github.com/libuv/libuv
[Thomas: fix Config.in help text, as noticed by Baruch.]
Signed-off-by: Jörg Krause <jkrause@posteo.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds a Config.in option to the "Build options" submenu to
enable paranoid checking of unsafe paths. This mechanism is added as
an option so that when we'll enable it in the autobuilders, people
trying to reproduce the build failures will be able to do so by just
downloading the configuration file. If instead we were leaving this
feature as an environment variable, everyone would have to remember to
pass this environment variable to reproduce build issues. And certain
build issues triggered by paranoid unsafe patch checking may not be
visible in the build output, for example when they happen during the
execution of configure scripts.
Since this option is fairly advanced, a new submenu inside "Build
options" is created, for Advanced options.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit enables the poison system directories option, which is now
available thanks to the binutils patches that have been added.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Romain Naour <romain.naour@openwide.fr>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit enables the poison system directories option, which is now
available thanks to the gcc patches that have been added.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Romain Naour <romain.naour@openwide.fr>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to gcc borrowed from CodeSourcery/Yocto that
warns about unsafe include paths (i.e /usr/include,
/usr/local/include, etc.). The patch was adapted to gcc 4.7.4, and
modified to support the BR_COMPILER_PARANOID_UNSAFE_PATH environment
variable to error out instead of just warn when unsafe paths are
used. Even though erroring out can be chosen by passing
-Werror=poison-system-directories, we are not sure this option in
CFLAGS will always be passed, so having an environment variable
guarantees it will always be passed, and also allows to have an
identical behavior to the external toolchain wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to gcc borrowed from CodeSourcery/Yocto that
warns about unsafe include paths (i.e /usr/include,
/usr/local/include, etc.). The patch was adapted to gcc arc-2014.08,
and modified to support the BR_COMPILER_PARANOID_UNSAFE_PATH
environment variable to error out instead of just warn when unsafe
paths are used. Even though erroring out can be chosen by passing
-Werror=poison-system-directories, we are not sure this option in
CFLAGS will always be passed, so having an environment variable
guarantees it will always be passed, and also allows to have an
identical behavior to the external toolchain wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to gcc borrowed from CodeSourcery/Yocto that
warns about unsafe include paths (i.e /usr/include,
/usr/local/include, etc.). The patch was adapted to gcc 4.8.3, and
modified to support the BR_COMPILER_PARANOID_UNSAFE_PATH environment
variable to error out instead of just warn when unsafe paths are
used. Even though erroring out can be chosen by passing
-Werror=poison-system-directories, we are not sure this option in
CFLAGS will always be passed, so having an environment variable
guarantees it will always be passed, and also allows to have an
identical behavior to the external toolchain wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to gcc borrowed from CodeSourcery/Yocto that
warns about unsafe include paths (i.e /usr/include,
/usr/local/include, etc.). The patch was adapted to gcc 4.9.1, and
modified to support the BR_COMPILER_PARANOID_UNSAFE_PATH environment
variable to error out instead of just warn when unsafe paths are
used. Even though erroring out can be chosen by passing
-Werror=poison-system-directories, we are not sure this option in
CFLAGS will always be passed, so having an environment variable
guarantees it will always be passed, and also allows to have an
identical behavior to the external toolchain wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to binutils borrowed from CodeSourcery/Yocto
that warns about unsafe library paths (i.e /usr/lib, /usr/local/lib,
etc.). The patch was adapted to binutils arc-2014.08, and modified to
support the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable to
error out instead of just warn when unsafe paths are used. Even though
erroring out can be chosen by passing
--error-poison-system-directories, we are not sure this option in
LDFLAGS will always be passed, so having an environment variable
guarantees it will always be passed, and also allows to have an
identical behavior to the external toolchain wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to binutils borrowed from CodeSourcery/Yocto
that warns about unsafe library paths (i.e /usr/lib, /usr/local/lib,
etc.). The patch was adapted to binutils 2.22, and modified to support
the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable to error out
instead of just warn when unsafe paths are used. Even though erroring
out can be chosen by passing --error-poison-system-directories, we are
not sure this option in LDFLAGS will always be passed, so having an
environment variable guarantees it will always be passed, and also
allows to have an identical behavior to the external toolchain
wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to binutils borrowed from CodeSourcery/Yocto
that warns about unsafe library paths (i.e /usr/lib, /usr/local/lib,
etc.). The patch was adapted to binutils 2.23, and modified to support
the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable to error out
instead of just warn when unsafe paths are used. Even though erroring
out can be chosen by passing --error-poison-system-directories, we are
not sure this option in LDFLAGS will always be passed, so having an
environment variable guarantees it will always be passed, and also
allows to have an identical behavior to the external toolchain
wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
This commit adds a patch to binutils borrowed from CodeSourcery/Yocto
that warns about unsafe library paths (i.e /usr/lib, /usr/local/lib,
etc.). The patch was adapted to binutils 2.24, and modified to support
the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable to error out
instead of just warn when unsafe paths are used. Even though erroring
out can be chosen by passing --error-poison-system-directories, we are
not sure this option in LDFLAGS will always be passed, so having an
environment variable guarantees it will always be passed, and also
allows to have an identical behavior to the external toolchain
wrapper.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
The CodeSourcery toolchains have a very interesting feature: they warn
the user when an unsafe header or library path is used, i.e a path
that will lead host headers or libraries to leak into the build.
This commit adds a similar functionality into our external toolchain
wrapper, so that it can be used with all external toolchains, and can
also be tuned as needed. By default, the external toolchain wrapper
now gives warnings such as:
arm-linux-gcc: WARNING: unsafe header/library path used in cross-compilation: '-I /usr/foo'
arm-linux-gcc: WARNING: unsafe header/library path used in cross-compilation: '-L /usr/bleh'
but the compilation continues successfully. One can then easily grep
in his build log to search for occurences of this message.
Optionally, if BR_COMPILER_PARANOID_UNSAFE_PATH is defined in the
environment to a non empty value, the external wrapper will instead
error out and abort the compilation.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Tested-by: Romain Naour <romain.naour@openwide.fr>
Also remove the install hook, all static libraries are
removed from TARGET_DIR/usr/lib by target-finalize target
Signed-off-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Thomas: remove useless --without-glib option, as noticed by Vincent.]
Signed-off-by: Ivan Sergeev <vsergeev@kumunetworks.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Uses pkgconfig since 1.16.1+ to find libraries.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Install poppler's header files and libraries into staging, since it
provides a library that can be used by other packages.
[Thomas: slightly extended the commit message.]
Signed-off-by: Alan Ott <alan@signal11.us>
Suggested-by: Jeremie Scheer <jeremie.scheer@armadeus.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
