kumquat-buildroot/toolchain/toolchain-wrapper.c

542 lines
15 KiB
C
Raw Normal View History

/**
* Buildroot wrapper for toolchains. This simply executes the real toolchain
* with a number of arguments (sysroot/arch/..) hardcoded, to ensure the
* toolchain uses the correct configuration.
* The hardcoded path arguments are defined relative to the actual location
* of the binary.
*
* (C) 2011 Peter Korsgaard <jacmet@sunsite.dk>
* (C) 2011 Daniel Nyström <daniel.nystrom@timeterminal.se>
* (C) 2012 Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
* (C) 2013 Spenser Gilliland <spenser@gillilanding.com>
*
* This file is licensed under the terms of the GNU General Public License
* version 2. This program is licensed "as is" without any warranty of any
* kind, whether express or implied.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <string.h>
#include <limits.h>
#include <unistd.h>
#include <stdlib.h>
#include <errno.h>
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
#include <time.h>
#include <stdbool.h>
#ifdef BR_CCACHE
static char ccache_path[PATH_MAX];
#endif
static char path[PATH_MAX];
static char sysroot[PATH_MAX];
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
/* As would be defined by gcc:
* https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html
* sizeof() on string literals includes the terminating \0. */
static char _time_[sizeof("-D__TIME__=\"HH:MM:SS\"")];
static char _date_[sizeof("-D__DATE__=\"MMM DD YYYY\"")];
/**
* GCC errors out with certain combinations of arguments (examples are
* -mfloat-abi={hard|soft} and -m{little|big}-endian), so we have to ensure
* that we only pass the predefined one to the real compiler if the inverse
* option isn't in the argument list.
* This specifies the worst case number of extra arguments we might pass
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
* Currently, we may have:
toolchain/external: fix wrapper by not passing conflicting flags In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual compiler, this in an attempt to always generate correct and optimised code for the target. But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel. For example, on the ARM Raspberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts: drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch (and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...) To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns. The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-01-07 23:46:05 +01:00
* -mfloat-abi=
* -march=
* -mcpu=
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
* -D__TIME__=
* -D__DATE__=
* -Wno-builtin-macro-redefined
toolchain/toolchain-wrapper: add BR2_RELRO_ The RELRO/PIE flags are currently passed via CFLAGS/LDFLAGS and this patch proposes moving them to the toolchain wrapper. (1) The flags should _always_ be passed, without leaving the possibility for any package to ignore them. I.e, when BR2_RELRO_FULL=y is used in a build, all executables should be built PIE. Passing those options through the wrapper ensures they are used during the build of all packages. (2) Some options are incompatible with -fPIE. For example, when building object files for a shared libraries, -fPIC is used, and -fPIE shouldn't be used in combination with -fPIE. Similarly, -r or -static are directly incompatible as they are different link time behaviors then the intent of PIE. Passing those options through the wrapper allows to add some "smart" logic to only pass -fPIE/-pie when relevant. (3) Some toolchain, kernel and bootloader packages may want to explicitly disable PIE in a build where the rest of the userspace has intentionally enabled it. The wrapper provides an option to key on the -fno-pie/-no-pie and bypass the appending of RELRO flags. The current Kernel and U-boot source trees include this option. https://github.com/torvalds/linux/commit/8438ee76b004ef66d125ade64c91fc128047d244 https://github.com/u-boot/u-boot/commit/6ace36e19a8cfdd16ce7c02625edf36864897bf5 If using PIE with a older Kernel and/or U-boot version, a backport of these changes might be required. However this patchset also uses the __KERNEL__ and __UBOOT__ defines as a way to disable PIE. NOTE: The current implementation via CFLAGS/LDFLAGS has caused some build time failures as the conditional logic doesn't yet exist in Buildroot: https://bugs.busybox.net/show_bug.cgi?id=11206 https://bugs.busybox.net/show_bug.cgi?id=11321 Good summary of the most common build failures related to enabling pie: https://wiki.ubuntu.com/SecurityTeam/PIE [Peter: minor cleanups] Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-17 23:21:49 +02:00
* -Wl,-z,now
* -Wl,-z,relro
* -fPIE
* -pie
*/
toolchain/toolchain-wrapper: add BR2_RELRO_ The RELRO/PIE flags are currently passed via CFLAGS/LDFLAGS and this patch proposes moving them to the toolchain wrapper. (1) The flags should _always_ be passed, without leaving the possibility for any package to ignore them. I.e, when BR2_RELRO_FULL=y is used in a build, all executables should be built PIE. Passing those options through the wrapper ensures they are used during the build of all packages. (2) Some options are incompatible with -fPIE. For example, when building object files for a shared libraries, -fPIC is used, and -fPIE shouldn't be used in combination with -fPIE. Similarly, -r or -static are directly incompatible as they are different link time behaviors then the intent of PIE. Passing those options through the wrapper allows to add some "smart" logic to only pass -fPIE/-pie when relevant. (3) Some toolchain, kernel and bootloader packages may want to explicitly disable PIE in a build where the rest of the userspace has intentionally enabled it. The wrapper provides an option to key on the -fno-pie/-no-pie and bypass the appending of RELRO flags. The current Kernel and U-boot source trees include this option. https://github.com/torvalds/linux/commit/8438ee76b004ef66d125ade64c91fc128047d244 https://github.com/u-boot/u-boot/commit/6ace36e19a8cfdd16ce7c02625edf36864897bf5 If using PIE with a older Kernel and/or U-boot version, a backport of these changes might be required. However this patchset also uses the __KERNEL__ and __UBOOT__ defines as a way to disable PIE. NOTE: The current implementation via CFLAGS/LDFLAGS has caused some build time failures as the conditional logic doesn't yet exist in Buildroot: https://bugs.busybox.net/show_bug.cgi?id=11206 https://bugs.busybox.net/show_bug.cgi?id=11321 Good summary of the most common build failures related to enabling pie: https://wiki.ubuntu.com/SecurityTeam/PIE [Peter: minor cleanups] Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-17 23:21:49 +02:00
#define EXCLUSIVE_ARGS 10
static char *predef_args[] = {
#ifdef BR_CCACHE
ccache_path,
#endif
path,
"--sysroot", sysroot,
#ifdef BR_ABI
"-mabi=" BR_ABI,
#endif
#ifdef BR_NAN
"-mnan=" BR_NAN,
#endif
#ifdef BR_FPU
"-mfpu=" BR_FPU,
#endif
#ifdef BR_SOFTFLOAT
"-msoft-float",
#endif /* BR_SOFTFLOAT */
#ifdef BR_MODE
"-m" BR_MODE,
#endif
#ifdef BR_64
"-m64",
#endif
#ifdef BR_OMIT_LOCK_PREFIX
"-Wa,-momit-lock-prefix=yes",
#endif
#ifdef BR_NO_FUSED_MADD
"-mno-fused-madd",
#endif
toolchain-wrapper: use -ffp-contract=off on MIPS Xburst for gcc >= 4.6 Since gcc 4.6, GCC deprecated -mfused-madd, -ffp-contract=off should be used for the Xburst workaround. Tested with the MIPS Sourcery 2011.03 toolchain (based on gcc 4.5), the toolchain wrapper uses -mno-fused-madd, as expected: $ BR2_DEBUG_WRAPPER=2 ./output/host/bin/mips-linux-gnu-gcc -o toto toto.c Toolchain wrapper executing: '/home/thomas/toolchains/mips-2011.03/bin/mips-linux-gnu-gcc' '--sysroot' '/home/thomas/projets/buildroot/output/host/mipsel-buildroot-linux-gnu/sysroot' '-mabi=32' '-msoft-float' '-mno-fused-madd' '-EL' '-march=mips32r2' '-o' 'toto' 'toto.c' And with the MIPS Sourcery 2012.09 toolchain (based on gcc 4.7), the toolchain wrapper uses -ffp-contract=off, as expected: $ BR2_DEBUG_WRAPPER=2 ./output/host/bin/mips-linux-gnu-gcc -o toto toto.c Toolchain wrapper executing: '/home/thomas/toolchains/mips-2012.09/bin/mips-linux-gnu-gcc' '--sysroot' '/home/thomas/projets/buildroot/output/host/mipsel-buildroot-linux-gnu/sysroot' '-mabi=32' '-msoft-float' '-ffp-contract=off' '-EL' '-march=mips32r2' '-o' 'toto' 'toto.c' Fixes the ci20_defconfig build: https://gitlab.com/buildroot.org/buildroot/-/jobs/60303132 Signed-off-by: Waldemar Brodkorb <wbx@openadk.org> [Thomas: rework to continue supporting pre-gcc-4.6 toolchains, extend the commit log after doing more testing.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-31 19:54:20 +02:00
#ifdef BR_FP_CONTRACT_OFF
"-ffp-contract=off",
#endif
#ifdef BR_BINFMT_FLAT
"-Wl,-elf2flt",
#endif
#ifdef BR_MIPS_TARGET_LITTLE_ENDIAN
"-EL",
#endif
#if defined(BR_MIPS_TARGET_BIG_ENDIAN) || defined(BR_ARC_TARGET_BIG_ENDIAN)
"-EB",
#endif
#ifdef BR_ADDITIONAL_CFLAGS
BR_ADDITIONAL_CFLAGS
#endif
};
/* A {string,length} tuple, to avoid computing strlen() on constants.
* - str must be a \0-terminated string
* - len does not account for the terminating '\0'
*/
struct str_len_s {
const char *str;
size_t len;
};
/* Define a {string,length} tuple. Takes an unquoted constant string as
* parameter. sizeof() on a string literal includes the terminating \0,
* but we don't want to count it.
*/
#define STR_LEN(s) { #s, sizeof(#s)-1 }
/* List of paths considered unsafe for cross-compilation.
*
* An unsafe path is one that points to a directory with libraries or
* headers for the build machine, which are not suitable for the target.
*/
static const struct str_len_s unsafe_paths[] = {
STR_LEN(/lib),
STR_LEN(/usr/include),
STR_LEN(/usr/lib),
STR_LEN(/usr/local/include),
STR_LEN(/usr/local/lib),
{ NULL, 0 },
};
/* Unsafe options are options that specify a potentialy unsafe path,
* that will be checked by check_unsafe_path(), below.
*/
static const struct str_len_s unsafe_opts[] = {
STR_LEN(-I),
STR_LEN(-idirafter),
STR_LEN(-iquote),
STR_LEN(-isystem),
STR_LEN(-L),
{ NULL, 0 },
};
/* Check if path is unsafe for cross-compilation. Unsafe paths are those
* pointing to the standard native include or library paths.
*
* We print the arguments leading to the failure. For some options, gcc
* accepts the path to be concatenated to the argument (e.g. -I/foo/bar)
* or separated (e.g. -I /foo/bar). In the first case, we need only print
* the argument as it already contains the path (arg_has_path), while in
* the second case we need to print both (!arg_has_path).
*
* If paranoid, exit in error instead of just printing a warning.
*/
static void check_unsafe_path(const char *arg,
const char *path,
int paranoid,
int arg_has_path)
{
const struct str_len_s *p;
for (p=unsafe_paths; p->str; p++) {
if (strncmp(path, p->str, p->len))
continue;
fprintf(stderr,
"%s: %s: unsafe header/library path used in cross-compilation: '%s%s%s'\n",
program_invocation_short_name,
paranoid ? "ERROR" : "WARNING",
arg,
arg_has_path ? "" : "' '", /* close single-quote, space, open single-quote */
arg_has_path ? "" : path); /* so that arg and path are properly quoted. */
if (paranoid)
exit(1);
}
}
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
/* Returns false if SOURCE_DATE_EPOCH was not defined in the environment.
*
* Returns true if SOURCE_DATE_EPOCH is in the environment and represent
* a valid timestamp, in which case the timestamp is formatted into the
* global variables _date_ and _time_.
*
* Aborts if SOURCE_DATE_EPOCH was set in the environment but did not
* contain a valid timestamp.
*
* Valid values are defined in the spec:
* https://reproducible-builds.org/specs/source-date-epoch/
* but we further restrict them to be positive or null.
*/
bool parse_source_date_epoch_from_env(void)
{
char *epoch_env, *endptr;
time_t epoch;
struct tm epoch_tm;
if ((epoch_env = getenv("SOURCE_DATE_EPOCH")) == NULL)
return false;
errno = 0;
epoch = (time_t) strtoll(epoch_env, &endptr, 10);
/* We just need to test if it is incorrect, but we do not
* care why it is incorrect.
*/
if ((errno != 0) || !*epoch_env || *endptr || (epoch < 0)) {
fprintf(stderr, "%s: invalid SOURCE_DATE_EPOCH='%s'\n",
program_invocation_short_name,
epoch_env);
exit(1);
}
tzset(); /* For localtime_r(), below. */
if (localtime_r(&epoch, &epoch_tm) == NULL) {
fprintf(stderr, "%s: cannot parse SOURCE_DATE_EPOCH=%s\n",
program_invocation_short_name,
getenv("SOURCE_DATE_EPOCH"));
exit(1);
}
if (!strftime(_time_, sizeof(_time_), "-D__TIME__=\"%T\"", &epoch_tm)) {
fprintf(stderr, "%s: cannot set time from SOURCE_DATE_EPOCH=%s\n",
program_invocation_short_name,
getenv("SOURCE_DATE_EPOCH"));
exit(1);
}
if (!strftime(_date_, sizeof(_date_), "-D__DATE__=\"%b %e %Y\"", &epoch_tm)) {
fprintf(stderr, "%s: cannot set date from SOURCE_DATE_EPOCH=%s\n",
program_invocation_short_name,
getenv("SOURCE_DATE_EPOCH"));
exit(1);
}
return true;
}
int main(int argc, char **argv)
{
char **args, **cur, **exec_args;
char *relbasedir, *absbasedir;
char *progpath = argv[0];
char *basename;
char *env_debug;
char *paranoid_wrapper;
int paranoid;
int ret, i, count = 0, debug = 0, found_shared = 0;
/* Debug the wrapper to see arguments it was called with.
* If environment variable BR2_DEBUG_WRAPPER is:
* unset, empty, or 0: do not trace
* set to 1 : trace all arguments on a single line
* set to 2 : trace one argument per line
*/
if ((env_debug = getenv("BR2_DEBUG_WRAPPER"))) {
debug = atoi(env_debug);
}
if (debug > 0) {
fprintf(stderr, "Toolchain wrapper was called with:");
for (i = 0; i < argc; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", argv[i]);
fprintf(stderr, "\n");
}
/* Calculate the relative paths */
basename = strrchr(progpath, '/');
if (basename) {
*basename = '\0';
basename++;
relbasedir = malloc(strlen(progpath) + 7);
if (relbasedir == NULL) {
perror(__FILE__ ": malloc");
return 2;
}
Eliminate $(HOST_DIR)/usr We currently use $(HOST_DIR)/usr as the prefix for host packages. That has a few disadvantages: - There are some things installed in $(HOST_DIR)/etc and $(HOST_DIR)/sbin, which is inconsistent. - To pack a buildroot-built toolchain into a tarball for use as an external toolchain, you have to pack output/host/usr instead of the more obvious output/host. - Because of the above, the internal toolchain wrapper breaks which forces us to work around it (call the actual toolchain executable directly). This is OK for us, but when used in another build system, that's a problem. - Paths are four characters longer. To allow us to gradually eliminate $(HOST_DIR)/usr while building packages, replace it with a symlink to . The symlinks from $(HOST_DIR)/usr/$(GNU_TARGET_NAME) and $(HOST_DIR)/usr/lib that were added previously are removed again. Note that the symlink creation will break when $(HOST_DIR)/usr already exists as a directory, i.e. when rebuilding in an existing output directory. This is necessary: if we don't break it now, the following commits (which remove the usr part from various variables) _will_ break it. At the same time as creating this symlink, we have to update the external toolchain wrapper and the external toolchain symlinks to go one directory less up. Indeed, $(HOST_DIR) is one level less up than it was before. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: Romain Naour <romain.naour@smile.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 16:03:53 +02:00
sprintf(relbasedir, "%s/..", argv[0]);
absbasedir = realpath(relbasedir, NULL);
} else {
basename = progpath;
absbasedir = malloc(PATH_MAX + 1);
ret = readlink("/proc/self/exe", absbasedir, PATH_MAX);
if (ret < 0) {
perror(__FILE__ ": readlink");
return 2;
}
absbasedir[ret] = '\0';
for (i = ret; i > 0; i--) {
if (absbasedir[i] == '/') {
absbasedir[i] = '\0';
if (++count == 2)
break;
}
}
}
if (absbasedir == NULL) {
perror(__FILE__ ": realpath");
return 2;
}
/* Fill in the relative paths */
#ifdef BR_CROSS_PATH_REL
ret = snprintf(path, sizeof(path), "%s/" BR_CROSS_PATH_REL "/%s" BR_CROSS_PATH_SUFFIX, absbasedir, basename);
#elif defined(BR_CROSS_PATH_ABS)
ret = snprintf(path, sizeof(path), BR_CROSS_PATH_ABS "/%s" BR_CROSS_PATH_SUFFIX, basename);
#else
ret = snprintf(path, sizeof(path), "%s/bin/%s" BR_CROSS_PATH_SUFFIX, absbasedir, basename);
#endif
if (ret >= sizeof(path)) {
perror(__FILE__ ": overflow");
return 3;
}
#ifdef BR_CCACHE
ret = snprintf(ccache_path, sizeof(ccache_path), "%s/bin/ccache", absbasedir);
if (ret >= sizeof(ccache_path)) {
perror(__FILE__ ": overflow");
return 3;
}
#endif
ret = snprintf(sysroot, sizeof(sysroot), "%s/" BR_SYSROOT, absbasedir);
if (ret >= sizeof(sysroot)) {
perror(__FILE__ ": overflow");
return 3;
}
cur = args = malloc(sizeof(predef_args) +
(sizeof(char *) * (argc + EXCLUSIVE_ARGS)));
if (args == NULL) {
perror(__FILE__ ": malloc");
return 2;
}
/* start with predefined args */
memcpy(cur, predef_args, sizeof(predef_args));
cur += sizeof(predef_args) / sizeof(predef_args[0]);
#ifdef BR_FLOAT_ABI
/* add float abi if not overridden in args */
for (i = 1; i < argc; i++) {
if (!strncmp(argv[i], "-mfloat-abi=", strlen("-mfloat-abi=")) ||
!strcmp(argv[i], "-msoft-float") ||
!strcmp(argv[i], "-mhard-float"))
break;
}
if (i == argc)
*cur++ = "-mfloat-abi=" BR_FLOAT_ABI;
#endif
#ifdef BR_FP32_MODE
/* add fp32 mode if soft-float is not args or hard-float overrides soft-float */
int add_fp32_mode = 1;
for (i = 1; i < argc; i++) {
if (!strcmp(argv[i], "-msoft-float"))
add_fp32_mode = 0;
else if (!strcmp(argv[i], "-mhard-float"))
add_fp32_mode = 1;
}
if (add_fp32_mode == 1)
*cur++ = "-mfp" BR_FP32_MODE;
#endif
toolchain/external: fix wrapper by not passing conflicting flags In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual compiler, this in an attempt to always generate correct and optimised code for the target. But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel. For example, on the ARM Raspberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts: drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch (and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...) To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns. The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-01-07 23:46:05 +01:00
#if defined(BR_ARCH) || \
defined(BR_CPU)
/* Add our -march/cpu flags, but only if none of
* -march/mtune/mcpu are already specified on the commandline
toolchain/external: fix wrapper by not passing conflicting flags In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual compiler, this in an attempt to always generate correct and optimised code for the target. But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel. For example, on the ARM Raspberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts: drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch (and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...) To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns. The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-01-07 23:46:05 +01:00
*/
for (i = 1; i < argc; i++) {
if (!strncmp(argv[i], "-march=", strlen("-march=")) ||
!strncmp(argv[i], "-mtune=", strlen("-mtune=")) ||
toolchain/external: fix wrapper by not passing conflicting flags In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual compiler, this in an attempt to always generate correct and optimised code for the target. But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel. For example, on the ARM Raspberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts: drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch (and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...) To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns. The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-01-07 23:46:05 +01:00
!strncmp(argv[i], "-mcpu=", strlen("-mcpu=" )))
break;
}
if (i == argc) {
#ifdef BR_ARCH
*cur++ = "-march=" BR_ARCH;
#endif
#ifdef BR_CPU
*cur++ = "-mcpu=" BR_CPU;
#endif
}
#endif /* ARCH || CPU */
toolchain/external: fix wrapper by not passing conflicting flags In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual compiler, this in an attempt to always generate correct and optimised code for the target. But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel. For example, on the ARM Raspberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts: drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch (and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...) To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns. The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-01-07 23:46:05 +01:00
toolchain/wrapper: fake __DATE_ and __TIME__ for older gcc Starting with version 7, gcc automatically recognises and enforces the environment variable SOURCE_DATE_EPOCH, and fakes __DATE__ and __TIME__ accordingly, to produce reproducible builds (at least in regards to date and time). However, older gcc versions do not offer this feature. So, we use our toolchain wrapper to force-feed __DATE__ and __TIME__ as macros, which will take precedence over those that gcc may compute itself. We compute them according to the specs: https://reproducible-builds.org/specs/source-date-epoch/ https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html Since we define macros otherwise internal to gcc, we have to tell it not to warn about that. The -Wno-builtin-macro-redefined flag was introduced in gcc-4.4.0. Therefore, we make BR2_REPRODUCIBLE depend on GCC >= 4.4. gcc-7 will ignore SOURCE_DATE_EPOCH when __DATE__ and __TIME__ are user-defined. Anyway, this is of no consequence: whether __DATE__ and __TIME__ or SOURCE_DATE_EPOCH takes precedence, it would yield the exact same end result since we use the same logic to compute it. Note that we didn't copy the code for it from gcc so using the same logic doesn't imply that we're inheriting GPL-3.0. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Jérôme Pouiller <jezz@sysmic.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Peter Korsgaard <peter@korsgaard.com> [Arnout: rewrite commit message] Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-10-21 22:31:02 +02:00
if (parse_source_date_epoch_from_env()) {
*cur++ = _time_;
*cur++ = _date_;
/* This has existed since gcc-4.4.0. */
*cur++ = "-Wno-builtin-macro-redefined";
}
#ifdef BR2_PIC_PIE
toolchain/toolchain-wrapper: add BR2_RELRO_ The RELRO/PIE flags are currently passed via CFLAGS/LDFLAGS and this patch proposes moving them to the toolchain wrapper. (1) The flags should _always_ be passed, without leaving the possibility for any package to ignore them. I.e, when BR2_RELRO_FULL=y is used in a build, all executables should be built PIE. Passing those options through the wrapper ensures they are used during the build of all packages. (2) Some options are incompatible with -fPIE. For example, when building object files for a shared libraries, -fPIC is used, and -fPIE shouldn't be used in combination with -fPIE. Similarly, -r or -static are directly incompatible as they are different link time behaviors then the intent of PIE. Passing those options through the wrapper allows to add some "smart" logic to only pass -fPIE/-pie when relevant. (3) Some toolchain, kernel and bootloader packages may want to explicitly disable PIE in a build where the rest of the userspace has intentionally enabled it. The wrapper provides an option to key on the -fno-pie/-no-pie and bypass the appending of RELRO flags. The current Kernel and U-boot source trees include this option. https://github.com/torvalds/linux/commit/8438ee76b004ef66d125ade64c91fc128047d244 https://github.com/u-boot/u-boot/commit/6ace36e19a8cfdd16ce7c02625edf36864897bf5 If using PIE with a older Kernel and/or U-boot version, a backport of these changes might be required. However this patchset also uses the __KERNEL__ and __UBOOT__ defines as a way to disable PIE. NOTE: The current implementation via CFLAGS/LDFLAGS has caused some build time failures as the conditional logic doesn't yet exist in Buildroot: https://bugs.busybox.net/show_bug.cgi?id=11206 https://bugs.busybox.net/show_bug.cgi?id=11321 Good summary of the most common build failures related to enabling pie: https://wiki.ubuntu.com/SecurityTeam/PIE [Peter: minor cleanups] Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-17 23:21:49 +02:00
/* Patterned after Fedora/Gentoo hardening approaches.
* https://fedoraproject.org/wiki/Changes/Harden_All_Packages
* https://wiki.gentoo.org/wiki/Hardened/Toolchain#Position_Independent_Executables_.28PIEs.29
*
* A few checks are added to allow disabling of PIE
* 1) -fno-pie and -no-pie are used by other distros to disable PIE in
* cases where the compiler enables it by default. The logic below
* maintains that behavior.
* Ref: https://wiki.ubuntu.com/SecurityTeam/PIE
* 2) A check for -fno-PIE has been used in older Linux Kernel builds
* in a similar way to -fno-pie or -no-pie.
* 3) A check is added for Kernel and U-boot defines
* (-D__KERNEL__ and -D__UBOOT__).
*/
for (i = 1; i < argc; i++) {
/* Apply all incompatible link flag and disable checks first */
if (!strcmp(argv[i], "-r") ||
!strcmp(argv[i], "-Wl,-r") ||
!strcmp(argv[i], "-static") ||
!strcmp(argv[i], "-D__KERNEL__") ||
!strcmp(argv[i], "-D__UBOOT__") ||
!strcmp(argv[i], "-fno-pie") ||
!strcmp(argv[i], "-fno-PIE") ||
!strcmp(argv[i], "-no-pie"))
break;
/* Record that shared was present which disables -pie but don't
* break out of loop as a check needs to occur that possibly
* still allows -fPIE to be set
*/
if (!strcmp(argv[i], "-shared"))
found_shared = 1;
}
if (i == argc) {
/* Compile and link condition checking have been kept split
* between these two loops, as there maybe already are valid
* compile flags set for position independence. In that case
* the wrapper just adds the -pie for link.
*/
for (i = 1; i < argc; i++) {
if (!strcmp(argv[i], "-fpie") ||
!strcmp(argv[i], "-fPIE") ||
!strcmp(argv[i], "-fpic") ||
!strcmp(argv[i], "-fPIC"))
break;
}
/* Both args below can be set at compile/link time
* and are ignored correctly when not used
*/
if(i == argc)
*cur++ = "-fPIE";
if (!found_shared)
*cur++ = "-pie";
}
#endif
/* Are we building the Linux Kernel or U-Boot? */
for (i = 1; i < argc; i++) {
if (!strcmp(argv[i], "-D__KERNEL__") ||
!strcmp(argv[i], "-D__UBOOT__"))
break;
}
if (i == argc) {
/* https://wiki.gentoo.org/wiki/Hardened/Toolchain#Mark_Read-Only_Appropriate_Sections */
#ifdef BR2_RELRO_PARTIAL
*cur++ = "-Wl,-z,relro";
#endif
#ifdef BR2_RELRO_FULL
*cur++ = "-Wl,-z,now";
*cur++ = "-Wl,-z,relro";
#endif
}
paranoid_wrapper = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
if (paranoid_wrapper && strlen(paranoid_wrapper) > 0)
paranoid = 1;
else
paranoid = 0;
/* Check for unsafe library and header paths */
for (i = 1; i < argc; i++) {
const struct str_len_s *opt;
for (opt=unsafe_opts; opt->str; opt++ ) {
/* Skip any non-unsafe option. */
if (strncmp(argv[i], opt->str, opt->len))
continue;
/* Handle both cases:
* - path is a separate argument,
* - path is concatenated with option.
*/
if (argv[i][opt->len] == '\0') {
i++;
if (i == argc)
break;
check_unsafe_path(argv[i-1], argv[i], paranoid, 0);
} else
check_unsafe_path(argv[i], argv[i] + opt->len, paranoid, 1);
}
}
/* append forward args */
memcpy(cur, &argv[1], sizeof(char *) * (argc - 1));
cur += argc - 1;
/* finish with NULL termination */
*cur = NULL;
exec_args = args;
#ifdef BR_CCACHE
if (getenv("BR_NO_CCACHE"))
/* Skip the ccache call */
exec_args++;
#endif
/* Debug the wrapper to see final arguments passed to the real compiler. */
if (debug > 0) {
fprintf(stderr, "Toolchain wrapper executing:");
ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Our current ccache disables hashing of the compiler executable itself, because using the default 'mtime' doesn't work in buildroot: we always rebuild the compiler, so the mtime is always different, so the cache always misses. However, in the current situation, if a user changes the compiler configuration (which would result in the compiler generating different object files than before) and does 'make clean all', ccache may in fact reuse object files from the previous run. This rarely gives problems, because (1) the cache expires quite quickly (it's only 1GB by default), (2) radically changing compiler options will cause cache misses because different header files are used, (3) many compiler changes (e.g. changing -mtune) have little practical effect because the resulting code is usually still compatible, (4) we currently don't use CCACHE_BASEDIR, and almost all object files will contain an absolute path (e.g. in debug info), so when building in a different directory, most of it will miss, (5) we do mostly build test, and many of the potential problems only appear at runtime. Still, when ccache _does_ use the wrong cached object files, the effects are really weird and hard to debug. Also, we want reproducible builds and obviously the above makes builds non-reproducible. So we have a FAQ entry that warns against using ccache and tells the user to clear the cache in case of problems. Now that ccache is called from the toolchain wrapper, it is in fact possible to at least use the 'mtime' compiler hash for the external toolchain and for the host-gcc. Indeed, in this case, the compiler executable comes from a tarball so the mtime will be a good reference for its state. Therefore, the patch (sed script) that changes the default from 'mtime' to 'none' is removed. For the internal toolchain, we can do better by providing a hash of the relevant toolchain options. We are only interested in things that affect the compiler itself, because ccache also processes the header files and it doesn't look at libraries because it doesn't cache the link step, just compilation. Everything that affects the compiler itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of course, also the compiler source itself is relevant, so the source tarball and all the patches are included in the hash. For this purpose, a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced. The following procedure tests the ccache behaviour: Use this defconfig: BR2_arm=y BR2_CCACHE=y make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" Now make menuconfig, change variant into BR2_cortex_a9 make clean; make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" should be "Cortex-A9" After this commit, it is "Cortex-A9". Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Danomi Manchego <danomimanchego123@gmail.com> Cc: Károly Kasza <kaszak@gmail.com> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-04 17:25:00 +02:00
#ifdef BR_CCACHE_HASH
fprintf(stderr, "%sCCACHE_COMPILERCHECK='string:" BR_CCACHE_HASH "'",
(debug == 2) ? "\n " : " ");
#endif
#ifdef BR_CCACHE_BASEDIR
fprintf(stderr, "%sCCACHE_BASEDIR='" BR_CCACHE_BASEDIR "'",
(debug == 2) ? "\n " : " ");
ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Our current ccache disables hashing of the compiler executable itself, because using the default 'mtime' doesn't work in buildroot: we always rebuild the compiler, so the mtime is always different, so the cache always misses. However, in the current situation, if a user changes the compiler configuration (which would result in the compiler generating different object files than before) and does 'make clean all', ccache may in fact reuse object files from the previous run. This rarely gives problems, because (1) the cache expires quite quickly (it's only 1GB by default), (2) radically changing compiler options will cause cache misses because different header files are used, (3) many compiler changes (e.g. changing -mtune) have little practical effect because the resulting code is usually still compatible, (4) we currently don't use CCACHE_BASEDIR, and almost all object files will contain an absolute path (e.g. in debug info), so when building in a different directory, most of it will miss, (5) we do mostly build test, and many of the potential problems only appear at runtime. Still, when ccache _does_ use the wrong cached object files, the effects are really weird and hard to debug. Also, we want reproducible builds and obviously the above makes builds non-reproducible. So we have a FAQ entry that warns against using ccache and tells the user to clear the cache in case of problems. Now that ccache is called from the toolchain wrapper, it is in fact possible to at least use the 'mtime' compiler hash for the external toolchain and for the host-gcc. Indeed, in this case, the compiler executable comes from a tarball so the mtime will be a good reference for its state. Therefore, the patch (sed script) that changes the default from 'mtime' to 'none' is removed. For the internal toolchain, we can do better by providing a hash of the relevant toolchain options. We are only interested in things that affect the compiler itself, because ccache also processes the header files and it doesn't look at libraries because it doesn't cache the link step, just compilation. Everything that affects the compiler itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of course, also the compiler source itself is relevant, so the source tarball and all the patches are included in the hash. For this purpose, a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced. The following procedure tests the ccache behaviour: Use this defconfig: BR2_arm=y BR2_CCACHE=y make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" Now make menuconfig, change variant into BR2_cortex_a9 make clean; make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" should be "Cortex-A9" After this commit, it is "Cortex-A9". Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Danomi Manchego <danomimanchego123@gmail.com> Cc: Károly Kasza <kaszak@gmail.com> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-04 17:25:00 +02:00
#endif
for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", exec_args[i]);
fprintf(stderr, "\n");
}
ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Our current ccache disables hashing of the compiler executable itself, because using the default 'mtime' doesn't work in buildroot: we always rebuild the compiler, so the mtime is always different, so the cache always misses. However, in the current situation, if a user changes the compiler configuration (which would result in the compiler generating different object files than before) and does 'make clean all', ccache may in fact reuse object files from the previous run. This rarely gives problems, because (1) the cache expires quite quickly (it's only 1GB by default), (2) radically changing compiler options will cause cache misses because different header files are used, (3) many compiler changes (e.g. changing -mtune) have little practical effect because the resulting code is usually still compatible, (4) we currently don't use CCACHE_BASEDIR, and almost all object files will contain an absolute path (e.g. in debug info), so when building in a different directory, most of it will miss, (5) we do mostly build test, and many of the potential problems only appear at runtime. Still, when ccache _does_ use the wrong cached object files, the effects are really weird and hard to debug. Also, we want reproducible builds and obviously the above makes builds non-reproducible. So we have a FAQ entry that warns against using ccache and tells the user to clear the cache in case of problems. Now that ccache is called from the toolchain wrapper, it is in fact possible to at least use the 'mtime' compiler hash for the external toolchain and for the host-gcc. Indeed, in this case, the compiler executable comes from a tarball so the mtime will be a good reference for its state. Therefore, the patch (sed script) that changes the default from 'mtime' to 'none' is removed. For the internal toolchain, we can do better by providing a hash of the relevant toolchain options. We are only interested in things that affect the compiler itself, because ccache also processes the header files and it doesn't look at libraries because it doesn't cache the link step, just compilation. Everything that affects the compiler itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of course, also the compiler source itself is relevant, so the source tarball and all the patches are included in the hash. For this purpose, a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced. The following procedure tests the ccache behaviour: Use this defconfig: BR2_arm=y BR2_CCACHE=y make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" Now make menuconfig, change variant into BR2_cortex_a9 make clean; make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" should be "Cortex-A9" After this commit, it is "Cortex-A9". Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Danomi Manchego <danomimanchego123@gmail.com> Cc: Károly Kasza <kaszak@gmail.com> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-04 17:25:00 +02:00
#ifdef BR_CCACHE_HASH
/* Allow compilercheck to be overridden through the environment */
if (setenv("CCACHE_COMPILERCHECK", "string:" BR_CCACHE_HASH, 0)) {
perror(__FILE__ ": Failed to set CCACHE_COMPILERCHECK");
return 3;
}
#endif
#ifdef BR_CCACHE_BASEDIR
/* Allow compilercheck to be overridden through the environment */
if (setenv("CCACHE_BASEDIR", BR_CCACHE_BASEDIR, 0)) {
perror(__FILE__ ": Failed to set CCACHE_BASEDIR");
return 3;
}
#endif
ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Our current ccache disables hashing of the compiler executable itself, because using the default 'mtime' doesn't work in buildroot: we always rebuild the compiler, so the mtime is always different, so the cache always misses. However, in the current situation, if a user changes the compiler configuration (which would result in the compiler generating different object files than before) and does 'make clean all', ccache may in fact reuse object files from the previous run. This rarely gives problems, because (1) the cache expires quite quickly (it's only 1GB by default), (2) radically changing compiler options will cause cache misses because different header files are used, (3) many compiler changes (e.g. changing -mtune) have little practical effect because the resulting code is usually still compatible, (4) we currently don't use CCACHE_BASEDIR, and almost all object files will contain an absolute path (e.g. in debug info), so when building in a different directory, most of it will miss, (5) we do mostly build test, and many of the potential problems only appear at runtime. Still, when ccache _does_ use the wrong cached object files, the effects are really weird and hard to debug. Also, we want reproducible builds and obviously the above makes builds non-reproducible. So we have a FAQ entry that warns against using ccache and tells the user to clear the cache in case of problems. Now that ccache is called from the toolchain wrapper, it is in fact possible to at least use the 'mtime' compiler hash for the external toolchain and for the host-gcc. Indeed, in this case, the compiler executable comes from a tarball so the mtime will be a good reference for its state. Therefore, the patch (sed script) that changes the default from 'mtime' to 'none' is removed. For the internal toolchain, we can do better by providing a hash of the relevant toolchain options. We are only interested in things that affect the compiler itself, because ccache also processes the header files and it doesn't look at libraries because it doesn't cache the link step, just compilation. Everything that affects the compiler itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of course, also the compiler source itself is relevant, so the source tarball and all the patches are included in the hash. For this purpose, a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced. The following procedure tests the ccache behaviour: Use this defconfig: BR2_arm=y BR2_CCACHE=y make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" Now make menuconfig, change variant into BR2_cortex_a9 make clean; make readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os -> Tag_CPU_name: "ARM926EJ-S" should be "Cortex-A9" After this commit, it is "Cortex-A9". Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Danomi Manchego <danomimanchego123@gmail.com> Cc: Károly Kasza <kaszak@gmail.com> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-04 17:25:00 +02:00
if (execv(exec_args[0], exec_args))
perror(path);
free(args);
return 2;
}