Our current ccache disables hashing of the compiler executable itself,
because using the default 'mtime' doesn't work in buildroot: we always
rebuild the compiler, so the mtime is always different, so the cache
always misses.
However, in the current situation, if a user changes the compiler
configuration (which would result in the compiler generating different
object files than before) and does 'make clean all', ccache may in fact
reuse object files from the previous run. This rarely gives problems,
because
(1) the cache expires quite quickly (it's only 1GB by default),
(2) radically changing compiler options will cause cache misses because
different header files are used,
(3) many compiler changes (e.g. changing -mtune) have little practical
effect because the resulting code is usually still compatible,
(4) we currently don't use CCACHE_BASEDIR, and almost all object files
will contain an absolute path (e.g. in debug info), so when
building in a different directory, most of it will miss,
(5) we do mostly build test, and many of the potential problems only
appear at runtime.
Still, when ccache _does_ use the wrong cached object files, the
effects are really weird and hard to debug. Also, we want reproducible
builds and obviously the above makes builds non-reproducible. So we
have a FAQ entry that warns against using ccache and tells the user to
clear the cache in case of problems.
Now that ccache is called from the toolchain wrapper, it is in fact
possible to at least use the 'mtime' compiler hash for the external
toolchain and for the host-gcc. Indeed, in this case, the compiler
executable comes from a tarball so the mtime will be a good reference
for its state. Therefore, the patch (sed script) that changes the
default from 'mtime' to 'none' is removed.
For the internal toolchain, we can do better by providing a hash of
the relevant toolchain options. We are only interested in things that
affect the compiler itself, because ccache also processes the header
files and it doesn't look at libraries because it doesn't cache the
link step, just compilation. Everything that affects the compiler
itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of
course, also the compiler source itself is relevant, so the source
tarball and all the patches are included in the hash. For this purpose,
a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced.
The following procedure tests the ccache behaviour:
Use this defconfig:
BR2_arm=y
BR2_CCACHE=y
make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
Now make menuconfig, change variant into BR2_cortex_a9
make clean; make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
should be "Cortex-A9"
After this commit, it is "Cortex-A9".
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: Károly Kasza <kaszak@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Bump ccache to minor version 3.2.2.
Also changed download URL to https.
Signed-off-by: Karoly Kasza <kaszak@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Bump ccache from v.3.1.x to v.3.2.x.
All functionality remained the same.
- Configure now accepts a parameter to use ccache's internal zlib.
- conf.c was introduced and some settings were moved from ccache.c,
with minor changes.
- The PATCH_CONFIGURATION hook was moved from POST_CONFIGURE to
POST_PATCH, as patching files should be done there, before calling
the configure script.
Signed-off-by: Karoly Kasza <kaszak@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit "packages: indentation cleanup" broke the build of master.
This commit reverse the patch on just ccache package and fix that issue.
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit doesn't touch infra packages.
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To be consistent with the recent change of FOO_MAKE_OPT into FOO_MAKE_OPTS,
make the same change for FOO_CONF_OPT.
Sed command used:
find * -type f | xargs sed -i 's#_CONF_OPT\>#&S#g'
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
For example, if your project is known to require more space than the
default max cache size, then you might want to increase the cache size
to a suitable amount using the -M (--max-size) option.
The string you specify here is passed verbatim to ccache. Refer to
ccache documentation for more details.
These initial settings are applied after ccache has been compiled.
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 433290761f changed the hard-coded
ccache directory location to use BR_CACHE_DIR (then BUILDROOT_CACHE_DIR),
which is exported by Makefile based on the BR2_CCACHE_DIR config option.
This allowed the cache location to be changed on-the-fly by setting a
"make" command line variable, but left the default location of ccache's
normal default at "$HOME/.ccache". Since this location does not match the
default for BR2_CCACHE_DIR, it is basically almost never correct, so
direct invocation of ccache outside of the buildroot Makefile, such as for
increasing the cache size, becomes cumbersome.
This patch changes the last-ditch cache location from "$HOME/.ccache" to
the BR_CCACHE_DIR value defined when host-ccache is configured. Note that
the ability to later override the cache location by using a BR_CACHE_DIR
command line variable is left intact.
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This ensures that the cache directory (and all of its ancestor
directories) exist. This is a nice thing to do because, if the parent
of the cache directory doesn't exist, then ccache will complain that
it cannot create the cache directory, causing the build to fail.
[Peter: drop BR2_CCACHE conditional, use POST_INSTALL hook]
Signed-off-by: Dan Moulding <dan.moulding@rackwareinc.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The ccache target package has been deprecated since 2013.02 and thus can be
removed in 2014.02. This does not change anything about host ccache (used
for speeding up builds).
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To make the naming consistent (qstripped variant of a config option
should be named BR_XXX).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
usage:
# set cache limit size
make CCACHE_OPTIONS="--max-size=5G" ccache-options
# zero statistics counters
make CCACHE_OPTIONS="--zero-stats" ccache-options
[Peter: drop the redundant ifeq]
Signed-off-by: Tzu-Jung Lee <tjlee@ambarella.com>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
[Peter: leave change xz tarball format to not end up with circular deps]
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This patch moves the host-ccache build target from BASE_TARGETS in Makefile
to an actual host prerequisite in support/dependencies. This causes
host-ccache to be built as part of the dependencies, before any real package
is built.
Since the dependencies are built without ccache anyway, there is no need to
set HOST_CCACHE_CONF_ENV anymore.
Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The existing ccache infrastructure sets the cache directory hardcoded in the
ccache binary. As this directory was set to ~/.buildroot-ccache, the cache
is not necessarily local (e.g. in corporate environments the home directories
may be mounted over NFS.)
Previous versions of buildroot did allow to set the cache directory, but this
was also hardcoded (so you had to rebuild ccache to change it), plus that
support was removed.
See http://lists.busybox.net/pipermail/buildroot/2011-July/044511.html for
a discussion on this.
This patch modifies ccache to respect a new shell variable (exported from
the Makefile, based on a configuration option) instead of CCACHE_DIR.
The name CCACHE_DIR itself is already used by autotargets for the ccache
package.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This allows ccache to re-use its cache contents even if the compiler
binary mtime has changed. It is the simplest approach to solve this
problem, and it works for the internal, external and crosstool-ng
toolchain backends.
Of course, it leaves the user responsible for invalidating the cache
when necessary, but there doesn't seem to be a real good solution that
allows both to: 1/ keep the cache contents accross builds and re-use
it and 2/ invalidate the cache automatically when the compiler chances
in an incompatible way.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
The problem is that without this, ccache would link against the zlib
of the build system, but we might build and install a different
version of zlib in $(O)/host afterwards, which ccache will pick
up. This might break if there is a version mismatch. A solution would
be to add host-zlib has a dependency of ccache, but it would require
tuning the zlib .mk file to use HOSTCC_NOCCACHE as the
compiler. Instead, we take the easy path: tell ccache to use its
internal copy of zlib, so that ccache has zero dependency besides the
C library.
Fixes bug #4808.
Thanks to Raúl Sánchez Siles <rsanchezs@infoglobal.es> for reporting
the bug and testing the proposed solution.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Thanks to the pkgparentdir and pkgname functions, we can rewrite the
AUTOTARGETS macro in a way that avoids the need for each package to
repeat its name and the directory in which it is present.
[Peter: pkgdir->pkgparentdir]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
* ccache is now a normal package (both for the host and the target).
* ccache option is now part of the "Build options" menu. It will
automatically build ccache for the host before building anything,
and will use it to cache builds for both host compilations and
target compilations.
* bump ccache to 3.1.3
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
