kumquat-buildroot/Makefile

1216 lines
44 KiB
Makefile
Raw Normal View History

# Makefile for buildroot
2001-12-22 01:56:11 +01:00
#
2005-02-07 23:19:26 +01:00
# Copyright (C) 1999-2005 by Erik Andersen <andersen@codepoet.org>
# Copyright (C) 2006-2014 by the Buildroot developers <buildroot@uclibc.org>
# Copyright (C) 2014-2020 by the Buildroot developers <buildroot@buildroot.org>
2001-12-22 01:56:11 +01:00
#
2002-04-26 13:45:55 +02:00
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
2001-12-22 01:56:11 +01:00
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
2002-04-26 13:45:55 +02:00
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
#--------------------------------------------------------------
# Just run 'make menuconfig', configure stuff, then run 'make'.
# You shouldn't need to mess with anything beyond this point...
#--------------------------------------------------------------
# Delete default rules. We don't use them. This saves a bit of time.
.SUFFIXES:
# we want bash as shell
SHELL := $(shell if [ -x "$$BASH" ]; then echo $$BASH; \
else if [ -x /bin/bash ]; then echo /bin/bash; \
else echo sh; fi; fi)
# Set O variable if not already done on the command line;
# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
# build by preventing it from being forwarded to sub-make calls.
ifneq ("$(origin O)", "command line")
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
O := $(CURDIR)/output
endif
# Check if the current Buildroot execution meets all the pre-requisites.
# If they are not met, Buildroot will actually do its job in a sub-make meeting
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
# its pre-requisites, which are:
# 1- Permissive enough umask:
# Wrong or too restrictive umask will prevent Buildroot and packages from
# creating files and directories.
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
# 2- Absolute canonical CWD (i.e. $(CURDIR)):
# Otherwise, some packages will use CWD as-is, others will compute its
# absolute canonical path. This makes harder tracking and fixing host
# machine path leaks.
# 3- Absolute canonical output location (i.e. $(O)):
# For the same reason as the one for CWD.
# Remove the trailing '/.' from $(O) as it can be added by the makefile wrapper
# installed in the $(O) directory.
# Also remove the trailing '/' the user can set when on the command line.
override O := $(patsubst %/,%,$(patsubst %.,%,$(O)))
# Make sure $(O) actually exists before calling realpath on it; this is to
# avoid empty CANONICAL_O in case on non-existing entry.
CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
# gcc fails to build when the srcdir contains a '@'
ifneq ($(findstring @,$(CANONICAL_O)),)
$(error The build directory can not contain a '@')
endif
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
CANONICAL_CURDIR = $(realpath $(CURDIR))
REQ_UMASK = 0022
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
# Make sure O= is passed (with its absolute canonical path) everywhere the
# toplevel makefile is called back.
EXTRAMAKEARGS := O=$(CANONICAL_O)
# Check Buildroot execution pre-requisites here.
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
.PHONY: _all $(MAKECMDGOALS)
$(MAKECMDGOALS): _all
@:
_all:
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
@umask $(REQ_UMASK) && \
$(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
$(MAKECMDGOALS) $(EXTRAMAKEARGS)
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
else # umask / $(CURDIR) / $(O)
# This is our default rule, so must come first
all:
.PHONY: all
# Set and export the version string
export BR2_VERSION := 2020.05-rc3
# Actual time the release is cut (for reproducible builds)
BR2_VERSION_EPOCH = 1590784000
# Save running make version since it's clobbered by the make package
RUNNING_MAKE_VERSION := $(MAKE_VERSION)
# Check for minimal make version (note: this check will break at make 10.x)
MIN_MAKE_VERSION = 3.81
ifneq ($(firstword $(sort $(RUNNING_MAKE_VERSION) $(MIN_MAKE_VERSION))),$(MIN_MAKE_VERSION))
$(error You have make '$(RUNNING_MAKE_VERSION)' installed. GNU make >= $(MIN_MAKE_VERSION) is required)
endif
# absolute path
TOPDIR := $(CURDIR)
CONFIG_CONFIG_IN = Config.in
CONFIG = support/kconfig
DATE := $(shell date +%Y%m%d)
2003-09-26 23:18:46 +02:00
# Compute the full local version string so packages can use it as-is
# Need to export it, so it can be got from environment in children (eg. mconf)
export BR2_VERSION_FULL := $(BR2_VERSION)$(shell $(TOPDIR)/support/scripts/setlocalversion)
# List of targets and target patterns for which .config doesn't need to be read in
noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconfig \
defconfig %_defconfig allyesconfig allnoconfig alldefconfig syncconfig release \
randpackageconfig allyespackageconfig allnopackageconfig \
print-version olddefconfig distclean manual manual-% check-package
# Some global targets do not trigger a build, but are used to collect
# metadata, or do various checks. When such targets are triggered,
# some packages should not do their configuration sanity
# checks. Provide them a BR_BUILDING variable set to 'y' when we're
# actually building and they should do their sanity checks.
#
# We're building in two situations: when MAKECMDGOALS is empty
# (default target is to build), or when MAKECMDGOALS contains
# something else than one of the nobuild_targets.
nobuild_targets := source %-source \
legal-info %-legal-info external-deps _external-deps \
clean distclean help show-targets graph-depends \
%-graph-depends %-show-depends %-show-version \
graph-build graph-size list-defconfigs \
savedefconfig update-defconfig printvars
ifeq ($(MAKECMDGOALS),)
BR_BUILDING = y
else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
BR_BUILDING = y
endif
# We call make recursively to build packages. The command-line overrides that
# are passed to Buildroot don't apply to those package build systems. In
# particular, we don't want to pass down the O=<dir> option for out-of-tree
# builds, because the value specified on the command line will not be correct
# for packages.
MAKEOVERRIDES :=
# Include some helper macros and variables
include support/misc/utils.mk
# Set variables related to in-tree or out-of-tree build.
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
# Here, both $(O) and $(CURDIR) are absolute canonical paths.
ifeq ($(O),$(CURDIR)/output)
CONFIG_DIR := $(CURDIR)
NEED_WRAPPER =
else
CONFIG_DIR := $(O)
NEED_WRAPPER = y
endif
# bash prints the name of the directory on 'cd <dir>' if CDPATH is
# set, so unset it here to not cause problems. Notice that the export
# line doesn't affect the environment of $(shell ..) calls.
export CDPATH :=
BASE_DIR := $(CANONICAL_O)
$(if $(BASE_DIR),, $(error output directory "$(O)" does not exist))
core: introduce the BR2_EXTERNAL variable This commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree. This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in. Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs). In details, this commit: * Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements. * Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include. * Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Ryan Barnett <rjbarnet@rockwellcollins.com> Tested-by: "Samuel Martin" <s.martin49@gmail.com> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2013-12-05 20:11:10 +01:00
# Handling of BR2_EXTERNAL.
#
# The value of BR2_EXTERNAL is stored in .br-external in the output directory.
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
# The location of the external.mk makefile fragments is computed in that file.
core: offload handling of BR2_EXTERNAL into the script Currently, we treat the case where we have no br2-external tree (BR2_EXTERNAL is empty) differently from the case where we do have one (BR2_EXTERNAL is not empty). There is now no reason to treat those two cases differently: - the kconfig snippet is always generated appropriately (i.e. it would include the br2-external tree if set, or include nothing otherwise); - we no longer have a dummy br-external tree either. Also, the Makefile code to handle BR2_EXTERNAL is currently quite readable if at least a little bit tricky. However, when we're going to add support for using multiple br2-external trees simultaneously, this code would need to get much, much more complex. To keep the Makefile (rather) simple, offload all of the handling of BR2_EXTERNAL to the recently added br2-external helper script. However, because of Makefiles idiosyncracies, we can't use a rule to generate that Makefile fragment. Instead, we use $(shell ...) to call the helper script, and include the fragment twice: once before the $(shell ...) so we can grab a previously defined BR2_EXTERNAL value, a second time to use the one passed on the command line, if any. Furthermore, we can't error out (e.g. on non-existent br2-external tree) directly from the fragment or we'd get that error on subsequent calls, with no chance to override it even from command line. Instead, we use a variable in which we store the error, set it to empty before the second inclusion, so that only the one newly generated, if any, is taken into account. Since we know the script will always be called from Makefile context first, we know validation will occur in Makefile context first. So we can assume that, if there is an error, it will be detected in Makefile context. Consequently, if the script is called to generate the kconfig fragment, validation has already occured, and there should be no error. So we change the error function to generate Makefile code, so that errors are caught as explained above. Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget' about the previous value of the BR2_EXTERNAL_MK variable, especially in the case where BR2_EXTERNAL is now set to empty, so that we do not try to include it later. That's why we first generate empty version of BR2_EXTERNAL_MK, and then assign it the new value, if any. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:15 +02:00
# On subsequent invocations of make, this file is read in. BR2_EXTERNAL can
# still be overridden on the command line, therefore the file is re-created
# every time make is run.
core: introduce the BR2_EXTERNAL variable This commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree. This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in. Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs). In details, this commit: * Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements. * Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include. * Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Ryan Barnett <rjbarnet@rockwellcollins.com> Tested-by: "Samuel Martin" <s.martin49@gmail.com> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2013-12-05 20:11:10 +01:00
BR2_EXTERNAL_FILE = $(BASE_DIR)/.br2-external.mk
core: introduce the BR2_EXTERNAL variable This commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree. This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in. Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs). In details, this commit: * Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements. * Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include. * Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Ryan Barnett <rjbarnet@rockwellcollins.com> Tested-by: "Samuel Martin" <s.martin49@gmail.com> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2013-12-05 20:11:10 +01:00
-include $(BR2_EXTERNAL_FILE)
$(shell support/scripts/br2-external -d '$(BASE_DIR)' $(BR2_EXTERNAL))
core: offload handling of BR2_EXTERNAL into the script Currently, we treat the case where we have no br2-external tree (BR2_EXTERNAL is empty) differently from the case where we do have one (BR2_EXTERNAL is not empty). There is now no reason to treat those two cases differently: - the kconfig snippet is always generated appropriately (i.e. it would include the br2-external tree if set, or include nothing otherwise); - we no longer have a dummy br-external tree either. Also, the Makefile code to handle BR2_EXTERNAL is currently quite readable if at least a little bit tricky. However, when we're going to add support for using multiple br2-external trees simultaneously, this code would need to get much, much more complex. To keep the Makefile (rather) simple, offload all of the handling of BR2_EXTERNAL to the recently added br2-external helper script. However, because of Makefiles idiosyncracies, we can't use a rule to generate that Makefile fragment. Instead, we use $(shell ...) to call the helper script, and include the fragment twice: once before the $(shell ...) so we can grab a previously defined BR2_EXTERNAL value, a second time to use the one passed on the command line, if any. Furthermore, we can't error out (e.g. on non-existent br2-external tree) directly from the fragment or we'd get that error on subsequent calls, with no chance to override it even from command line. Instead, we use a variable in which we store the error, set it to empty before the second inclusion, so that only the one newly generated, if any, is taken into account. Since we know the script will always be called from Makefile context first, we know validation will occur in Makefile context first. So we can assume that, if there is an error, it will be detected in Makefile context. Consequently, if the script is called to generate the kconfig fragment, validation has already occured, and there should be no error. So we change the error function to generate Makefile code, so that errors are caught as explained above. Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget' about the previous value of the BR2_EXTERNAL_MK variable, especially in the case where BR2_EXTERNAL is now set to empty, so that we do not try to include it later. That's why we first generate empty version of BR2_EXTERNAL_MK, and then assign it the new value, if any. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:15 +02:00
BR2_EXTERNAL_ERROR =
include $(BR2_EXTERNAL_FILE)
ifneq ($(BR2_EXTERNAL_ERROR),)
$(error $(BR2_EXTERNAL_ERROR))
core: introduce the BR2_EXTERNAL variable This commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree. This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in. Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs). In details, this commit: * Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements. * Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include. * Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Ryan Barnett <rjbarnet@rockwellcollins.com> Tested-by: "Samuel Martin" <s.martin49@gmail.com> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2013-12-05 20:11:10 +01:00
endif
# Workaround bug in make-4.3: https://savannah.gnu.org/bugs/?57676
$(BASE_DIR)/.br2-external.mk:;
# To make sure that the environment variable overrides the .config option,
# set this before including .config.
ifneq ($(BR2_DL_DIR),)
DL_DIR := $(BR2_DL_DIR)
endif
ifneq ($(BR2_CCACHE_DIR),)
BR_CACHE_DIR := $(BR2_CCACHE_DIR)
endif
# Need that early, before we scan packages
# Avoids doing the $(or...) everytime
BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
core: introduce the BR2_EXTERNAL variable This commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree. This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in. Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs). In details, this commit: * Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements. * Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include. * Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Acked-by: Ryan Barnett <rjbarnet@rockwellcollins.com> Tested-by: "Samuel Martin" <s.martin49@gmail.com> Acked-by: "Samuel Martin" <s.martin49@gmail.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2013-12-05 20:11:10 +01:00
BUILD_DIR := $(BASE_DIR)/build
BINARIES_DIR := $(BASE_DIR)/images
BASE_TARGET_DIR := $(BASE_DIR)/target
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
PER_PACKAGE_DIR := $(BASE_DIR)/per-package
# initial definition so that 'make clean' works for most users, even without
# .config. HOST_DIR will be overwritten later when .config is included.
HOST_DIR := $(BASE_DIR)/host
GRAPHS_DIR := $(BASE_DIR)/graphs
LEGAL_INFO_DIR = $(BASE_DIR)/legal-info
REDIST_SOURCES_DIR_TARGET = $(LEGAL_INFO_DIR)/sources
REDIST_SOURCES_DIR_HOST = $(LEGAL_INFO_DIR)/host-sources
LICENSE_FILES_DIR_TARGET = $(LEGAL_INFO_DIR)/licenses
LICENSE_FILES_DIR_HOST = $(LEGAL_INFO_DIR)/host-licenses
LEGAL_MANIFEST_CSV_TARGET = $(LEGAL_INFO_DIR)/manifest.csv
LEGAL_MANIFEST_CSV_HOST = $(LEGAL_INFO_DIR)/host-manifest.csv
LEGAL_WARNINGS = $(LEGAL_INFO_DIR)/.warnings
LEGAL_REPORT = $(LEGAL_INFO_DIR)/README
BR2_CONFIG = $(CONFIG_DIR)/.config
# Pull in the user's configuration file
ifeq ($(filter $(noconfig_targets),$(MAKECMDGOALS)),)
-include $(BR2_CONFIG)
endif
ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),)
# Disable top-level parallel build if per-package directories is not
# used. Indeed, per-package directories is necessary to guarantee
# determinism and reproducibility with top-level parallel build.
.NOTPARALLEL:
endif
# timezone and locale may affect build output
ifeq ($(BR2_REPRODUCIBLE),y)
export TZ = UTC
export LANG = C
export LC_ALL = C
endif
# To put more focus on warnings, be less verbose as default
# Use 'make V=1' to see the full commands
ifeq ("$(origin V)", "command line")
KBUILD_VERBOSE = $(V)
endif
ifndef KBUILD_VERBOSE
KBUILD_VERBOSE = 0
endif
ifeq ($(KBUILD_VERBOSE),1)
Q =
ifndef VERBOSE
VERBOSE = 1
endif
export VERBOSE
else
Q = @
endif
# kconfig uses CONFIG_SHELL
CONFIG_SHELL := $(SHELL)
export SHELL CONFIG_SHELL Q KBUILD_VERBOSE
ifndef HOSTAR
HOSTAR := ar
endif
ifndef HOSTAS
HOSTAS := as
endif
ifndef HOSTCC
HOSTCC := gcc
HOSTCC := $(shell which $(HOSTCC) || type -p $(HOSTCC) || echo gcc)
endif
HOSTCC_NOCCACHE := $(HOSTCC)
ifndef HOSTCXX
HOSTCXX := g++
HOSTCXX := $(shell which $(HOSTCXX) || type -p $(HOSTCXX) || echo g++)
endif
HOSTCXX_NOCCACHE := $(HOSTCXX)
2007-09-28 21:46:58 +02:00
ifndef HOSTCPP
HOSTCPP := cpp
2007-09-28 21:46:58 +02:00
endif
ifndef HOSTLD
HOSTLD := ld
endif
ifndef HOSTLN
HOSTLN := ln
endif
2007-09-28 21:46:58 +02:00
ifndef HOSTNM
HOSTNM := nm
2007-09-28 21:46:58 +02:00
endif
ifndef HOSTOBJCOPY
HOSTOBJCOPY := objcopy
endif
ifndef HOSTRANLIB
HOSTRANLIB := ranlib
endif
HOSTAR := $(shell which $(HOSTAR) || type -p $(HOSTAR) || echo ar)
HOSTAS := $(shell which $(HOSTAS) || type -p $(HOSTAS) || echo as)
HOSTCPP := $(shell which $(HOSTCPP) || type -p $(HOSTCPP) || echo cpp)
HOSTLD := $(shell which $(HOSTLD) || type -p $(HOSTLD) || echo ld)
HOSTLN := $(shell which $(HOSTLN) || type -p $(HOSTLN) || echo ln)
HOSTNM := $(shell which $(HOSTNM) || type -p $(HOSTNM) || echo nm)
HOSTOBJCOPY := $(shell which $(HOSTOBJCOPY) || type -p $(HOSTOBJCOPY) || echo objcopy)
HOSTRANLIB := $(shell which $(HOSTRANLIB) || type -p $(HOSTRANLIB) || echo ranlib)
SED := $(shell which sed || type -p sed) -i -e
export HOSTAR HOSTAS HOSTCC HOSTCXX HOSTLD
export HOSTCC_NOCCACHE HOSTCXX_NOCCACHE
core: fix setting of HOSTARCH Currently, we set HOSTARCH to the output of `uname -m`. This gives us the architecture as seen by the running kernel. For example, we would end up with 'x86_64' for a 64-bit kernel running on an x86_64 processor. We use that value to determine whether we can run some binary tools, like our pre-configured external toolchains. However, one may be running a userland in a different bitness than that of the running kernel. For example, one may run in a 32-bit chroot, even though the kernel is running in 64-bit. Up until recently, this was not an issue because the pre-configured external toolchains were all requiring an i386 (x86 in Buildroot parlance). But since we introduced the latest Linaro toolchains, we now have toolchains that require a 64-bit userland. So, when running on a 64-bit kernel, we believe those toolchains are available, even when the user is running a 32-bit userland. This causes build failures for our autobuilders, like so: http://autobuild.buildroot.org/results/9cd/9cdf10ec5b31144b2e03ea09cf128702339895b3/ with the following symptoms: >>> toolchain-external undefined Configuring Cannot execute cross-compiler '/home/test/autobuild/instance-3/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-gcc' So, instead of relying on the output of `uname -r`, look for the host gcc and extract the target it was configured to generate code for. Fixes: http://autobuild.buildroot.org/results/9cd/9cdf10ec5b31144b2e03ea09cf128702339895b3/ (aarch64) http://autobuild.buildroot.org/results/888/8889aa7d9fb48370e4760a6edbc6d3ae945f02f2/ (arm) and many more... Besides fixing those issues, it will also allow us to add the 64-bit variants of toolchains when they exist, like the upcoming Codescape MTI and IMG toolchains for MIPS from Imagination Technologies. [Peter: use HOSTCC_NOCCACHE] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-09 20:00:21 +01:00
# Determine the userland we are running on.
#
# Note that, despite its name, we are not interested in the actual
# architecture name. This is mostly used to determine whether some
# of the binary tools (e.g. pre-built external toolchains) can run
# on the current host. So we need to know if the userland we're
# running on can actually run those toolchains.
#
# For example, a 64-bit prebuilt toolchain will not run on a 64-bit
# kernel if the userland is 32-bit (e.g. in a chroot for example).
#
# So, we extract the first part of the tuple the host gcc was
# configured to generate code for; we assume this is our userland.
#
export HOSTARCH := $(shell LC_ALL=C $(HOSTCC_NOCCACHE) -v 2>&1 | \
core: fix setting of HOSTARCH Currently, we set HOSTARCH to the output of `uname -m`. This gives us the architecture as seen by the running kernel. For example, we would end up with 'x86_64' for a 64-bit kernel running on an x86_64 processor. We use that value to determine whether we can run some binary tools, like our pre-configured external toolchains. However, one may be running a userland in a different bitness than that of the running kernel. For example, one may run in a 32-bit chroot, even though the kernel is running in 64-bit. Up until recently, this was not an issue because the pre-configured external toolchains were all requiring an i386 (x86 in Buildroot parlance). But since we introduced the latest Linaro toolchains, we now have toolchains that require a 64-bit userland. So, when running on a 64-bit kernel, we believe those toolchains are available, even when the user is running a 32-bit userland. This causes build failures for our autobuilders, like so: http://autobuild.buildroot.org/results/9cd/9cdf10ec5b31144b2e03ea09cf128702339895b3/ with the following symptoms: >>> toolchain-external undefined Configuring Cannot execute cross-compiler '/home/test/autobuild/instance-3/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-gcc' So, instead of relying on the output of `uname -r`, look for the host gcc and extract the target it was configured to generate code for. Fixes: http://autobuild.buildroot.org/results/9cd/9cdf10ec5b31144b2e03ea09cf128702339895b3/ (aarch64) http://autobuild.buildroot.org/results/888/8889aa7d9fb48370e4760a6edbc6d3ae945f02f2/ (arm) and many more... Besides fixing those issues, it will also allow us to add the 64-bit variants of toolchains when they exist, like the upcoming Codescape MTI and IMG toolchains for MIPS from Imagination Technologies. [Peter: use HOSTCC_NOCCACHE] Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-09 20:00:21 +01:00
sed -e '/^Target: \([^-]*\).*/!d' \
-e 's//\1/' \
-e 's/i.86/x86/' \
-e 's/sun4u/sparc64/' \
-e 's/arm.*/arm/' \
-e 's/sa110/arm/' \
-e 's/ppc64/powerpc64/' \
-e 's/ppc/powerpc/' \
-e 's/macppc/powerpc/' \
-e 's/sh.*/sh/' )
# When adding a new host gcc version in Config.in,
# update the HOSTCC_MAX_VERSION variable:
HOSTCC_MAX_VERSION := 9
HOSTCC_VERSION := $(shell V=$$($(HOSTCC_NOCCACHE) --version | \
sed -n -r 's/^.* ([0-9]*)\.([0-9]*)\.([0-9]*)[ ]*.*/\1 \2/p'); \
[ "$${V%% *}" -le $(HOSTCC_MAX_VERSION) ] || V=$(HOSTCC_MAX_VERSION); \
printf "%s" "$${V}")
# For gcc >= 5.x, we only need the major version.
ifneq ($(firstword $(HOSTCC_VERSION)),4)
HOSTCC_VERSION := $(firstword $(HOSTCC_VERSION))
endif
ifeq ($(BR2_NEEDS_HOST_UTF8_LOCALE),y)
# First, we try to use the user's configured locale (as that's the
# language they'd expect messages to be displayed), then we favour
# a non language-specific locale like C.UTF-8 if one is available,
# so we sort with the C locale to get it at the top.
# This is guaranteed to not be empty, because of the check in
# support/dependencies/dependencies.sh
HOST_UTF8_LOCALE := $(shell \
( echo $${LC_ALL:-$${LC_MESSAGES:-$${LANG}}}; \
locale -a 2>/dev/null | LC_ALL=C sort \
) \
| grep -i -E 'utf-?8$$' \
| head -n 1)
HOST_UTF8_LOCALE_ENV := LC_ALL=$(HOST_UTF8_LOCALE)
endif
# Make sure pkg-config doesn't look outside the buildroot tree
HOST_PKG_CONFIG_PATH := $(PKG_CONFIG_PATH)
unexport PKG_CONFIG_PATH
unexport PKG_CONFIG_SYSROOT_DIR
unexport PKG_CONFIG_LIBDIR
# Having DESTDIR set in the environment confuses the installation
# steps of some packages.
unexport DESTDIR
# Causes breakage with packages that needs host-ruby
unexport RUBYOPT
include package/pkg-utils.mk
include package/doc-asciidoc.mk
ifeq ($(BR2_HAVE_DOT_CONFIG),y)
2003-01-18 22:52:46 +01:00
################################################################################
#
# Hide troublesome environment variables from sub processes
#
################################################################################
unexport CROSS_COMPILE
unexport ARCH
unexport CC
unexport LD
unexport AR
unexport CXX
unexport CPP
unexport RANLIB
unexport CFLAGS
unexport CXXFLAGS
unexport GREP_OPTIONS
unexport TAR_OPTIONS
unexport CONFIG_SITE
unexport QMAKESPEC
unexport TERMINFO to correct ncurses behavior The ncurses build can become polluted by the user's TERMINFO environment variable, causing the user's ~/.terminfo to be modified and preventing the install from succeeding: /bin/sh ./run_tic.sh ** Building terminfo database, please wait... Running tic to install /home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr/share/emacs/24.0.97/etc/ ... You may see messages regarding extended capabilities, e.g., AX. These are extended terminal capabilities which are compiled using tic -x If you have ncurses 4.2 applications, you should read the INSTALL document, and install the terminfo without the -x option. 1562 entries written to /home/nathanl/.terminfo ** built new /home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr/share/emacs/24.0.97/etc/ installing std installing stdcrt installing vt100 installing vt300 make[2]: Leaving directory `/home/nathanl/devel/buildroot.git/output/build/ncurses-5.7/misc' make[1]: Leaving directory `/home/nathanl/devel/buildroot.git/output/build/ncurses-5.7' for i in $(find /home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr/lib* -name "*.la"); do cp -f $i $i~; /usr/bin/sed -i -e "s:\(['= ]\)/usr:\\1/home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr:g" $i; done >>> ncurses 5.7 Installing to target mkdir -p /home/nathanl/devel/buildroot.git/output/target/usr/lib cp -dpf /home/nathanl/devel/buildroot.git/output/build/ncurses-5.7/lib/libncurses.so* /home/nathanl/devel/buildroot.git/output/target/usr/lib/ ln -snf /usr/share/terminfo /home/nathanl/devel/buildroot.git/output/target/usr/lib/terminfo mkdir -p /home/nathanl/devel/buildroot.git/output/target/usr/share/terminfo/x cp -dpf /home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr/share/terminfo/x/xterm /home/nathanl/devel/buildroot.git/output/target/usr/share/terminfo/x cp: cannot stat `/home/nathanl/devel/buildroot.git/output/host/usr/i686-unknown-linux-gnu/sysroot/usr/share/terminfo/x/xterm': No such file or directory make: *** [/home/nathanl/devel/buildroot.git/output/build/ncurses-5.7/.stamp_target_installed] Error 1 So unexport TERMINFO in the top-level Makefile. Signed-off-by: Nathan Lynch <ntl@pobox.com> Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar> Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
2012-07-10 09:37:22 +02:00
unexport TERMINFO
unexport MACHINE
unexport O
unexport GCC_COLORS
unexport PLATFORM
unexport OS
GNU_HOST_NAME := $(shell support/gnuconfig/config.guess)
PACKAGES :=
PACKAGES_ALL :=
# silent mode requested?
QUIET := $(if $(findstring s,$(filter-out --%,$(MAKEFLAGS))),-q)
Remove the "project" feature The "project" feature was designed to allow to several projects to be built inside the same Buildroot source tree and allowing the toolchain and non-configurable packages to be shared between the different projects on the same architecture. While being interesting in theory, this feature adds a level of complexity to Buildroot, both from an user perspective and from a developer perspective, while one of the main Buildroot strengh is to be simple. Moreover, this feature is only seldomly used by our users. From a user-level perspective, this for example allows to remove the project_build_ARCH directory, which was very confusing. The autotools-stamps directory is also removed, since these stamps are back at their normal location. Description of the changes involved : * project/, directory removed * Makefile - Don't include project/Makefile.in and project/project.mk anymore - Grab a copy of the contents of project/Makefile.in at the location it was imported, but remove the definition related to PROJECT_BUILD_DIR. The TARGET_DIR is now in $(BUILD_DIR)/target_dir - Remove the creation/removal of the $(PROJECT_BUILD_DIR) and $(PROJECT_BUILD_DIR)/autotools-stamps directories - Don't make world depends on target-host-info. This target was defined by project/project.mk to customize /etc/issue, /etc/hostname and create /etc/br-version depending on the project definitions. We can of course imagine re-adding such a feature later. - Replace PROJECT_BUILD_DIR by BUILD_DIR everywhere - Remove the update, log and lognr.$(PROJECT) target, they were specific to the project feature. * package/Makefile.autotools.in - Replace PROJECT_BUILD_DIR by BUILD_DIR for the location of the configure cache - Move the INSTALL_TARGET and HOOK_POST_INSTALL stamps to the same directory as the other stamps (i.e, in the package directory). * package/Makefile.in - Replace PROJECT_BUILD_DIR by BUILD_DIR for the location of the configure cache * package/at/at.mk, package/busybox/busybox.mk, package/busybox/initramfs.mk, package/customize/customize.mk, package/linux-fusion/linux-fusion.mk, package/ltp-testsuite/ltp-testsuite.mk, package/nfs-utils/nfs-utils.mk, target/cpio/cpioroot.mk, target/cramfs/cramfs.mk, target/device/Atmel/DataFlashBoot/DataflashBoot.mk, target/device/Atmel/Makefile.in, target/device/Atmel/at91bootstrap/at91bootstrap.mk, target/device/KwikByte/Makefile.in, target/ext2/ext2root.mk, target/initramfs/initramfs.mk, target/iso9660/iso9660.mk, target/jffs2/jffs2root.mk, target/linux/Makefile.in, target/romfs/romfs.mk, target/squashfs/squashfsroot.mk, target/tar/tarroot.mk, target/ubifs/ubifsroot.mk - Replace PROJECT_BUILD_DIR by BUILD_DIR * target/device/Config.in - Do not include project/Config.in anymore * target/linux/Makefile.in.advanced - Replace PROJECT_BUILD_DIR by BUILD_DIR - Store the stamps file in $(STAMP_DIR) instead of $(PROJECT_BUILD_DIR)/autotools-stamps * target/u-boot/Makefile.in - Replace PROJECT_BUILD_DIR by BUILD_DIR - Remove $(PROJECT) from the U-Boot target binary name - Remove the insertion in the configuration of the project name as the hostname - The u-boot-autoscript target now generates $(U_BOOT_AUTOSCRIPT).img instead of $(U_BOOT_AUTOSCRIPT).$(PROJECT) * toolchain/gcc/gcc-uclibc-3.x.mk toolchain/gcc/gcc-uclibc-4.x.mk - Move the stamps files to $(STAMP_DIR) Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2009-09-05 15:49:30 +02:00
# Strip off the annoying quoting
ARCH := $(call qstrip,$(BR2_ARCH))
KERNEL_ARCH := $(shell echo "$(ARCH)" | sed -e "s/-.*//" \
-e s/i.86/i386/ -e s/sun4u/sparc64/ \
-e s/arcle/arc/ \
-e s/arceb/arc/ \
-e s/arm.*/arm/ -e s/sa110/arm/ \
-e s/aarch64.*/arm64/ \
-e s/nds32.*/nds32/ \
-e s/or1k/openrisc/ \
-e s/parisc64/parisc/ \
-e s/powerpc64.*/powerpc/ \
-e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
-e s/riscv.*/riscv/ \
-e s/sh.*/sh/ \
-e s/microblazeel/microblaze/)
ZCAT := $(call qstrip,$(BR2_ZCAT))
BZCAT := $(call qstrip,$(BR2_BZCAT))
XZCAT := $(call qstrip,$(BR2_XZCAT))
LZCAT := $(call qstrip,$(BR2_LZCAT))
TAR_OPTIONS = $(call qstrip,$(BR2_TAR_OPTIONS)) -xf
Remove the "project" feature The "project" feature was designed to allow to several projects to be built inside the same Buildroot source tree and allowing the toolchain and non-configurable packages to be shared between the different projects on the same architecture. While being interesting in theory, this feature adds a level of complexity to Buildroot, both from an user perspective and from a developer perspective, while one of the main Buildroot strengh is to be simple. Moreover, this feature is only seldomly used by our users. From a user-level perspective, this for example allows to remove the project_build_ARCH directory, which was very confusing. The autotools-stamps directory is also removed, since these stamps are back at their normal location. Description of the changes involved : * project/, directory removed * Makefile - Don't include project/Makefile.in and project/project.mk anymore - Grab a copy of the contents of project/Makefile.in at the location it was imported, but remove the definition related to PROJECT_BUILD_DIR. The TARGET_DIR is now in $(BUILD_DIR)/target_dir - Remove the creation/removal of the $(PROJECT_BUILD_DIR) and $(PROJECT_BUILD_DIR)/autotools-stamps directories - Don't make world depends on target-host-info. This target was defined by project/project.mk to customize /etc/issue, /etc/hostname and create /etc/br-version depending on the project definitions. We can of course imagine re-adding such a feature later. - Replace PROJECT_BUILD_DIR by BUILD_DIR everywhere - Remove the update, log and lognr.$(PROJECT) target, they were specific to the project feature. * package/Makefile.autotools.in - Replace PROJECT_BUILD_DIR by BUILD_DIR for the location of the configure cache - Move the INSTALL_TARGET and HOOK_POST_INSTALL stamps to the same directory as the other stamps (i.e, in the package directory). * package/Makefile.in - Replace PROJECT_BUILD_DIR by BUILD_DIR for the location of the configure cache * package/at/at.mk, package/busybox/busybox.mk, package/busybox/initramfs.mk, package/customize/customize.mk, package/linux-fusion/linux-fusion.mk, package/ltp-testsuite/ltp-testsuite.mk, package/nfs-utils/nfs-utils.mk, target/cpio/cpioroot.mk, target/cramfs/cramfs.mk, target/device/Atmel/DataFlashBoot/DataflashBoot.mk, target/device/Atmel/Makefile.in, target/device/Atmel/at91bootstrap/at91bootstrap.mk, target/device/KwikByte/Makefile.in, target/ext2/ext2root.mk, target/initramfs/initramfs.mk, target/iso9660/iso9660.mk, target/jffs2/jffs2root.mk, target/linux/Makefile.in, target/romfs/romfs.mk, target/squashfs/squashfsroot.mk, target/tar/tarroot.mk, target/ubifs/ubifsroot.mk - Replace PROJECT_BUILD_DIR by BUILD_DIR * target/device/Config.in - Do not include project/Config.in anymore * target/linux/Makefile.in.advanced - Replace PROJECT_BUILD_DIR by BUILD_DIR - Store the stamps file in $(STAMP_DIR) instead of $(PROJECT_BUILD_DIR)/autotools-stamps * target/u-boot/Makefile.in - Replace PROJECT_BUILD_DIR by BUILD_DIR - Remove $(PROJECT) from the U-Boot target binary name - Remove the insertion in the configuration of the project name as the hostname - The u-boot-autoscript target now generates $(U_BOOT_AUTOSCRIPT).img instead of $(U_BOOT_AUTOSCRIPT).$(PROJECT) * toolchain/gcc/gcc-uclibc-3.x.mk toolchain/gcc/gcc-uclibc-4.x.mk - Move the stamps files to $(STAMP_DIR) Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2009-09-05 15:49:30 +02:00
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
HOST_DIR = $(if $(PKG),$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/host,$(call qstrip,$(BR2_HOST_DIR)))
TARGET_DIR = $(if $(ROOTFS),$(ROOTFS_$(ROOTFS)_TARGET_DIR),$(if $(PKG),$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/target,$(BASE_TARGET_DIR)))
else
HOST_DIR := $(call qstrip,$(BR2_HOST_DIR))
TARGET_DIR = $(if $(ROOTFS),$(ROOTFS_$(ROOTFS)_TARGET_DIR),$(BASE_TARGET_DIR))
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
endif
ifneq ($(HOST_DIR),$(BASE_DIR)/host)
HOST_DIR_SYMLINK = $(BASE_DIR)/host
$(HOST_DIR_SYMLINK): $(BASE_DIR)
ln -snf $(HOST_DIR) $(HOST_DIR_SYMLINK)
endif
Makefile: don't recreate staging symlink if it exists Create the staging symlink the same way as the host symlink. This means using a make dependency rather than recreating it every time. In coreutils versions below 8.27, re-creation of symbolic links was not atomic. This means that there is a period in time where the existing link is removed, before the new one is created. In coreutils 8.27 this was fixed, see [1]. Note that CentOS 7 ships with coreutils 8.22. In the following scenario, this is a problem: - an application is compiled using the sysroot prepared by Buildroot and links against Xenomai userspace libraries, but its build process is steered from outside of Buildroot - to know the correct flags, the application makefile uses the 'xeno-config' file to request them, and passes DESTDIR=/buildroot/output/staging - the xeno-config responds with flags based on the path '/buildroot/output/staging/...' - while the application build is ongoing, a 'make' happens in Buildroot, causing the 'staging' symlink to be recreated (even though it already existed) - when exactly at this time, the application calls the compiler with -I flags pointing to output/staging, the build fails with: -I/buildroot/output/staging/usr/include/xenomai/mercury: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai/xenomai: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai/psos: Error: ^ is not a directory Failed: ** ^ * Work around this problem by only creating the staging symlink once, similar to how the host symlink (if any) is created. See also commit d0f4f95e390bcb1c953efa125f5277a8a235396e which changed the way these symlinks are made. The reasoning in this commit is to move away from the 'dirs' target. [1] https://github.com/coreutils/coreutils/commit/376967889ed7ed561e46ff6d88a66779db62737a Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-18 16:01:00 +01:00
STAGING_DIR_SYMLINK = $(BASE_DIR)/staging
$(STAGING_DIR_SYMLINK): $(BASE_DIR)
ln -snf $(STAGING_DIR) $(STAGING_DIR_SYMLINK)
# Quotes are needed for spaces and all in the original PATH content.
BR_PATH = "$(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(PATH)"
# Location of a file giving a big fat warning that output/target
# should not be used as the root filesystem.
Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR In commit 7e9870ce32d6329d9e3d602247fbe1709a2275a4 ("core: introduce intermediate BASE_TARGET_DIR variable"), the definition of TARGET_DIR_WARNING_FILE was changed to use $(BASE_TARGET_DIR) instead of $(TARGET_DIR). However, this change is incompatible with per-package directories, and is in fact not needed. With per-package directories, using $(BASE_TARGET_DIR) means that TARGET_DIR_WARNING_FILE is output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM. Due to this, when skeleton-init-common or skeleton-custom attempt to install it, it fails, because it should be installed to their package per-package target directory, and not the global output/target directory that doesn't exist yet. The failure looks like this: /usr/bin/install -m 0644 support/misc/target-dir-warning.txt /home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM /usr/bin/install: cannot create regular file '/home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM': No such file or directory make[1]: *** [package/pkg-generic.mk:336: /home/thomas/projets/buildroot/output/build/skeleton-init-common/.stamp_target_installed] Error 1 TARGET_DIR_WARNING_FILE is used in three places: - In skeleton-custom.mk and skeleton-init-common.mk, where as explained above, using $(TARGET_DIR) fixes the use of $(TARGET_DIR_WARNING_FILE) in the context of per-package target directories. - In fs/common.mk, where it is used as argument to $(notdir ...) to retrieve just the name of the warning file. So in this case, we really don't care about the path of the file, just its name. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-23 15:58:10 +01:00
TARGET_DIR_WARNING_FILE = $(TARGET_DIR)/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
ifeq ($(BR2_CCACHE),y)
CCACHE = $(HOST_DIR)/bin/ccache
BR_CACHE_DIR ?= $(call qstrip,$(BR2_CCACHE_DIR))
export BR_CACHE_DIR
HOSTCC = $(CCACHE) $(HOSTCC_NOCCACHE)
HOSTCXX = $(CCACHE) $(HOSTCXX_NOCCACHE)
else
export BR_NO_CCACHE
endif
# Scripts in support/ or post-build scripts may need to reference
# these locations, so export them so it is easier to use
export BR2_CONFIG
export BR2_REPRODUCIBLE
export TARGET_DIR
export STAGING_DIR
export HOST_DIR
export BINARIES_DIR
export BASE_DIR
################################################################################
#
# You should probably leave this stuff alone unless you know
# what you are doing.
#
################################################################################
2007-08-22 14:35:41 +02:00
all: world
2001-12-22 01:56:11 +01:00
# Include legacy before the other things, because package .mk files
# may rely on it.
include Makefile.legacy
include system/system.mk
include package/Makefile.in
# arch/arch.mk must be after package/Makefile.in because it may need to
# complement variables defined therein, like BR_NO_CHECK_HASH_FOR.
include arch/arch.mk
include support/dependencies/dependencies.mk
include $(sort $(wildcard toolchain/*.mk))
include $(sort $(wildcard toolchain/*/*.mk))
ifeq ($(BR2_REPRODUCIBLE),y)
# If SOURCE_DATE_EPOCH has not been set then use the commit date, or the last
# release date if the source tree is not within a Git repository.
# See: https://reproducible-builds.org/specs/source-date-epoch/
BR2_VERSION_GIT_EPOCH := $(shell $(GIT) log -1 --format=%at 2> /dev/null)
export SOURCE_DATE_EPOCH ?= $(or $(BR2_VERSION_GIT_EPOCH),$(BR2_VERSION_EPOCH))
endif
# Include the package override file if one has been provided in the
# configuration.
PACKAGE_OVERRIDE_FILE = $(call qstrip,$(BR2_PACKAGE_OVERRIDE_FILE))
ifneq ($(PACKAGE_OVERRIDE_FILE),)
-include $(PACKAGE_OVERRIDE_FILE)
endif
include $(sort $(wildcard package/*/*.mk))
include boot/common.mk
include linux/linux.mk
include fs/common.mk
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
# If using a br2-external tree, the BR2_EXTERNAL_$(NAME)_PATH variables
# are also present in the .config file. Since .config is included after
# we defined them in the Makefile, the values for those variables are
# quoted. We just include the generated Makefile fragment .br2-external.mk
# a third time, which will set those variables to the un-quoted values.
core: introduce per br2-external NAME This unique NAME is used to construct a per br2-external tree variable, BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external tree. This variable is available both from Kconfig (set in the Kconfig snippet) and from the .mk files. Also, display the NAME and its path as a comment in the menuconfig. This will ultimately allow us to support multiple br2-external trees at once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining which br2-external tree is being used. The obvious outcome is that BR2_EXTERNAL should now no longer be used to refer to the files in the br2-external tree; that location is now known from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no longer need to expose, and must stop from from exposing BR2_EXTERNAL as a Kconfig variable. Finally, this also fixes a latent bug in the pkg-generic infra, where we would so far always refer to BR2_EXTERNAL (even if not set) to filter the names of packages (to decide whether they are a bootloader, a toolchain or a simple package). Note: since the variables in the Makefile and in Kconfig are named the same, the one we computed early on in the Makefile will be overridden by the one in .config when we have it. Thus, even though they are set to the same raw value, the one from .config is quoted and, being included later in the Makefile, will take precedence, so we just re-include the generated Makefile fragment a third time before includeing the br2-external's Makefiles. That's unfortunate, but there is no easy way around that as we do want the two variables to be named the same in Makefile and Kconfig (and we can't ask the user to un-quote that variable himself either), hence this little dirty triple-inclusion trick. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:17 +02:00
include $(BR2_EXTERNAL_FILE)
# Nothing to include if no BR2_EXTERNAL tree in use
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
include $(BR2_EXTERNAL_MKS)
# Now we are sure we have all the packages scanned and defined. We now
# check for each package in the list of enabled packages, that all its
# dependencies are indeed enabled.
#
# Only trigger the check for default builds. If the user forces building
# a package, even if not enabled in the configuration, we want to accept
# it. However; we also want to be able to force checking the dependencies
# if the user so desires. Forcing a dependency check is useful in the case
# of test-pkg, as we want to make sure during testing, that a package has
# all the dependencies selected in the config file.
#
ifeq ($(MAKECMDGOALS),)
BR_FORCE_CHECK_DEPENDENCIES = YES
endif
ifeq ($(BR_FORCE_CHECK_DEPENDENCIES),YES)
define CHECK_ONE_DEPENDENCY
ifeq ($$($(2)_TYPE),target)
ifeq ($$($(2)_IS_VIRTUAL),)
ifneq ($$($$($(2)_KCONFIG_VAR)),y)
$$(error $$($(2)_NAME) is in the dependency chain of $$($(1)_NAME) that \
has added it to its _DEPENDENCIES variable without selecting it or \
depending on it from Config.in)
endif
endif
endif
endef
$(foreach pkg,$(call UPPERCASE,$(PACKAGES)),\
$(foreach dep,$(call UPPERCASE,$($(pkg)_FINAL_ALL_DEPENDENCIES)),\
$(eval $(call CHECK_ONE_DEPENDENCY,$(pkg),$(dep))$(sep))))
endif
$(BUILD_DIR)/buildroot-config/auto.conf: $(BR2_CONFIG)
$(MAKE1) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" syncconfig
.PHONY: prepare
prepare: $(BUILD_DIR)/buildroot-config/auto.conf
.PHONY: world
world: target-post-image
core/sdk: generate the SDK tarball ourselves Currently, the wording in the manual instructs the user to generate a tarball from "the contents of the +output/host+ directory". This is pretty confusing, because taken literally, this would amount to running a command like: tar cf my-sdk.tar -C output/host/ . This creates a tarbomb [0], which is very bad practice, because when extracted, it creates multiple files in the current directory. What one really wants to do, is create a tarball of the host/ directory, with something like: tar cf my-sdk.tar -C output host/ However, this is not much better, because the top-most directory would have a very common name, host/, which is pretty easy to get conflict with when it gets extracted. So, we fix that mess by giving the top-most directory a recognisable name, based on the target tuple, which we also use as the name of the archive (suffixed with the usual +.tar.gz+.) We offer the user the possibility to override that default by specifying the +BR2_SDK_PREFIX+ variable on the command line. Since this is an output file, we place it in the images/ directory. As some users expressed a very strong feeling that they do not want to generate a tarball at all, and that doing so would badly hurt their workflows [1], we actually prepare the SDK as was previously done, but under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule obviously depend on that before generating the tarball. We choose to make the existing rule to generate the tarball, and introduce a new rule to just prepare the SDK, rather than keep the existing rule as-is and introduce a new one to generate the tarball, because it makes sense to have the simplest rule do the correct thing, leaving advanced, power users use the longest command. If someone already had a wrapper that called 'sdk' and expected just the host directory to be prepared, then this is not broken; it just takes a bit longer (gzip is pretty fast). Update the manual accordingly. [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb [1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377 and some messages in the ensuing thread... Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Wolfgang Grandegger <wg@grandegger.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Stefan Becker <chemobejk@gmail.com> Cc: Trent Piepho <tpiepho@impinj.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Reviewed-by: Stefan Becker <chemobejk@gmail.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-04 10:27:41 +02:00
.PHONY: prepare-sdk
prepare-sdk: world
@$(call MESSAGE,"Rendering the SDK relocatable")
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
PER_PACKAGE_DIR=$(PER_PACKAGE_DIR) $(TOPDIR)/support/scripts/fix-rpath host
PER_PACKAGE_DIR=$(PER_PACKAGE_DIR) $(TOPDIR)/support/scripts/fix-rpath staging
$(INSTALL) -m 755 $(TOPDIR)/support/misc/relocate-sdk.sh $(HOST_DIR)/relocate-sdk.sh
mkdir -p $(HOST_DIR)/share/buildroot
echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
core/sdk: generate the SDK tarball ourselves Currently, the wording in the manual instructs the user to generate a tarball from "the contents of the +output/host+ directory". This is pretty confusing, because taken literally, this would amount to running a command like: tar cf my-sdk.tar -C output/host/ . This creates a tarbomb [0], which is very bad practice, because when extracted, it creates multiple files in the current directory. What one really wants to do, is create a tarball of the host/ directory, with something like: tar cf my-sdk.tar -C output host/ However, this is not much better, because the top-most directory would have a very common name, host/, which is pretty easy to get conflict with when it gets extracted. So, we fix that mess by giving the top-most directory a recognisable name, based on the target tuple, which we also use as the name of the archive (suffixed with the usual +.tar.gz+.) We offer the user the possibility to override that default by specifying the +BR2_SDK_PREFIX+ variable on the command line. Since this is an output file, we place it in the images/ directory. As some users expressed a very strong feeling that they do not want to generate a tarball at all, and that doing so would badly hurt their workflows [1], we actually prepare the SDK as was previously done, but under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule obviously depend on that before generating the tarball. We choose to make the existing rule to generate the tarball, and introduce a new rule to just prepare the SDK, rather than keep the existing rule as-is and introduce a new one to generate the tarball, because it makes sense to have the simplest rule do the correct thing, leaving advanced, power users use the longest command. If someone already had a wrapper that called 'sdk' and expected just the host directory to be prepared, then this is not broken; it just takes a bit longer (gzip is pretty fast). Update the manual accordingly. [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb [1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377 and some messages in the ensuing thread... Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Wolfgang Grandegger <wg@grandegger.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Stefan Becker <chemobejk@gmail.com> Cc: Trent Piepho <tpiepho@impinj.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Reviewed-by: Stefan Becker <chemobejk@gmail.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-04 10:27:41 +02:00
BR2_SDK_PREFIX ?= $(GNU_TARGET_NAME)_sdk-buildroot
.PHONY: sdk
sdk: prepare-sdk $(BR2_TAR_HOST_DEPENDENCY)
@$(call MESSAGE,"Generating SDK tarball")
$(if $(BR2_SDK_PREFIX),,$(error BR2_SDK_PREFIX can not be empty))
$(Q)mkdir -p $(BINARIES_DIR)
$(TAR) czf "$(BINARIES_DIR)/$(BR2_SDK_PREFIX).tar.gz" \
--owner=0 --group=0 --numeric-owner \
2018-12-22 16:18:52 +01:00
--transform='s#^$(patsubst /%,%,$(HOST_DIR))#$(BR2_SDK_PREFIX)#' \
-C / $(patsubst /%,%,$(HOST_DIR))
core/sdk: generate the SDK tarball ourselves Currently, the wording in the manual instructs the user to generate a tarball from "the contents of the +output/host+ directory". This is pretty confusing, because taken literally, this would amount to running a command like: tar cf my-sdk.tar -C output/host/ . This creates a tarbomb [0], which is very bad practice, because when extracted, it creates multiple files in the current directory. What one really wants to do, is create a tarball of the host/ directory, with something like: tar cf my-sdk.tar -C output host/ However, this is not much better, because the top-most directory would have a very common name, host/, which is pretty easy to get conflict with when it gets extracted. So, we fix that mess by giving the top-most directory a recognisable name, based on the target tuple, which we also use as the name of the archive (suffixed with the usual +.tar.gz+.) We offer the user the possibility to override that default by specifying the +BR2_SDK_PREFIX+ variable on the command line. Since this is an output file, we place it in the images/ directory. As some users expressed a very strong feeling that they do not want to generate a tarball at all, and that doing so would badly hurt their workflows [1], we actually prepare the SDK as was previously done, but under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule obviously depend on that before generating the tarball. We choose to make the existing rule to generate the tarball, and introduce a new rule to just prepare the SDK, rather than keep the existing rule as-is and introduce a new one to generate the tarball, because it makes sense to have the simplest rule do the correct thing, leaving advanced, power users use the longest command. If someone already had a wrapper that called 'sdk' and expected just the host directory to be prepared, then this is not broken; it just takes a bit longer (gzip is pretty fast). Update the manual accordingly. [0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb [1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377 and some messages in the ensuing thread... Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Wolfgang Grandegger <wg@grandegger.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Stefan Becker <chemobejk@gmail.com> Cc: Trent Piepho <tpiepho@impinj.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Reviewed-by: Stefan Becker <chemobejk@gmail.com> Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-04 10:27:41 +02:00
RSYNC_VCS_EXCLUSIONS = \
--exclude .svn --exclude .git --exclude .hg --exclude .bzr \
--exclude CVS
# When stripping, obey to BR2_STRIP_EXCLUDE_DIRS and
# BR2_STRIP_EXCLUDE_FILES
STRIP_FIND_COMMON_CMD = \
find $(TARGET_DIR) \
$(if $(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)), \
\( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) \
-prune -o \
) \
$(if $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES)), \
-not \( $(call findfileclauses,$(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) )
# Regular stripping for everything, except libpthread, ld-*.so and
# kernel modules:
# - libpthread.so: a non-stripped libpthread shared library is needed for
# proper debugging of pthread programs using gdb.
# - ld.so: a non-stripped dynamic linker library is needed for valgrind
# - kernel modules (*.ko): do not function properly when stripped like normal
# applications and libraries. Normally kernel modules are already excluded
# by the executable permission check, so the explicit exclusion is only
# done for kernel modules with incorrect permissions.
STRIP_FIND_CMD = \
$(STRIP_FIND_COMMON_CMD) \
-type f \( -perm /111 -o -name '*.so*' \) \
-not \( $(call findfileclauses,libpthread*.so* ld-*.so* *.ko) \) \
-print0
# Special stripping (only debugging symbols) for libpthread and ld-*.so.
STRIP_FIND_SPECIAL_LIBS_CMD = \
$(STRIP_FIND_COMMON_CMD) \
\( -name 'ld-*.so*' -o -name 'libpthread*.so*' \) \
-print0
ifeq ($(BR2_ECLIPSE_REGISTER),y)
define TOOLCHAIN_ECLIPSE_REGISTER
./support/scripts/eclipse-register-toolchain `readlink -f $(O)` \
$(notdir $(TARGET_CROSS)) $(BR2_ARCH)
endef
TARGET_FINALIZE_HOOKS += TOOLCHAIN_ECLIPSE_REGISTER
endif
# Generate locale data. Basically, we call the localedef program
# (built by the host-localedef package) for each locale. The input
# data comes preferably from the toolchain, or if the toolchain does
# not have them (Linaro toolchains), we use the ones available on the
# host machine.
ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y)
GLIBC_GENERATE_LOCALES = $(call qstrip,$(BR2_GENERATE_LOCALE))
ifneq ($(GLIBC_GENERATE_LOCALES),)
PACKAGES += host-localedef
define GENERATE_GLIBC_LOCALES
$(Q)mkdir -p $(TARGET_DIR)/usr/lib/locale/
$(Q)for locale in $(GLIBC_GENERATE_LOCALES) ; do \
inputfile=`echo $${locale} | cut -f1 -d'.'` ; \
charmap=`echo $${locale} | cut -f2 -d'.' -s` ; \
if test -z "$${charmap}" ; then \
charmap="UTF-8" ; \
fi ; \
echo "Generating locale $${inputfile}.$${charmap}" ; \
I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
$(HOST_DIR)/bin/localedef \
--prefix=$(TARGET_DIR) \
--$(call LOWERCASE,$(BR2_ENDIAN))-endian \
-i $${inputfile} -f $${charmap} \
$${locale} ; \
done
endef
TARGET_FINALIZE_HOOKS += GENERATE_GLIBC_LOCALES
endif
endif
ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)
LOCALE_WHITELIST = $(BUILD_DIR)/locales.nopurge
LOCALE_NOPURGE = $(call qstrip,$(BR2_ENABLE_LOCALE_WHITELIST))
# This piece of junk does the following:
# First collect the whitelist in a file.
# Then go over all the locale dirs and for each subdir, check if it exists
# in the whitelist file. If it doesn't, kill it.
# Finally, specifically for X11, regenerate locale.dir from the whitelist.
define PURGE_LOCALES
rm -f $(LOCALE_WHITELIST)
for i in $(LOCALE_NOPURGE) locale-archive; do echo $$i >> $(LOCALE_WHITELIST); done
Makefile: fix locale purge when BR2_PER_PACKAGE_DIRECTORIES=y With BR2_PER_PACKAGE_DIRECTORIES=y, we have the following code in the main Makefile: target-finalize: $(PACKAGES) $(TARGET_DIR) host-finalize @$(call MESSAGE,"Finalizing target directory") $(call per-package-rsync,$(sort $(PACKAGES)),target,$(TARGET_DIR)) $(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep)) The per-package-rsync call creates the global $(TARGET_DIR) from the per-package $(TARGET_DIR). Then, we call the TARGET_FINALIZE_HOOKS. One of the TARGET_FINALIZE_HOOKS, PURGE_LOCALES, remove locales that are not desired by the user. It does so using a loop with the $(wildcard ...) function. However, the $(wildcard ...) function is expanded at the moment the rule is evaluated. And with per-package directory, at the time the rule is evaluated, the global $(TARGET_DIR) is empty, so $(wildcard ...) will return nothing. It is indeed only after the call to per-package-rsync that the TARGET_DIR will be populated. This commit fixes that by moving away from $(wildcard ...) and use a shell test instead, since we are anyway in big block of shell code. With this, locales are properly purged again when BR2_PER_PACKAGE_DIRECTORIES=y. Fixes: c4e6d5c8be6ada8e7c60950e3b499c55d48761cb ("core: implement per-package SDK and target") Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> [yann.morin.1998@free.fr: - make the style look like the code around (no space in front of ;) ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-16 16:46:23 +01:00
for dir in $(addprefix $(TARGET_DIR),/usr/share/locale /usr/share/X11/locale /usr/lib/locale); \
do \
Makefile: fix locale purge when BR2_PER_PACKAGE_DIRECTORIES=y With BR2_PER_PACKAGE_DIRECTORIES=y, we have the following code in the main Makefile: target-finalize: $(PACKAGES) $(TARGET_DIR) host-finalize @$(call MESSAGE,"Finalizing target directory") $(call per-package-rsync,$(sort $(PACKAGES)),target,$(TARGET_DIR)) $(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep)) The per-package-rsync call creates the global $(TARGET_DIR) from the per-package $(TARGET_DIR). Then, we call the TARGET_FINALIZE_HOOKS. One of the TARGET_FINALIZE_HOOKS, PURGE_LOCALES, remove locales that are not desired by the user. It does so using a loop with the $(wildcard ...) function. However, the $(wildcard ...) function is expanded at the moment the rule is evaluated. And with per-package directory, at the time the rule is evaluated, the global $(TARGET_DIR) is empty, so $(wildcard ...) will return nothing. It is indeed only after the call to per-package-rsync that the TARGET_DIR will be populated. This commit fixes that by moving away from $(wildcard ...) and use a shell test instead, since we are anyway in big block of shell code. With this, locales are properly purged again when BR2_PER_PACKAGE_DIRECTORIES=y. Fixes: c4e6d5c8be6ada8e7c60950e3b499c55d48761cb ("core: implement per-package SDK and target") Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> [yann.morin.1998@free.fr: - make the style look like the code around (no space in front of ;) ] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-16 16:46:23 +01:00
if [ ! -d $$dir ]; then continue; fi; \
for langdir in $$dir/*; \
do \
if [ -e "$${langdir}" ]; \
then \
grep -qx "$${langdir##*/}" $(LOCALE_WHITELIST) || rm -rf $$langdir; \
fi \
done; \
done
if [ -d $(TARGET_DIR)/usr/share/X11/locale ]; \
then \
for lang in $(LOCALE_NOPURGE); \
do \
if [ -f $(TARGET_DIR)/usr/share/X11/locale/$$lang/XLC_LOCALE ]; \
then \
echo "$$lang/XLC_LOCALE: $$lang"; \
fi \
done > $(TARGET_DIR)/usr/share/X11/locale/locale.dir; \
fi
endef
TARGET_FINALIZE_HOOKS += PURGE_LOCALES
endif
$(TARGETS_ROOTFS): target-finalize
# Avoid the rootfs name leaking down the dependency chain
target-finalize: ROOTFS=
TARGET_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list.txt))
HOST_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-host.txt))
STAGING_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-staging.txt))
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
.PHONY: host-finalize
host-finalize: $(PACKAGES) $(HOST_DIR) $(HOST_DIR_SYMLINK)
@$(call MESSAGE,"Finalizing host directory")
$(call per-package-rsync,$(sort $(PACKAGES)),host,$(HOST_DIR))
.PHONY: staging-finalize
Makefile: don't recreate staging symlink if it exists Create the staging symlink the same way as the host symlink. This means using a make dependency rather than recreating it every time. In coreutils versions below 8.27, re-creation of symbolic links was not atomic. This means that there is a period in time where the existing link is removed, before the new one is created. In coreutils 8.27 this was fixed, see [1]. Note that CentOS 7 ships with coreutils 8.22. In the following scenario, this is a problem: - an application is compiled using the sysroot prepared by Buildroot and links against Xenomai userspace libraries, but its build process is steered from outside of Buildroot - to know the correct flags, the application makefile uses the 'xeno-config' file to request them, and passes DESTDIR=/buildroot/output/staging - the xeno-config responds with flags based on the path '/buildroot/output/staging/...' - while the application build is ongoing, a 'make' happens in Buildroot, causing the 'staging' symlink to be recreated (even though it already existed) - when exactly at this time, the application calls the compiler with -I flags pointing to output/staging, the build fails with: -I/buildroot/output/staging/usr/include/xenomai/mercury: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai/xenomai: Error: ^ is not a directory -I/buildroot/output/staging/usr/include/xenomai/psos: Error: ^ is not a directory Failed: ** ^ * Work around this problem by only creating the staging symlink once, similar to how the host symlink (if any) is created. See also commit d0f4f95e390bcb1c953efa125f5277a8a235396e which changed the way these symlinks are made. The reasoning in this commit is to move away from the 'dirs' target. [1] https://github.com/coreutils/coreutils/commit/376967889ed7ed561e46ff6d88a66779db62737a Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-18 16:01:00 +01:00
staging-finalize: $(STAGING_DIR_SYMLINK)
.PHONY: target-finalize
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
target-finalize: $(PACKAGES) $(TARGET_DIR) host-finalize
@$(call MESSAGE,"Finalizing target directory")
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
$(call per-package-rsync,$(sort $(PACKAGES)),target,$(TARGET_DIR))
$(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep))
rm -rf $(TARGET_DIR)/usr/include $(TARGET_DIR)/usr/share/aclocal \
$(TARGET_DIR)/usr/lib/pkgconfig $(TARGET_DIR)/usr/share/pkgconfig \
package/pkg-qmake: new qmake package infrastructure This provides generic functions for Qt5 qmake based packages. It will make it possible to remove lots of redefinition of QT5_xxx_{CONFIGURE|BUILD|INSTALL_STAGING}_CMDS. Additionally it provides a generic target install method which will make most of the package specific commands obsolete. This is done by re-running the install step of the qmake generated Makefile with the package build directory prepended (to the staging/host path). Even though this does create lengthy pathes it allows for easy separation of the staging files from the host destined files by just omitting the resulting BUILD_DIR+HOST_DIR path from the following rsync call to the real target folder. The cleanup of many files we dont want in target is deferred to the target-finalize step. In addition to what's being removed already, we also have to cleanup some Qt5 specific files (prl) and the documentation directory. This approach was chosen over copying all files recorded in the pkg-files-list after some discussion which Thomas Petazzoni summed up: "We don't yet use pkg-files-list really as part of the build process anywhere, I feel a bit more comfortable at this point with what Andreas is proposing." Thanks to this infrastructure, it will be possible to get rid of the many conditional install commands because qmake already takes care of this when generating the Makefile install targets with the given or autodetected configure options of each package. However, custom install steps may have to remain in cases where a particular Buildroot option has no corresponding setting in the packages configuration options. Signed-off-by: Andreas Naumann <anaumann@ultratronik.de> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-17 22:23:24 +01:00
$(TARGET_DIR)/usr/lib/cmake $(TARGET_DIR)/usr/share/cmake \
$(TARGET_DIR)/usr/doc
find $(TARGET_DIR)/usr/{lib,share}/ -name '*.cmake' -print0 | xargs -0 rm -f
find $(TARGET_DIR)/lib/ $(TARGET_DIR)/usr/lib/ $(TARGET_DIR)/usr/libexec/ \
package/pkg-qmake: new qmake package infrastructure This provides generic functions for Qt5 qmake based packages. It will make it possible to remove lots of redefinition of QT5_xxx_{CONFIGURE|BUILD|INSTALL_STAGING}_CMDS. Additionally it provides a generic target install method which will make most of the package specific commands obsolete. This is done by re-running the install step of the qmake generated Makefile with the package build directory prepended (to the staging/host path). Even though this does create lengthy pathes it allows for easy separation of the staging files from the host destined files by just omitting the resulting BUILD_DIR+HOST_DIR path from the following rsync call to the real target folder. The cleanup of many files we dont want in target is deferred to the target-finalize step. In addition to what's being removed already, we also have to cleanup some Qt5 specific files (prl) and the documentation directory. This approach was chosen over copying all files recorded in the pkg-files-list after some discussion which Thomas Petazzoni summed up: "We don't yet use pkg-files-list really as part of the build process anywhere, I feel a bit more comfortable at this point with what Andreas is proposing." Thanks to this infrastructure, it will be possible to get rid of the many conditional install commands because qmake already takes care of this when generating the Makefile install targets with the given or autodetected configure options of each package. However, custom install steps may have to remain in cases where a particular Buildroot option has no corresponding setting in the packages configuration options. Signed-off-by: Andreas Naumann <anaumann@ultratronik.de> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-02-17 22:23:24 +01:00
\( -name '*.a' -o -name '*.la' -o -name '*.prl' \) -print0 | xargs -0 rm -f
ifneq ($(BR2_PACKAGE_GDB),y)
rm -rf $(TARGET_DIR)/usr/share/gdb
endif
ifneq ($(BR2_PACKAGE_BASH),y)
rm -rf $(TARGET_DIR)/usr/share/bash-completion
endif
ifneq ($(BR2_PACKAGE_ZSH),y)
rm -rf $(TARGET_DIR)/usr/share/zsh
endif
rm -rf $(TARGET_DIR)/usr/man $(TARGET_DIR)/usr/share/man
rm -rf $(TARGET_DIR)/usr/info $(TARGET_DIR)/usr/share/info
rm -rf $(TARGET_DIR)/usr/doc $(TARGET_DIR)/usr/share/doc
rm -rf $(TARGET_DIR)/usr/share/gtk-doc
rmdir $(TARGET_DIR)/usr/share 2>/dev/null || true
$(STRIP_FIND_CMD) | xargs -0 $(STRIPCMD) 2>/dev/null || true
$(STRIP_FIND_SPECIAL_LIBS_CMD) | xargs -0 -r $(STRIPCMD) $(STRIP_STRIP_DEBUG) 2>/dev/null || true
test -f $(TARGET_DIR)/etc/ld.so.conf && \
{ echo "ERROR: we shouldn't have a /etc/ld.so.conf file"; exit 1; } || true
test -d $(TARGET_DIR)/etc/ld.so.conf.d && \
{ echo "ERROR: we shouldn't have a /etc/ld.so.conf.d directory"; exit 1; } || true
mkdir -p $(TARGET_DIR)/etc
( \
echo "NAME=Buildroot"; \
echo "VERSION=$(BR2_VERSION_FULL)"; \
echo "ID=buildroot"; \
echo "VERSION_ID=$(BR2_VERSION)"; \
echo "PRETTY_NAME=\"Buildroot $(BR2_VERSION)\"" \
) > $(TARGET_DIR)/usr/lib/os-release
ln -sf ../usr/lib/os-release $(TARGET_DIR)/etc
@$(call MESSAGE,"Sanitizing RPATH in target tree")
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
PER_PACKAGE_DIR=$(PER_PACKAGE_DIR) $(TOPDIR)/support/scripts/fix-rpath target
# For a merged /usr, ensure that /lib, /bin and /sbin and their /usr
# counterparts are appropriately setup as symlinks ones to the others.
ifeq ($(BR2_ROOTFS_MERGED_USR),y)
@$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \
$(call MESSAGE,"Sanity check in overlay $(d)"); \
not_merged_dirs="$$(support/scripts/check-merged-usr.sh $(d))"; \
test -n "$$not_merged_dirs" && { \
echo "ERROR: The overlay in $(d) is not" \
"using a merged /usr for the following directories:" \
$$not_merged_dirs; \
exit 1; \
} || true$(sep))
endif # merged /usr
@$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \
$(call MESSAGE,"Copying overlay $(d)"); \
$(call SYSTEM_RSYNC,$(d),$(TARGET_DIR))$(sep))
$(if $(TARGET_DIR_FILES_LISTS), \
cat $(TARGET_DIR_FILES_LISTS)) > $(BUILD_DIR)/packages-file-list.txt
$(if $(HOST_DIR_FILES_LISTS), \
cat $(HOST_DIR_FILES_LISTS)) > $(BUILD_DIR)/packages-file-list-host.txt
$(if $(STAGING_DIR_FILES_LISTS), \
cat $(STAGING_DIR_FILES_LISTS)) > $(BUILD_DIR)/packages-file-list-staging.txt
core: fix packages-file-list.txt after an incremental build The package instrumentation step 'step_pkg_size' is populating the files: output/build/packages-file-list.txt output/build/packages-file-list-staging.txt output/build/packages-file-list-host.txt by comparing the list of files before and after installation of a package, with some clever tricks to detect changes to existing files etc. As an optimization, instead of gathering this list before and after each package, where the 'after-state' of one package is the same as the 'before-state' of the next package, only the 'after-state' is used and is shared between packages. This works fine, except at the end of the build, as explained next. In the target-finalize step, many files will be touched. For example, files like /etc/hosts, /etc/os-release, but also all object files that are stripped, and all files touched by post-build scripts or created by rootfs overlays. This means that the 'after-state' of the last package does not reflect the actual situation after target-finalize is run. For a single complete build this poses no problem. But, if one incrementally rebuilds a package after the initial build, e.g. with 'make foo-rebuild', then all changes that happened in target-finalize at the end of the initial build (the 'after-state' of the last package built) will be detected as changes caused by the rebuild of package foo. As a result, all these files will incorrectly be treated as 'owned' by package foo. Correct this situation by capturing a new state at the end of target-finalize, so that the 'before-state' of an incremental build will be correct. Note: the reasoning above talks about packages-file-list.txt and target-finalize, but also applies to packages-file-list-staging.txt/staging-finalize and packages-file-list-host.txt/host-finalize. Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-02-14 20:57:33 +01:00
@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_BUILD_SCRIPT)), \
$(call MESSAGE,"Executing post-build script $(s)"); \
$(EXTRA_ENV) $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
touch $(TARGET_DIR)/usr
.PHONY: target-post-image
target-post-image: $(TARGETS_ROOTFS) target-finalize staging-finalize
@rm -f $(ROOTFS_COMMON_TAR)
$(Q)mkdir -p $(BINARIES_DIR)
@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
$(call MESSAGE,"Executing post-image script $(s)"); \
$(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
.PHONY: source
source: $(foreach p,$(PACKAGES),$(p)-all-source)
2002-04-26 13:45:55 +02:00
.PHONY: _external-deps external-deps
_external-deps: $(foreach p,$(PACKAGES),$(p)-all-external-deps)
external-deps:
@$(MAKE1) -Bs $(EXTRAMAKEARGS) _external-deps | sort -u
.PHONY: legal-info-clean
legal-info-clean:
@rm -fr $(LEGAL_INFO_DIR)
.PHONY: legal-info-prepare
legal-info-prepare: $(LEGAL_INFO_DIR)
@$(call MESSAGE,"Buildroot $(BR2_VERSION_FULL) Collecting legal info")
@$(call legal-license-file,buildroot,buildroot,support/legal-info/buildroot.hash,COPYING,COPYING,HOST)
@$(call legal-manifest,TARGET,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,DEPENDENCIES WITH LICENSES)
@$(call legal-manifest,HOST,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,DEPENDENCIES WITH LICENSES)
@$(call legal-manifest,HOST,buildroot,$(BR2_VERSION_FULL),GPL-2.0+,COPYING,not saved,not saved)
@$(call legal-warning,the Buildroot source code has not been saved)
@cp $(BR2_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
.PHONY: legal-info
legal-info: legal-info-clean legal-info-prepare $(foreach p,$(PACKAGES),$(p)-all-legal-info) \
$(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST)
@cat support/legal-info/README.header >>$(LEGAL_REPORT)
@if [ -r $(LEGAL_WARNINGS) ]; then \
cat support/legal-info/README.warnings-header \
$(LEGAL_WARNINGS) >>$(LEGAL_REPORT); \
cat $(LEGAL_WARNINGS); fi
@rm -f $(LEGAL_WARNINGS)
@(cd $(LEGAL_INFO_DIR); \
find * -type f -exec sha256sum {} + | LC_ALL=C sort -k2 \
>.legal-info.sha256; \
mv .legal-info.sha256 legal-info.sha256)
@echo "Legal info produced in $(LEGAL_INFO_DIR)"
.PHONY: show-targets
show-targets:
@echo $(sort $(PACKAGES)) $(sort $(TARGETS_ROOTFS))
.PHONY: show-build-order
show-build-order: $(patsubst %,%-show-build-order,$(PACKAGES))
.PHONY: graph-build
graph-build: $(O)/build/build-time.log
@install -d $(GRAPHS_DIR)
$(foreach o,name build duration,./support/scripts/graph-build-time \
--type=histogram --order=$(o) --input=$(<) \
--output=$(GRAPHS_DIR)/build.hist-$(o).$(BR_GRAPH_OUT) \
$(if $(BR2_GRAPH_ALT),--alternate-colors)$(sep))
$(foreach t,packages steps,./support/scripts/graph-build-time \
--type=pie-$(t) --input=$(<) \
--output=$(GRAPHS_DIR)/build.pie-$(t).$(BR_GRAPH_OUT) \
$(if $(BR2_GRAPH_ALT),--alternate-colors)$(sep))
.PHONY: graph-depends-requirements
graph-depends-requirements:
@dot -? >/dev/null 2>&1 || \
{ echo "ERROR: The 'dot' program from Graphviz is needed for graph-depends" >&2; exit 1; }
.PHONY: graph-depends
graph-depends: graph-depends-requirements
@$(INSTALL) -d $(GRAPHS_DIR)
@cd "$(CONFIG_DIR)"; \
$(TOPDIR)/support/scripts/graph-depends $(BR2_GRAPH_DEPS_OPTS) \
--direct -o $(GRAPHS_DIR)/$(@).dot
dot $(BR2_GRAPH_DOT_OPTS) -T$(BR_GRAPH_OUT) \
-o $(GRAPHS_DIR)/$(@).$(BR_GRAPH_OUT) \
$(GRAPHS_DIR)/$(@).dot
.PHONY: graph-size
graph-size:
$(Q)mkdir -p $(GRAPHS_DIR)
$(Q)$(TOPDIR)/support/scripts/size-stats --builddir $(BASE_DIR) \
--graph $(GRAPHS_DIR)/graph-size.$(BR_GRAPH_OUT) \
--file-size-csv $(GRAPHS_DIR)/file-size-stats.csv \
--package-size-csv $(GRAPHS_DIR)/package-size-stats.csv \
$(BR2_GRAPH_SIZE_OPTS)
.PHONY: check-dependencies
check-dependencies:
@cd "$(CONFIG_DIR)"; \
$(TOPDIR)/support/scripts/graph-depends -C
core: introduce new global show-info Users are increasingly trying to extract information about packages. For example, they might need to get the list of URIs, or the dependencies of a package. Although we do have a bunch of rules to generate some of that, this is done in ad-hoc way, with most of the output formats just ad-hoc, raw, unformatted blurbs, mostly internal data dumped as-is. Introduce a new rule, show-info, that provides a properly formatted output of all the meta-information about packages: name, type, version, licenses, dependencies... We choose to use JSON as the output format, because it is pretty versatile, has parsers in virtually all languages, has tools to parse from the shell (jq). It also closely matches Python data structure, which makes it easy to use with our own internal tools as well. Finally, JSON being a key-value store, allows for easy expanding the output without requiring existing consumers to be updated; new, unknown keys are simply ignored by those (as long as they are true JSON parsers). The complex part of this change was the conditional output of parts of the data: virtual packages have no source, version, license or downloads, unlike non-virtual packages. Same goes for filesystems. We use a wrapper macro, show-info, that de-multiplexes unto either the package-related- or filesystem-related macros, and for packages, we also use a detailed macro for non-virtual packages. It is non-trivial to properly output correct JSON blurbs, especially when trying to output an array of objects, like so, where the last item shall not be followed by a comma: [ { ... }, { ... } ] So, we use a trick (as sugegsted by Arnout), to $(subst) any pair of ",}" or ", }" or ",]" or ", ]" with only the respective closing symbol, "}" or "]". The whole stuff is $(strip)ed to make it a somewhat-minified JSON blurb that fits on a single line with all spaces squashed (but still with spaces, as it is not possible to differentiate spaces between JSON elements from spaces inside JSON strings). Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-04-15 21:47:31 +02:00
.PHONY: show-info
show-info:
@:
$(info $(call clean-json, \
{ $(foreach p, \
$(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \
$(i) \
$($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \
) \
), \
$(call json-info,$(call UPPERCASE,$(p)))$(comma) \
) } \
) \
)
else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
# Some subdirectories are also package names. To avoid that "make linux"
# on an unconfigured tree produces "Nothing to be done", add an explicit
# rule for it.
# Also for 'all' we error out and ask the user to configure first.
.PHONY: linux toolchain
linux toolchain all: outputmakefile
$(error Please configure Buildroot first (e.g. "make menuconfig"))
@exit 1
endif # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
# configuration
# ---------------------------------------------------------------------------
HOSTCFLAGS = $(CFLAGS_FOR_BUILD)
export HOSTCFLAGS
$(BUILD_DIR)/buildroot-config/%onf:
mkdir -p $(@D)/lxdialog
PKG_CONFIG_PATH="$(HOST_PKG_CONFIG_PATH)" $(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)" \
obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
DEFCONFIG = $(call qstrip,$(BR2_DEFCONFIG))
# We don't want to fully expand BR2_DEFCONFIG here, so Kconfig will
# recognize that if it's still at its default $(CONFIG_DIR)/defconfig
COMMON_CONFIG_ENV = \
BR2_DEFCONFIG='$(call qstrip,$(value BR2_DEFCONFIG))' \
KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \
KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \
KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
BR2_CONFIG=$(BR2_CONFIG) \
HOST_GCC_VERSION="$(HOSTCC_VERSION)" \
BASE_DIR=$(BASE_DIR) \
SKIP_LEGACY=
xconfig: $(BUILD_DIR)/buildroot-config/qconf outputmakefile
@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
gconfig: $(BUILD_DIR)/buildroot-config/gconf outputmakefile
@$(COMMON_CONFIG_ENV) srctree=$(TOPDIR) $< $(CONFIG_CONFIG_IN)
menuconfig: $(BUILD_DIR)/buildroot-config/mconf outputmakefile
@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
nconfig: $(BUILD_DIR)/buildroot-config/nconf outputmakefile
@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
config: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
# For the config targets that automatically select options, we pass
# SKIP_LEGACY=y to disable the legacy options. However, in that case
# no values are set for the legacy options so a subsequent oldconfig
# will query them. Therefore, run an additional olddefconfig.
randconfig allyesconfig alldefconfig allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --$@ $(CONFIG_CONFIG_IN)
@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
randpackageconfig allyespackageconfig allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
$< --$(subst package,,$@) $(CONFIG_CONFIG_IN)
@rm -f $(CONFIG_DIR)/.config.nopkg
@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
oldconfig syncconfig olddefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@$(COMMON_CONFIG_ENV) $< --$@ $(CONFIG_CONFIG_IN)
defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
define percent_defconfig
# Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig outputmakefile
@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(1)/configs/$$@ \
$$< --defconfig=$(1)/configs/$$@ $$(CONFIG_CONFIG_IN)
endef
$(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_defconfig,$(d))$(sep)))
update-defconfig: savedefconfig
savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@$(COMMON_CONFIG_ENV) $< \
--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
$(CONFIG_CONFIG_IN)
@$(SED) '/BR2_DEFCONFIG=/d' $(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig)
.PHONY: defconfig savedefconfig update-defconfig
################################################################################
#
# Cleanup and misc junk
#
################################################################################
# staging and target directories do NOT list these as
# dependencies anywhere else
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
@mkdir -p $@
# outputmakefile generates a Makefile in the output directory, if using a
# separate output directory. This allows convenient use of make in the
# output directory.
.PHONY: outputmakefile
outputmakefile:
ifeq ($(NEED_WRAPPER),y)
$(Q)$(TOPDIR)/support/scripts/mkmakefile $(TOPDIR) $(O)
endif
# printvars prints all the variables currently defined in our
# Makefiles. Alternatively, if a non-empty VARS variable is passed,
# only the variables matching the make pattern passed in VARS are
# displayed.
.PHONY: printvars
printvars:
@:
$(foreach V, \
$(sort $(filter $(VARS),$(.VARIABLES))), \
$(if $(filter-out environment% default automatic, \
$(origin $V)), \
core: enhance printvars Currently, the output of printvars copntains the name of the variable, its expanded value and its un-expanded value. However, most of the time, we need the actual, expanded value, so it can be re-used from a (non-Buildroot) infrastructure script, like a post-build script, or a build-farm driver (e.g. a Jenkins job...) Add two options that a user may set to change the output of printvars: - QUOTED_VARS, if set, will quote the value - RAW_VARS, if set, will print the unexpanded value The new output by default only prints the expanded value now. So that it can be used as such: $ make -s printvars VARS=BUSYBOX_VERSION BUSYBOX_VERSION=1.26.2 $ make -s printvars VARS=BUSYBOX_RDEPENDENCIES QUOTED_VARS=YES BUSYBOX_RDEPENDENCIES='ncurses util-linux' $ make -s printvars VARS=BUSYBOX_FINAL_PATCH_DEPENDENCIES RAW_VARS=YES BUSYBOX_FINAL_PATCH_DEPENDENCIES=$(sort $(BUSYBOX_PATCH_DEPENDENCIES)) And it is even possible to directly evaluate it in a shell script: eval $(make -s printvars VARS=BUSYBOX_VERSION QUOTED_VARS=YES) Backward compatibility of the output is not maintained. It is believed that scripts that depended on the previous output were very fragile to begin with, because they had to filter the non-formatted output (splitting on spaces or braces was not really possible, because values could contain either). Document printvars and its options in the manual; list it in the output of 'make help'. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-29 11:43:05 +02:00
$(if $(QUOTED_VARS),\
$(info $V='$(subst ','\'',$(if $(RAW_VARS),$(value $V),$($V)))'), \
$(info $V=$(if $(RAW_VARS),$(value $V),$($V))))))
# ' Syntax colouring...
.PHONY: clean
clean:
rm -rf $(BASE_TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) $(HOST_DIR_SYMLINK) \
$(BUILD_DIR) $(BASE_DIR)/staging \
core: implement per-package SDK and target This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 17:46:40 +01:00
$(LEGAL_INFO_DIR) $(GRAPHS_DIR) $(PER_PACKAGE_DIR)
.PHONY: distclean
distclean: clean
ifeq ($(O),$(CURDIR)/output)
rm -rf $(O)
endif
rm -rf $(TOPDIR)/dl $(BR2_CONFIG) $(CONFIG_DIR)/.config.old $(CONFIG_DIR)/..config.tmp \
$(CONFIG_DIR)/.auto.deps $(BASE_DIR)/.br2-external.*
.PHONY: help
help:
2007-07-08 14:20:58 +02:00
@echo 'Cleaning:'
@echo ' clean - delete all files created by build'
2007-07-08 14:20:58 +02:00
@echo ' distclean - delete all non-source files (including .config)'
@echo
@echo 'Build:'
@echo ' all - make world'
@echo ' toolchain - build toolchain'
@echo ' sdk - build relocatable SDK'
2007-07-08 14:20:58 +02:00
@echo
@echo 'Configuration:'
@echo ' menuconfig - interactive curses-based configurator'
@echo ' nconfig - interactive ncurses-based configurator'
@echo ' xconfig - interactive Qt-based configurator'
@echo ' gconfig - interactive GTK-based configurator'
2007-07-08 14:20:58 +02:00
@echo ' oldconfig - resolve any unresolved symbols in .config'
@echo ' syncconfig - Same as oldconfig, but quietly, additionally update deps'
@echo ' olddefconfig - Same as syncconfig but sets new symbols to their default value'
@echo ' randconfig - New config with random answer to all options'
@echo ' defconfig - New config with default answer to all options;'
@echo ' BR2_DEFCONFIG, if set on the command line, is used as input'
@echo ' savedefconfig - Save current config to BR2_DEFCONFIG (minimal config)'
@echo ' update-defconfig - Same as savedefconfig'
@echo ' allyesconfig - New config where all options are accepted with yes'
@echo ' allnoconfig - New config where all options are answered with no'
@echo ' alldefconfig - New config where all options are set to default'
@echo ' randpackageconfig - New config with random answer to package options'
@echo ' allyespackageconfig - New config where pkg options are accepted with yes'
@echo ' allnopackageconfig - New config where package options are answered with no'
@echo
@echo 'Package-specific:'
@echo ' <pkg> - Build and install <pkg> and all its dependencies'
@echo ' <pkg>-source - Only download the source files for <pkg>'
@echo ' <pkg>-extract - Extract <pkg> sources'
@echo ' <pkg>-patch - Apply patches to <pkg>'
@echo ' <pkg>-depends - Build <pkg>'\''s dependencies'
@echo ' <pkg>-configure - Build <pkg> up to the configure step'
@echo ' <pkg>-build - Build <pkg> up to the build step'
@echo ' <pkg>-show-info - generate info about <pkg>, as a JSON blurb'
@echo ' <pkg>-show-depends - List packages on which <pkg> depends'
@echo ' <pkg>-show-rdepends - List packages which have <pkg> as a dependency'
@echo ' <pkg>-show-recursive-depends'
@echo ' - Recursively list packages on which <pkg> depends'
@echo ' <pkg>-show-recursive-rdepends'
@echo ' - Recursively list packages which have <pkg> as a dependency'
@echo ' <pkg>-graph-depends - Generate a graph of <pkg>'\''s dependencies'
@echo ' <pkg>-graph-rdepends - Generate a graph of <pkg>'\''s reverse dependencies'
@echo ' <pkg>-dirclean - Remove <pkg> build directory'
@echo ' <pkg>-reconfigure - Restart the build from the configure step'
@echo ' <pkg>-rebuild - Restart the build from the build step'
core: name the package before its help text Currently, all our internal packages provide actions that are prefixed with their own names. This makes it obvious what package the action refer to. However, the help commands are really free-form. This means that packages (and especially packages from a br2-external tree) may provide completely arbitrary help text. As such, all that text can get pretty easily mixed up, and it will be very difficult to read. Prefix each package-specific help text with the name of the package it refers to. This generate a "make help" that looks like: [...] Package-specific: <pkg> - Build and install <pkg> and all its dependencies <pkg>-source - Only download the source files for <pkg> <pkg>-extract - Extract <pkg> sources <pkg>-patch - Apply patches to <pkg> <pkg>-depends - Build <pkg>'s dependencies <pkg>-configure - Build <pkg> up to the configure step <pkg>-build - Build <pkg> up to the build step <pkg>-graph-depends - Generate a graph of <pkg>'s dependencies <pkg>-dirclean - Remove <pkg> build directory <pkg>-reconfigure - Restart the build from the configure step <pkg>-rebuild - Restart the build from the build step busybox: busybox-menuconfig - Run BusyBox menuconfig busybox-nconfig - Run BusyBox nconfig barebox: barebox-menuconfig - Run barebox menuconfig barebox-savedefconfig - Run barebox savedefconfig linux: linux-menuconfig - Run Linux kernel menuconfig linux-savedefconfig - Run Linux kernel savedefconfig linux-update-defconfig - Save the Linux configuration to the path specified by BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE Documentation: manual - build manual in all formats manual-html - build manual in HTML [...] (Note: busybox, barebox, linux help will be converted in followup commits, they are represented here as an example of what this patch does look like.) Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-04 18:30:48 +02:00
$(foreach p,$(HELP_PACKAGES), \
@echo $(sep) \
@echo '$($(p)_NAME):' $(sep) \
$($(p)_HELP_CMDS)$(sep))
@echo
@echo 'Documentation:'
@echo ' manual - build manual in all formats'
@echo ' manual-html - build manual in HTML'
@echo ' manual-split-html - build manual in split HTML'
@echo ' manual-pdf - build manual in PDF'
@echo ' manual-text - build manual in text'
@echo ' manual-epub - build manual in ePub'
@echo ' graph-build - generate graphs of the build times'
@echo ' graph-depends - generate graph of the dependency tree'
@echo ' graph-size - generate stats of the filesystem size'
@echo ' list-defconfigs - list all defconfigs (pre-configured minimal systems)'
2007-07-08 14:20:58 +02:00
@echo
@echo 'Miscellaneous:'
@echo ' source - download all sources needed for offline-build'
@echo ' external-deps - list external packages used'
@echo ' legal-info - generate info about license compliance'
core: introduce new global show-info Users are increasingly trying to extract information about packages. For example, they might need to get the list of URIs, or the dependencies of a package. Although we do have a bunch of rules to generate some of that, this is done in ad-hoc way, with most of the output formats just ad-hoc, raw, unformatted blurbs, mostly internal data dumped as-is. Introduce a new rule, show-info, that provides a properly formatted output of all the meta-information about packages: name, type, version, licenses, dependencies... We choose to use JSON as the output format, because it is pretty versatile, has parsers in virtually all languages, has tools to parse from the shell (jq). It also closely matches Python data structure, which makes it easy to use with our own internal tools as well. Finally, JSON being a key-value store, allows for easy expanding the output without requiring existing consumers to be updated; new, unknown keys are simply ignored by those (as long as they are true JSON parsers). The complex part of this change was the conditional output of parts of the data: virtual packages have no source, version, license or downloads, unlike non-virtual packages. Same goes for filesystems. We use a wrapper macro, show-info, that de-multiplexes unto either the package-related- or filesystem-related macros, and for packages, we also use a detailed macro for non-virtual packages. It is non-trivial to properly output correct JSON blurbs, especially when trying to output an array of objects, like so, where the last item shall not be followed by a comma: [ { ... }, { ... } ] So, we use a trick (as sugegsted by Arnout), to $(subst) any pair of ",}" or ", }" or ",]" or ", ]" with only the respective closing symbol, "}" or "]". The whole stuff is $(strip)ed to make it a somewhat-minified JSON blurb that fits on a single line with all spaces squashed (but still with spaces, as it is not possible to differentiate spaces between JSON elements from spaces inside JSON strings). Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle <arnout@mind.be> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-04-15 21:47:31 +02:00
@echo ' show-info - generate info about packages, as a JSON blurb'
@echo ' printvars - dump internal variables selected with VARS=...'
2007-07-08 14:20:58 +02:00
@echo
@echo ' make V=0|1 - 0 => quiet build (default), 1 => verbose build'
@echo ' make O=dir - Locate all output files in "dir", including .config'
@echo
@echo 'For further details, see README, generate the Buildroot manual, or consult'
@echo 'it on-line at http://buildroot.org/docs.html'
@echo
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
# List the defconfig files
# $(1): base directory
# $(2): br2-external name, empty for bundled
define list-defconfigs
@first=true; \
for defconfig in $(1)/configs/*_defconfig; do \
[ -f "$${defconfig}" ] || continue; \
if $${first}; then \
if [ "$(2)" ]; then \
printf 'External configs in "$(call qstrip,$(2))":\n'; \
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
else \
printf "Built-in configs:\n"; \
fi; \
first=false; \
fi; \
defconfig="$${defconfig##*/}"; \
printf " %-35s - Build for %s\n" "$${defconfig}" "$${defconfig%_defconfig}"; \
done; \
$${first} || printf "\n"
endef
# We iterate over BR2_EXTERNAL_NAMES rather than BR2_EXTERNAL_DIRS,
# because we want to display the name of the br2-external tree.
.PHONY: list-defconfigs
list-defconfigs:
core: add support for multiple br2-external trees Currently, we only support at most one br2-external tree. Being able to use more than one br2-external tree can be very useful. A use-case would be for having a br2-external to contain the basic packages, basic board defconfigs and board files, provided by one team responsible for the "board-bringup", while other teams consume that br2-external as a base, and complements it each with their own set of packages, defconfigs and extra board files. Another use-case would be for third-parties to provide their own Buildroot packaging in a br2-external tree, along-side the archives for their stuff. Finally, another use-case is to be able to add FLOSS packages in a br2-external tree, and proprietary packages in another. This allows to not touch the Buildroot tree at all, and still be able to get in compliance by providing only that br2-external tree(s) that contains FLOSS packages, leaving aside the br2-external tree(s) with the proprietary bits. What we do is to treat BR2_EXTERNAL as a colon-separated (space- separated also work, and we use that internally) list of paths, on which we iterate to construct: - the list of all br2-external names, BR2_EXTERNAL_NAMES, - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which point each to the actual location of the corresponding tree, - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS, - the space-separated list of absolute paths to the external trees, BR2_EXTERNAL_DIRS. Once we have all those variables, we replace references to BR2_EXTERNAL with either one of those. This cascades into how we display the list of defconfigs, so that it is easy to see what br2-external tree provides what defconfigs. As suggested by Arnout, tweak the comment from "User-provided configs" to "External configs", on the assumption that some br2-external trees could be provided by vendors, so not necessarily user-provided. Ditto the menu in Kconfig, changed from "User-provided options" to "External options". Now, when more than one br2-external tree is used, each gets its own sub-menu in the "User-provided options" menu. The sub-menu is labelled with that br2-external tree's name and the sub-menu's first item is a comment with the path to that br2-external tree. If there's only one br2-external tree, then there is no sub-menu; there is a single comment that contains the name and path to the br2-external tree. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Romain Naour <romain.naour@openwide.fr> Cc: Julien CORJON <corjon.j@ecagroup.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 16:39:20 +02:00
$(call list-defconfigs,$(TOPDIR))
$(foreach name,$(BR2_EXTERNAL_NAMES),\
$(call list-defconfigs,$(BR2_EXTERNAL_$(name)_PATH),\
$(BR2_EXTERNAL_$(name)_DESC))$(sep))
release: OUT = buildroot-$(BR2_VERSION)
# Create release tarballs. We need to fiddle a bit to add the generated
# documentation to the git output
release:
git archive --format=tar --prefix=$(OUT)/ HEAD > $(OUT).tar
$(MAKE) O=$(OUT) manual-html manual-text manual-pdf
$(MAKE) O=$(OUT) distclean
tar rf $(OUT).tar $(OUT)
gzip -9 -c < $(OUT).tar > $(OUT).tar.gz
bzip2 -9 -c < $(OUT).tar > $(OUT).tar.bz2
rm -rf $(OUT) $(OUT).tar
2009-01-15 20:36:06 +01:00
print-version:
@echo $(BR2_VERSION_FULL)
check-package:
find $(TOPDIR) -type f \( -name '*.mk' -o -name '*.hash' -o -name 'Config.*' \) \
-exec ./utils/check-package {} +
.PHONY: .gitlab-ci.yml
.gitlab-ci.yml: .gitlab-ci.yml.in
Makefile: offload .gitlab-ci.yml generation GitLab has severe limitations imposed to triggers. Using a variable in a regexp is not allowed: | only: | - /-$CI_JOB_NAME$/ | - /-\$CI_JOB_NAME$/ | - /-%CI_JOB_NAME%$/ Using the key 'variables' always lead to an AND with 'refs', so: | only: | refs: | - branches | - tags | variables: | - $CI_JOB_NAME == $CI_COMMIT_REF_NAME would make the push of a tag not to trigger all jobs anymore. Inheritance is used only for the second level of keys, so: |.runtime_test: &runtime_test | only: | - tags |tests.package.test_python_txaio.TestPythonPy2Txaio: | <<: *runtime_test | only: | - /-TestPythonPy2Txaio$/ would override the entire key 'only', making the push of a tag not to trigger all jobs anymore. So, in order to have a trigger per job and still allow the push of a tag to trigger all jobs (all this in a follow up patch), the regexp for each job must be hardcoded in the .gitlab-ci.yml and also the inherited values for key 'only' must be repeated for every job. This is not a big issue, .gitlab-ci.yml is already automatically generated from a template and there will be no need to hand-editing it when jobs are added or removed. Since the logic to generate the yaml file from the template will become more complex, move the commands from the main Makefile to a script. Using Python or other advanced scripting language for that script would be the most versatile solution, but that would bring another dependency on the host machine, pyyaml if Python is used. So every developer that needs to run 'make .gitlab-ci.yml' and also the docker image used in the GitLab pipelines would need to have pyyaml pre-installed. Instead of adding the mentioned dependency, keep using a bash script. While moving the commands to the script: - mimic the behavior of the previous make target and fail on any command that fails, by using 'set -e'; - break the original lines in one command per line, making the diff for any patch to be applied to this file to look nicer; - keep the script as simple as possible, without functions, just a script that executes from the top to bottom; - do not perform validations on the input parameters, any command that fails already makes the script to fail; - do not add an usage message, the script is not intended to be called directly. This patch does not change functionality. Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> [Thomas: make the script output on stdout rather than take the output file name as second argument.] Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-29 00:58:38 +01:00
./support/scripts/generate-gitlab-ci-yml $< > $@
include docs/manual/manual.mk
-include $(foreach dir,$(BR2_EXTERNAL_DIRS),$(sort $(wildcard $(dir)/docs/*/*.mk)))
.PHONY: $(noconfig_targets)
core: re-enter make if $(CURDIR) or $(O) are not canonical paths When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be resolved differently, depending on each package build-system (whether it uses the given paths or get the absolute canonical ones). Using absolute canonical paths will help achieving reproducible builds and will make easier tracking down host machine paths leaking into the host, target or staging trees. So, this change ensures the build takes place with the CURDIR and O variables are set to their absolute canonical paths. In order to recall the toplevel makefile with absolute canonical paths for $(CURDIR) and $(O), we need to: 1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will be passed to the sub-make. This is achieved using the 'realpath' make primitive. However, some care must be taken when manipulating O: - the out-of-tree makefile wrapper happens a trailing "/.", we need to strip this part away to not break the comparison driving the sub-make call; - the user can leave a trailing '/' to $(O); - according to [1,2], realpath returns an empty string in case of non-existing entry. So, to avoid passing an empty O= variable to sub-make, it is necessary to define the output directory and create it prior to call realpath on it (because on the first invocation, $(O) usually does not yet exists), hence the trick doing the mkdir right before calling realpath. 2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it when call recalling the top-level makefile with umask and paths correctly set. 3- Lastly, update the condition for setting the CONFIG_DIR and NEED_WRAPPER variables. Note: * This change takes care of the makefile wrapper installed in $(O) to avoid unneeded make recursion. [1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html [2] http://man7.org/linux/man-pages/man3/realpath.3.html Reported-by: Matthew Weber <matt@thewebers.ws> Cc: Matthew Weber <matt@thewebers.ws> Cc: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-17 23:05:43 +02:00
endif #umask / $(CURDIR) / $(O)