2010-03-10 00:11:58 +01:00
|
|
|
#
|
|
|
|
|
# Macro that builds the needed Makefile target to create a root
|
|
|
|
|
# filesystem image.
|
|
|
|
|
#
|
|
|
|
|
# The following variable must be defined before calling this macro
|
|
|
|
|
#
|
|
|
|
|
# ROOTFS_$(FSTYPE)_CMD, the command that generates the root
|
|
|
|
|
# filesystem image. A single command is allowed. The filename of the
|
|
|
|
|
# filesystem image that it must generate is $$@.
|
|
|
|
|
#
|
|
|
|
|
# The following variables can optionaly be defined
|
|
|
|
|
#
|
|
|
|
|
# ROOTFS_$(FSTYPE)_DEPENDENCIES, the list of dependencies needed to
|
|
|
|
|
# build the root filesystem (usually host tools)
|
|
|
|
|
#
|
|
|
|
|
# ROOTFS_$(FSTYPE)_PRE_GEN_HOOKS, a list of hooks to call before
|
|
|
|
|
# generating the filesystem image
|
|
|
|
|
#
|
2015-07-31 10:23:20 +02:00
|
|
|
# ROOTFS_$(FSTYPE)_POST_GEN_HOOKS, a list of hooks to call after
|
|
|
|
|
# generating the filesystem image
|
|
|
|
|
#
|
2010-03-10 00:11:58 +01:00
|
|
|
# In terms of configuration option, this macro assumes that the
|
|
|
|
|
# BR2_TARGET_ROOTFS_$(FSTYPE) config option allows to enable/disable
|
|
|
|
|
# the generation of a filesystem image of a particular type. If
|
2016-05-08 05:27:48 +02:00
|
|
|
# the configuration options BR2_TARGET_ROOTFS_$(FSTYPE)_GZIP,
|
2010-03-10 00:11:58 +01:00
|
|
|
# BR2_TARGET_ROOTFS_$(FSTYPE)_BZIP2 or
|
|
|
|
|
# BR2_TARGET_ROOTFS_$(FSTYPE)_LZMA exist and are enabled, then the
|
|
|
|
|
# macro will automatically generate a compressed filesystem image.
|
|
|
|
|
|
2017-11-12 18:45:47 +01:00
|
|
|
FS_DIR = $(BUILD_DIR)/buildroot-fs
|
2013-03-13 03:28:16 +01:00
|
|
|
ROOTFS_DEVICE_TABLES = $(call qstrip,$(BR2_ROOTFS_DEVICE_TABLE) \
|
2014-10-25 20:29:31 +02:00
|
|
|
$(BR2_ROOTFS_STATIC_DEVICE_TABLE))
|
2018-11-12 18:33:13 +01:00
|
|
|
|
2014-04-20 19:33:09 +02:00
|
|
|
ROOTFS_USERS_TABLES = $(call qstrip,$(BR2_ROOTFS_USERS_TABLES))
|
2010-03-10 00:11:58 +01:00
|
|
|
|
2018-12-03 22:05:06 +01:00
|
|
|
ROOTFS_FULL_DEVICES_TABLE = $(FS_DIR)/full_devices_table.txt
|
2018-11-12 18:33:13 +01:00
|
|
|
ROOTFS_FULL_USERS_TABLE = $(FS_DIR)/full_users_table.txt
|
|
|
|
|
|
2018-03-31 11:05:57 +02:00
|
|
|
ifeq ($(BR2_REPRODUCIBLE),y)
|
|
|
|
|
define ROOTFS_REPRODUCIBLE
|
|
|
|
|
find $(TARGET_DIR) -print0 | xargs -0 -r touch -hd @$(SOURCE_DATE_EPOCH)
|
|
|
|
|
endef
|
|
|
|
|
endif
|
|
|
|
|
|
2018-03-31 11:05:56 +02:00
|
|
|
ROOTFS_COMMON_DEPENDENCIES = \
|
|
|
|
|
host-fakeroot host-makedevs \
|
fs/common: always depend on build host-tar if needed
Currently, the filesystems do not depend on building host-tar when it
is needed, even though all of them have to extract the intermediate
tarball.
However, in degenerate (but legally valid) configurations with no
user-selectable package selected, host-tar would not be built, so the
rootfs images would use whatever improper tar the system has.
Add the conditional dependency to host-tar to the rootfs-common
intermediate image. Since this is the internal step that all real rootfs
generators depend on, they now properly depend on host-tar when needed.
In practice, when host-tar is needed, it will always be built before the
rootfs images, because it is a dependency of all packages (except a very
few, like the skeleton), of which host-fakeroot, which is a mandatory
dependency of rootfs-comon anyway. But for consistency sake, let's
explicitly add host-tar as a dependency to rootfs-common too.
Note that rootfs-tar already had that dependency, and we leave it as-is
because it is semantically correct, even if superfluous.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-08 22:40:20 +02:00
|
|
|
$(BR2_TAR_HOST_DEPENDENCY) \
|
2018-03-31 11:05:56 +02:00
|
|
|
$(if $(PACKAGES_USERS)$(ROOTFS_USERS_TABLES),host-mkpasswd)
|
|
|
|
|
|
fs: don't use an intermediate tarball
Since 118534fe54b (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-12 18:33:11 +01:00
|
|
|
.PHONY: rootfs-common
|
|
|
|
|
rootfs-common: $(ROOTFS_COMMON_DEPENDENCIES) target-finalize
|
|
|
|
|
@$(call MESSAGE,"Generating root filesystems common tables")
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
rm -rf $(FS_DIR)
|
|
|
|
|
mkdir -p $(FS_DIR)
|
2018-10-03 01:02:20 +02:00
|
|
|
|
2018-11-12 18:33:13 +01:00
|
|
|
$(call PRINTF,$(PACKAGES_USERS)) >> $(ROOTFS_FULL_USERS_TABLE)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
ifneq ($(ROOTFS_USERS_TABLES),)
|
2018-11-12 18:33:13 +01:00
|
|
|
cat $(ROOTFS_USERS_TABLES) >> $(ROOTFS_FULL_USERS_TABLE)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
endif
|
2018-12-03 22:05:07 +01:00
|
|
|
|
|
|
|
|
$(call PRINTF,$(PACKAGES_PERMISSIONS_TABLE)) > $(ROOTFS_FULL_DEVICES_TABLE)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
ifneq ($(ROOTFS_DEVICE_TABLES),)
|
2018-12-03 22:05:07 +01:00
|
|
|
cat $(ROOTFS_DEVICE_TABLES) >> $(ROOTFS_FULL_DEVICES_TABLE)
|
2018-12-03 22:05:08 +01:00
|
|
|
endif
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
ifeq ($(BR2_ROOTFS_DEVICE_CREATION_STATIC),y)
|
2018-12-03 22:05:06 +01:00
|
|
|
$(call PRINTF,$(PACKAGES_DEVICES_TABLE)) >> $(ROOTFS_FULL_DEVICES_TABLE)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
endif
|
2018-03-31 11:05:56 +02:00
|
|
|
|
|
|
|
|
rootfs-common-show-depends:
|
|
|
|
|
@echo $(ROOTFS_COMMON_DEPENDENCIES)
|
|
|
|
|
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 21:12:24 +02:00
|
|
|
# Since this function will be called from within an $(eval ...)
|
|
|
|
|
# all variable references except the arguments must be $$-quoted.
|
2017-12-28 11:40:32 +01:00
|
|
|
define inner-rootfs
|
2010-03-10 00:11:58 +01:00
|
|
|
|
2018-12-01 10:14:36 +01:00
|
|
|
ROOTFS_$(2)_IMAGE_NAME ?= rootfs.$(1)
|
|
|
|
|
ROOTFS_$(2)_FINAL_IMAGE_NAME = $$(strip $$(ROOTFS_$(2)_IMAGE_NAME))
|
2018-03-31 11:05:52 +02:00
|
|
|
ROOTFS_$(2)_DIR = $$(FS_DIR)/$(1)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
ROOTFS_$(2)_TARGET_DIR = $$(ROOTFS_$(2)_DIR)/target
|
2018-03-31 11:05:52 +02:00
|
|
|
|
2018-03-31 11:05:56 +02:00
|
|
|
ROOTFS_$(2)_DEPENDENCIES += rootfs-common
|
2013-11-07 00:12:32 +01:00
|
|
|
|
|
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y)
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .gz
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = gzip -9 -c
|
|
|
|
|
endif
|
|
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_BZIP2),y)
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .bz2
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = bzip2 -9 -c
|
|
|
|
|
endif
|
|
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_LZMA),y)
|
|
|
|
|
ROOTFS_$(2)_DEPENDENCIES += host-lzma
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .lzma
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = $$(LZMA) -9 -c
|
|
|
|
|
endif
|
2017-10-13 21:16:52 +02:00
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_LZ4),y)
|
|
|
|
|
ROOTFS_$(2)_DEPENDENCIES += host-lz4
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .lz4
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = lz4 -l -9 -c
|
|
|
|
|
endif
|
2013-11-07 00:12:32 +01:00
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_LZO),y)
|
|
|
|
|
ROOTFS_$(2)_DEPENDENCIES += host-lzop
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .lzo
|
|
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = $$(LZOP) -9 -c
|
|
|
|
|
endif
|
|
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)_XZ),y)
|
2015-03-09 10:34:13 +01:00
|
|
|
ROOTFS_$(2)_DEPENDENCIES += host-xz
|
2013-11-07 00:12:32 +01:00
|
|
|
ROOTFS_$(2)_COMPRESS_EXT = .xz
|
2014-06-04 08:33:17 +02:00
|
|
|
ROOTFS_$(2)_COMPRESS_CMD = xz -9 -C crc32 -c
|
2013-11-07 00:12:32 +01:00
|
|
|
endif
|
2010-11-19 15:05:20 +01:00
|
|
|
|
2018-12-01 10:14:36 +01:00
|
|
|
$$(BINARIES_DIR)/$$(ROOTFS_$(2)_FINAL_IMAGE_NAME): ROOTFS=$(2)
|
|
|
|
|
$$(BINARIES_DIR)/$$(ROOTFS_$(2)_FINAL_IMAGE_NAME): FAKEROOT_SCRIPT=$$(ROOTFS_$(2)_DIR)/fakeroot
|
|
|
|
|
$$(BINARIES_DIR)/$$(ROOTFS_$(2)_FINAL_IMAGE_NAME): $$(ROOTFS_$(2)_DEPENDENCIES)
|
|
|
|
|
@$$(call MESSAGE,"Generating filesystem image $$(ROOTFS_$(2)_FINAL_IMAGE_NAME)")
|
Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.
For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.
This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.
Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.
Thanks to those changes, the global "dirs" target is entirely removed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-23 15:58:08 +01:00
|
|
|
mkdir -p $$(@D)
|
fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:05:58 +02:00
|
|
|
rm -rf $$(ROOTFS_$(2)_DIR)
|
|
|
|
|
mkdir -p $$(ROOTFS_$(2)_DIR)
|
fs: don't use an intermediate tarball
Since 118534fe54b (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-12 18:33:11 +01:00
|
|
|
rsync -auH \
|
|
|
|
|
--exclude=/$$(notdir $$(TARGET_DIR_WARNING_FILE)) \
|
|
|
|
|
$$(BASE_TARGET_DIR)/ \
|
|
|
|
|
$$(TARGET_DIR)
|
|
|
|
|
|
2016-06-07 22:09:41 +02:00
|
|
|
echo '#!/bin/sh' > $$(FAKEROOT_SCRIPT)
|
2015-02-24 16:47:34 +01:00
|
|
|
echo "set -e" >> $$(FAKEROOT_SCRIPT)
|
fs: don't use an intermediate tarball
Since 118534fe54b (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-12 18:33:11 +01:00
|
|
|
|
|
|
|
|
echo "chown -h -R 0:0 $$(TARGET_DIR)" >> $$(FAKEROOT_SCRIPT)
|
2018-11-12 18:33:13 +01:00
|
|
|
PATH=$$(BR_PATH) $$(TOPDIR)/support/scripts/mkusers $$(ROOTFS_FULL_USERS_TABLE) $$(TARGET_DIR) >> $$(FAKEROOT_SCRIPT)
|
2018-12-03 22:05:06 +01:00
|
|
|
echo "$$(HOST_DIR)/bin/makedevs -d $$(ROOTFS_FULL_DEVICES_TABLE) $$(TARGET_DIR)" >> $$(FAKEROOT_SCRIPT)
|
fs: don't use an intermediate tarball
Since 118534fe54b (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-12 18:33:11 +01:00
|
|
|
$$(foreach s,$$(call qstrip,$$(BR2_ROOTFS_POST_FAKEROOT_SCRIPT)),\
|
|
|
|
|
echo "echo '$$(TERM_BOLD)>>> Executing fakeroot script $$(s)$$(TERM_RESET)'" >> $$(FAKEROOT_SCRIPT); \
|
|
|
|
|
echo $$(EXTRA_ENV) $$(s) $$(TARGET_DIR) $$(BR2_ROOTFS_POST_SCRIPT_ARGS) >> $$(FAKEROOT_SCRIPT)$$(sep))
|
|
|
|
|
$$(foreach hook,$$(ROOTFS_PRE_CMD_HOOKS),\
|
|
|
|
|
$$(call PRINTF,$$($$(hook))) >> $$(FAKEROOT_SCRIPT)$$(sep))
|
|
|
|
|
|
2018-03-31 11:05:47 +02:00
|
|
|
$$(foreach hook,$$(ROOTFS_$(2)_PRE_GEN_HOOKS),\
|
|
|
|
|
$$(call PRINTF,$$($$(hook))) >> $$(FAKEROOT_SCRIPT)$$(sep))
|
2018-03-31 11:05:57 +02:00
|
|
|
$$(call PRINTF,$$(ROOTFS_REPRODUCIBLE)) >> $$(FAKEROOT_SCRIPT)
|
2016-06-06 22:43:39 +02:00
|
|
|
$$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)
|
2013-01-20 15:03:13 +01:00
|
|
|
chmod a+x $$(FAKEROOT_SCRIPT)
|
2019-04-03 22:13:25 +02:00
|
|
|
PATH=$$(BR_PATH) FAKEROOTDONTTRYCHOWN=1 $$(HOST_DIR)/bin/fakeroot -- $$(FAKEROOT_SCRIPT)
|
fs: remove intermediate artefacts
Each of the intermediate, per-rootfs target directories, as well as the
intermediate tarball, can take quite some place, and is mostly a
duplication of what's already in target/. The only delta, if any, would
be the tweaks made by the filesystem image generations, but those tweaks
are most probably only meaningful when seen as root.
We normally do not remove intermediate files, but those can be quite
large, and are not directly usable by, nor accessible to the user.
So, get rid of them once the filesystem has been generated.
This does not need to be done in fakeroot.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 11:06:01 +02:00
|
|
|
$(Q)rm -rf $$(TARGET_DIR)
|
2013-11-07 00:12:32 +01:00
|
|
|
ifneq ($$(ROOTFS_$(2)_COMPRESS_CMD),)
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 21:12:24 +02:00
|
|
|
PATH=$$(BR_PATH) $$(ROOTFS_$(2)_COMPRESS_CMD) $$@ > $$@$$(ROOTFS_$(2)_COMPRESS_EXT)
|
2013-01-22 02:03:43 +01:00
|
|
|
endif
|
2015-07-31 10:23:20 +02:00
|
|
|
$$(foreach hook,$$(ROOTFS_$(2)_POST_GEN_HOOKS),$$(call $$(hook))$$(sep))
|
2010-03-10 00:11:58 +01:00
|
|
|
|
2010-11-19 15:01:13 +01:00
|
|
|
rootfs-$(1)-show-depends:
|
2013-01-20 15:03:13 +01:00
|
|
|
@echo $$(ROOTFS_$(2)_DEPENDENCIES)
|
2010-05-13 19:20:21 +02:00
|
|
|
|
2018-12-01 10:14:36 +01:00
|
|
|
rootfs-$(1): $$(BINARIES_DIR)/$$(ROOTFS_$(2)_FINAL_IMAGE_NAME)
|
2010-03-10 00:11:58 +01:00
|
|
|
|
2015-04-12 18:37:46 +02:00
|
|
|
.PHONY: rootfs-$(1) rootfs-$(1)-show-depends
|
|
|
|
|
|
2010-03-10 00:11:58 +01:00
|
|
|
ifeq ($$(BR2_TARGET_ROOTFS_$(2)),y)
|
2014-02-14 10:55:06 +01:00
|
|
|
TARGETS_ROOTFS += rootfs-$(1)
|
2018-11-15 16:25:41 +01:00
|
|
|
PACKAGES += $$(filter-out rootfs-%,$$(ROOTFS_$(2)_DEPENDENCIES) $$(ROOTFS_COMMON_DEPENDENCIES))
|
2010-03-10 00:11:58 +01:00
|
|
|
endif
|
fs/common: get rid of post-target rules
The only users of post-target rules were ext2, cpio and initramfs.
Of those, ext2 and cpio were changed to use post-gen hooks, while
initramfs was not even using the generic rootfs infra and was fixed
to no longer reference post-target rules.
Besides, the comment in the infra was really misleading: it referenced
initramfs implying it was the sole user of that feature, even though
initramfs was not using the fs infra.
Furthermore, using post-target rules was inherently broken for top-level
parallel builds, because filesystems had to ensure the ordering by
themselves. Of the two real users of post-target rules (cpio and ext2),
one did enforce rules ordering (apparently correctly), while the other
forgot to do so.
We can get rid of post-target rules altogether, now.
Add a legacy check, to catch out-of-tree (e.g. br2-external) users of
post-target rules, and instruct them to switch to post-gen hooks instead.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-12 18:45:45 +01:00
|
|
|
|
|
|
|
|
# Check for legacy POST_TARGETS rules
|
|
|
|
|
ifneq ($$(ROOTFS_$(2)_POST_TARGETS),)
|
|
|
|
|
$$(error Filesystem $(1) uses post-target rules, which are no longer supported.\
|
|
|
|
|
Update $(1) to use post-gen hooks instead)
|
|
|
|
|
endif
|
|
|
|
|
|
2010-03-10 00:11:58 +01:00
|
|
|
endef
|
|
|
|
|
|
2017-12-28 11:40:32 +01:00
|
|
|
# $(pkgname) also works well to return the filesystem name
|
|
|
|
|
rootfs = $(call inner-rootfs,$(pkgname),$(call UPPERCASE,$(pkgname)))
|
2010-03-10 22:30:06 +01:00
|
|
|
|
2013-09-03 10:45:41 +02:00
|
|
|
include $(sort $(wildcard fs/*/*.mk))
|
