When multiple hooks are registred, both pre-a and post-command hooks'
foreach loops need to have a separator at the end in order for the
code to work as intended. Without the separator all hooks end up as a
one single line command thus making all but the first hook into
no-ops.
Fixes: 4628b6f3b4 ("fs: add pre- and post-command hooks")
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/
For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.
One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.
Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.
The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.
We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.
So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.
Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, when there is no syztem device table (permissions or static
devices) defined, then package permissions are not applied, because they
are guarded by the check on the system device tables being non empty.
Fix that by narrowing the guarding condition.
Note that the dependency on host-makedevs was not conditional; we always
build it even if we don't need it. Making it conditional is not
possible, because we don't know all the packages permissions by the time
the fs infra is parsed (packages from br2-external are parsed after it).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This reverts commit 8035ceb56c.
Moving to pseudo brought a number of issues (and longer compilation time),
so lets stick with fakeroot now that the reported ubi issue has been worked
around.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Make sure all files in $TARGET_DIR have a defined modification time
before to generate filesystems.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
fakeroot misbhaves when there are SElinux contexts, as reported in #9386,
and further detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1238802
A proposal in the FC bug is to use pseudo instead of fakeroot. Pseudo is
a from-scratch re-implementation of fakeroot; it is used in Poky/OE
instead of fakeroot.
Fixes#9386.
Reported-by: Andrey Yurovsky <yurovsky@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Andrey Yurovsky <yurovsky@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some users have the need to be able to tweak the content of the target
rootfs with root-like rights, that is, from inside the fakeroot script.
Add a new system option to allow those users to provide a list of
scripts, like the post-build and post-image scripts, that will be run
from our fakeroot script.
[Peter: pass TARGET_DIR to scripts, tweak help text]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Cam Hutchison <camh@xdna.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We directly running this script, so it should start with a sha-bang (not
sure why/how it works today...).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use the newly-introduced PRINTF macro to generate printf formats
that do "The Right Thing (TM)".
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixed comment typos as to improve clarity.
Signed-off-by: Brian 'redbeard' Harrington <redbeard@coreos.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We will need the users and groups to get defined before we can use them
from makedevs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The common filesystem infrastructure already supports a
<fs>_PRE_GEN_HOOKS variable, which allows filesystem makefiles to
register some actions to be done before the root filesystem image is
generated.
This commit adds a similiar <fs>_POST_GEN_HOOKS variable, which will
allow filesystem makefiles to do some actions after the filesystem
image has been generated. It will initially be used by the iso9660
filesystem to delete the temporary directory it creates.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Noé Rubinstein <nrubinstein@aldebaran.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The code in fs/common.mk properly triggers the build of host-mkpasswd
when at least one package specifies a <pkg>_USERS variable. However,
when no selected package specifies a <pkg>_USERS variable but the user
uses a custom users table through BR2_ROOTFS_USERS_TABLES, then we
forget to build host-mkpasswd, leading to build failures if you don't
have mkpasswd installed in your machine.
This commit fixes that by taking into account both the package users
table and the custom users table when deciding to depend or not on
host-mkpasswd.
Reported-by: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Commit 4570dbcb14
("4570dbcb14b394d8f3c97baf4c9e5f4f225d1d37") added the dependencies of
rootfs building logic to PACKAGES, only thinking about the case where
rootfs building logic actually depends on real, normal packages
(host-mtd, host-genext2fs, etc.). But some file systems (rootfs-ubi)
depend on other filesystem images (rootfs-ubifs). And such targets
should not be added to PACKAGES, otherwise an incorrect circular
dependency is created.
This commit fixes that by only adding the rootfs building logic
dependencies that do *not* start with rootfs- to the global PACKAGES
variable.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
The logic for creating most of the filesystem images requires a
certain number of host packages to be built. However, those packages
are not currently listed in the global PACKAGES variables, and they
are not dependencies of any other package listed in the PACKAGES
variable.
While it does not have any practical implications, it makes sense to
have those packages listed in the global PACKAGES variable, which this
commit implements.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
This commit improves the filesystem handling code to declare its
various targets as PHONY when appropriate.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
When a filesystem image is xz-compressed, the XZ variable is used to refer
to the xz compression tool, but it is not necessarily available. Add a
proper dependency to fix that.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the generated fakeroot script has no error checking causing
make to continue building even if some of the fakeroot script commands
have failed. This can cause e.g. using an invalid device tables to go
unnoticed.
So add a "set -e" to the start of the fakeroot script so it will exit
with a failure code as soon as one of the script commands fails.
Signed-off-by: Stefan Sørensen <stefan.sorensen@spectralink.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the symlinks in the generated filesystems will have the
UID of the user running the build, because 'chown' does not change
the ownership of symlinks, by default.
Although the implications are limited, some may not want that UID
to leak in the generated filesystems.
So, use 'chown -h' so even symlinks get properly chowned.
Reported-by: Angelo Dureghello <angelo@barix.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Closes#7160
Since f46f81de13 (Don't build host-xz needlessly), host-xz is no longer
unconditionally built when xz fs compression is requested, so don't
explicitly refer to it.
Instead, rely on our check-host-xzcat logic to ensure xz is available and
set the path to ensure our host variant gets picked up if not available on
the build machine.
While we're at it, get rid of the now unused XZ variable.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
export PATH=$(BR_PATH) so that the fakerooted script uses our
own tools before any of the host-system ones.
Reported-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Export PATH=$(BR_PATH) so that our own mkpasswd is found before
any existing host-system one.
Reported-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Tested-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A new entry has been added to the "System Configuration" menu to allow
the user to set the location of additional user tables (besides the ones
defined in packages).
A user table is a text file, formatted using the mkusers syntax, which
describes the users on the target system, with their UID/GID, home
directory, password, etc.
The target root file system will be populated according the content of
these files.
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
[yann.morin.1998@free.fr: use plural TABLES; we need to remove the
intermediate users_table file, as it is no longer generated in
one shot, in case a previous run failed and did not remove it]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit d1f325f554 (xzcat: treat as host prerequisite and build if
needed) host-xz is always built when the host does not have xz installed.
Removed all other host-xz dependencies.
[Peter: don't drop for host-squashfs, as that needs libxz development files]
Cc: Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To be able to use top-level parallel make we must not depend in a rule
on the order of evaluation of the prerequisites, so instead of relyng on
the left to right ordering of evaluation of the prerequisites add an
explicit rule to describe the dependencies.
Add explicit rules to describe the following dependency chain:
$(TARGETS) -> target-finalize -> rootfs-* -> target-post-image
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
If the Buildroot tree is read-only, then $(TARGET_DIR_WARNING_FILE) is
copied read-only into target/ but we may want to remove it during the
build process.
This poses no real problem, since target/ itself is guaranteed to be
writable, but for good measure, force $(TARGET_DIR_WARNING_FILE) to be
writable itself.
Reported-by: Danomi Manchego <danomimanchego123@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When using a custom rootfs skeleton, the host-mkpasswd target is not
automatically built, even if some packages add users (i.e. when
PACKAGES_USERS is not empty).
In this case, the rootfs generation may fail because the mkpasswd
provided by some distribution does not support all options used in the
mkusers script.
This patch avoids such trouble by automatically adding the host-mkpasswd
package to the dependency list of the rootfs targets if users needs
to be created.
[Peter: minor rewording of commit message]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This makes the compression extension available in a variable, so it
can be used by the fs-specific commands. In this patch, it is used
by iso9660. Following patches show more use cases.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This will allow us to remove the unused ROOTFS_$(FSTYPE)_POST_GEN_HOOKS.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Make 3.82 no longer sort the result of wildcards (see
http://comments.gmane.org/gmane.comp.gnu.make.bugs/4260). This may break
build reproducibility.
This patch sort results of wildcards to ensure reproducibility.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Packages that install daemons may need those daemons to run as a non-root,
or an otherwise non-system (eg. 'daemon'), user.
Add infrastructure for packages to create users, by declaring the FOO_USERS
variable that contain a makedev-syntax-like description of the user(s) to
add.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Cam Hutchison <camh@xdna.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Add support for LZO and XZ compression methods to cpio, ext2, tar and
ubifs filesystem targets.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Since ROOTFS_DEVICE_TABLES is a concatenation of two strings separated by
spaces, there will always be a space in it, which means it's never empty.
Therefore, when testing for empty, the condition never evaluate to false.
The following change fixes this problem; it runs qstrip on the overall
combination of the variables, causing the space to be removed if it's the
only thing left.
Tested-by: Ezequiel Garcia <ezequiel.garcia@free-electrons.com>
Signed-off-by: Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This makes things easier to understand and more consistent with the pkg-infra.
For example, it removes the need for '$$@' in the CMD variables of fs/*/*.mk.
It also makes it possible to update the variables later, e.g. in the package
override file.
It also makes sure that the date will be recorded correctly in Yann E. Morin's
patch that logs the MESSAGE macros to a file.
The fs/*/*.mk must be updated as well because the '$@' shouldn't be quoted
anymore in the CMD variables or the hooks.
The $(eval ...) for the dependencies is redundant, because the $(ROOTFS_TARGET)
variable is already eval'd. Note that it is only redundant if the evaluation of
the uses of the variable is also delayed.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
A very common mistake done by our users is that they use
output/target/ directory as their root filesystem. Even though this is
loudly documented in our Buildroot manual, people don't read
documentation, so it is not sufficient.
This patch adds a text file named
output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM which explains why
output/target isn't appropriate to use as the root filesystem. The
process is:
* At the beginning of the build, right after the skeleton has been
copied, support/misc/target-dir-warning.txt is copied to
output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
* In the filesystem images creation code, this file is removed before
launching fakeroot, and restored right after that, so that this
file is not present in the generated root filesystem images.
Note that the file has not been added to the default skeleton for two
reasons:
* It would have annoying to have in our source tree a file named in
capital letters inside system/skeleton/
* The proposed way works even if the user uses a custom skeleton.
[Peter: fixed typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Juha Lumme <juha.lumme@gmail.com>
Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
echo -e is not a portable way to do this, better use printf.
Works with MacOS X.
Signed-off-by: Waldemar Brodkorb <mail@waldemar-brodkorb.de>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
There used to be a mechanism using which packages could leave a
.fakeroot.<something> file which could contain commands to be executed
within the fakeroot environment. Since this mechanism is no longer
used by any package, remove it from the common infrastructure.
The latest user was nfs-utils, which used this mechanism to do the
"make install" as root, since doing otherwise was not supported. But
since 16e7b8255c, nfs-utils has been
upgraded and converted to the package infrastructure, and this hack is
no longer necessary. Another past user was the ltp-testsuite package,
for the same reason, and since
a72a670489, the fakeroot hack is no
longer needed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Add a way for packages to declare files they need instead of relying
only on device tables, which creates files no matter if the package is
indeed enabled, as we can see for busybox.
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
As discussed here:
http://lists.busybox.net/pipermail/buildroot/2011-May/043251.html
Add BR2_ROOTFS_STATIC_DEVICE_TABLE for the extra device table file(s)
to create device nodes in /dev, rather than complicated logic in
BR2_ROOTFS_DEVICE_TABLE, making it complicated to move between static
and dynamic modes.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This allows to have a device table for all directories/files and
another device table for the device files themselves. Both are needed
for static /dev, but only the first one is needed when
devtmpfs/mdev/udev are used.
We take this opportunity to move the documentation of the device table
format in a common location, package/makedevs/README.
[Peter: simplify code slightly, fix indentation]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Use rootfs-* rather than *-root, to match the convention used under
package/ and which fits with the ROOTFS_*_ variables.
This will also help with the host dependencies.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
