Commit Graph

88 Commits

Author SHA1 Message Date
Peter Korsgaard
55a098dc77 fs/common.mk: disable real chown calls in fakeroot
fakeroot by default forwards {f,l,}chown calls to libc and ignores
permission issues, which may cause issues when building in restricted
environments like user namespaces as set up with bubblewrap where a chown
call with a uid/gid not mapped in the user namespace instead returns EINVAL.
This error is not masked by fakeroot and returned to the caller, causing
failures.

There is no real reason to really perform the *chown calls in the context of
Buildroot (as the calls will likely just fail and files are not accessed
outside the fakeroot environment any way).

This forwarding can be disabled by setting the FAKEROOTDONTTRYCHOWN
environment variable, so set it when fakeroot is executed.

Reported-by: Esben Nielsen <nielsen.esben@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 655acd1df0)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-04-14 22:23:37 +02:00
Thomas Petazzoni
ac3b476bf5 fs/common.mk: make sure that static devices from packages are created
The static devices defined by packages are currently added to the full
device table when two conditions are met:

 (1) ROOTFS_DEVICE_TABLES is non-empty
 (2) BR2_ROOTFS_DEVICE_CREATION_STATIC=y

(2) is obviously correct. However, depending on (1) is not correct: if
the user doesn't provide any custom permission table and custom device
table, then ROOTFS_DEVICE_TABLES will be empty.

So instead, move the addition of the package-defined static devices
outside of condition (1), and have it only under condition (2).

Reported-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-12-04 21:53:14 +01:00
Yann E. MORIN
b67f0a99a9 fs/common.mk: allow user provided permissions to override packages permissions
Reported-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-12-04 21:27:01 +01:00
Thomas Petazzoni
93434aedaa fs/common.mk: rename FULL_DEVICE_TABLE to ROOTFS_FULL_DEVICES_TABLE
In commit 6b50f988ad ("fs/common.mk:
rename internal variable"), USERS_TABLE was renamed to
ROOTFS_FULL_USERS_TABLE.

This commit follows the same direction by renaming the
FULL_DEVICE_TABLE variable to ROOTFS_FULL_DEVICE_TABLE.

In addition, for consistency, the file itself is renamed
full_device_table.txt.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-12-04 21:26:39 +01:00
Yann E. MORIN
6b50f988ad fs/common.mk: rename internal variable
In preparation of more renames, rename the variable that points to the
final users table.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: as suggested by Arnout, use ROOTFS_FULL_USERS_TABLE instead
of ROOTFS_FINAL_USERS_TABLE.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 21:30:12 +01:00
Peter Korsgaard
13c43455a0 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-02 08:16:10 +01:00
Carlos Santos
21c6fb2589 fs/common: allow filesystems to set the name of their output file
Some filesystems may want to tweak their output names, rather than using
the fixed "rootfs.foo" scheme. Add a ROOTFS_FOO_IMAGE_NAME variable for
this purpose.

Signed-off-by: Carlos Santos <casantos@datacom.com.br>
[yann.morin.1998@free.fr: fix the patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-01 22:53:45 +01:00
Thomas Petazzoni
d0f4f95e39 Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.

For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.

This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.

Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.

Thanks to those changes, the global "dirs" target is entirely removed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:09:46 +01:00
Serj Kalichev
305e4487e5 fs/common.mk: Fix show-build-order
The command "make show-build-order" doesn't show dependencies of rootfs-common target.

This patch adds $(ROOTFS_COMMON_DEPENDENCIES) to PACKAGES variable.

Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-18 09:46:52 +01:00
Yann E. MORIN
409d4c3fe9 fs: don't use an intermediate tarball
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.

However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.

Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.

Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:

    File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring

This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.

The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.

The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.

So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216

Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.

[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-13 00:47:36 +01:00
Yann E. MORIN
89c92487cf fs/common: always depend on build host-tar if needed
Currently, the filesystems do not depend on building host-tar when it
is needed, even though all of them have to extract the intermediate
tarball.

However, in degenerate (but legally valid) configurations with no
user-selectable package selected, host-tar would not be built, so the
rootfs images would use whatever improper tar the system has.

Add the conditional dependency to host-tar to the rootfs-common
intermediate image. Since this is the internal step that all real rootfs
generators depend on, they now properly depend on host-tar when needed.

In practice, when host-tar is needed, it will always be built before the
rootfs images, because it is a dependency of all packages (except a very
few, like the skeleton), of which host-fakeroot, which is a mandatory
dependency of rootfs-comon anyway. But for consistency sake, let's
explicitly add host-tar as a dependency to rootfs-common too.

Note that rootfs-tar already had that dependency, and we leave it as-is
because it is semantically correct, even if superfluous.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-01 22:01:46 +01:00
Matt Weber
c3edec0018 fs/common: allow custom user table to override package-defined users
Currently, when a custom user table and a package define the same user,
the settings from the package takes precedence over the ones from the
custom user table.

However, it makes sense to allow the settings from the custom user table
take precedence. For example, it would allow redirecting the user's
home directory to an alternate location (e.g. away from tmp and into a
partition that is persistent).

The support/scripts/mkusers script will only retain settings from the
latest definition it finds.

Thus, by passing the custom user table after the package defined users,
it is possible to override the package provided user definitions.

Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-10-20 14:33:24 +01:00
Peter Korsgaard
065e7672e1 fs: ensure hard links in TARGET_DIR are correctly copied for filesystem input
Fixes #11046

Commit bb2a57a17a (fs: run packages' filesystem hooks in a copy of target/)
changed the file system logic to run file system hooks from packages on a
copy of TARGET_DIR, and finally use this copy as input for the file system
generation.

This copy was done with rsync, which by default does not preserve hard
links, leading to an expansion of the file system images when hard links are
present.

Fix it by passing the -H option to rsync (preserve hard links).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-05-27 23:46:29 +02:00
Yann E. MORIN
543107d390 fs: remove intermediate artefacts
Each of the intermediate, per-rootfs target directories, as well as the
intermediate tarball, can take quite some place, and is mostly a
duplication of what's already in target/. The only delta, if any, would
be the tweaks made by the filesystem image generations, but those tweaks
are most probably only meaningful when seen as root.

We normally do not remove intermediate files, but those can be quite
large, and are not directly usable by, nor accessible to the user.
So, get rid of them once the filesystem has been generated.

This does not need to be done in fakeroot.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
fcdf58cad1 fs: get rid of package-provided post-fs hooks
Now that the pre-fs ones are run on a transient copy of target/, the
post-fs hooks are no longer needed because we no longer need to restore
the target/ directory as it is only a internal copy.

Remove support for the post-fs hooks, and update the sole package using
them.

We do not add a legacy check because this was mostly a purely-internal
detail that was never really exposed nor documented.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
bb2a57a17a fs: run packages' filesystem hooks in a copy of target/
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.

However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.

We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.

This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
118534fe54 fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:

  - the content of the target directory for a specific filesystems may
    depend on whether another filesystem is enabled or not; for example,
    cpio will create a /init script or symlink and a /dev/console node;

  - the filesystems can not be built in parallel, because they may change
    the content of the target directory while another is being assembled.

Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...

We introduce an intermediate tarball, for which we do the full fakeroot
shebang.

That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.

Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
31fd0b1cac fs: introduce a macro for reproducible command
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
98d1e7f928 fs: new intermediate rootfs-common to gather common dependencies
Before we can create an intermediate tarball for all filesystems, we
nedd to move the common dependencies needed to generate that
intermediate tarball, rather than leave those dependencies to each
filesystem.

So, we introduce rootfs-common, which gathers all those common
dependencies.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
c6e425729e fs: introduce per-rootfs TARGET_DIR variable
... which for now still points to the base target directory, but this is
a step forward.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
2a6c55de49 fs: split per-rootfs dependency line
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
4815240830 fs: use a per-rootfs fakeroot script
... and locate that script in a per-rootfs directory.

Just like for ROOTFS, this variable will leak down the dependency tree to
target-finalize and packages - But it doesn't matter as it isn't used
outside fs/.

[Peter: extend commit message]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
0c7adf75d2 fs: define per-rootfs directory
This will serve in future commits to store pre-rootfs files, like
fakeroot script...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
2765973e01 fs: set per-rootfs variable name
Like we do for packages with the PKG variable, set ROOTFS to contain the
upper-case name of the rootfs currently being generated.

This will be useful in later patches, when we need more per-rootfs
variables, like a per-rootfs TARGET_DIR for example.

In Makefiles, per-rule variables trickle down the dependency chain, to
all dependencies of that rule, so we have to stop ROOTFS as soon as
we're not in a rootfs. This means we have to stop it at target-finalize
(which is a dependency of all filesystems), and for each package
individually, since some packages (host or target) can be direct
dependencies of filesystems as well.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:52:52 +02:00
Yann E. MORIN
405f912004 fs: run filesystem hooks under fakeroot
Currently, the pre-gen hooks of the various filesystems are run before
we enter the fakeroot.

However, this precludes those hooks from doing actions that require
root, like creating a pseudo-device or the likes.

So, move those pre-gen hooks under fakeroot.

This has currently no side-effect, as they are still called before
everything else in the fakeroot script, even the system-wide chown call.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:47:25 +02:00
Yann E. MORIN
4b3d1b36a3 fs: pass EXTRA_ENV to post-fakeroot script
Like we do for post-build and post-image scripts, pass EXTRA_ENV to
post-fakeroot script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-15 21:37:39 +01:00
Peter Korsgaard
51fa8d8e28 fs/common.mk: support lz4 compression
Similar to the other compressors.  Notice that we use the -l (legacy format)
for Linux kernel initrd compatibility.

Lz4 decompression is supported by the Linux kernel since 3.11.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:43:58 +01:00
Yann E. MORIN
b6842c4097 fs: make it behave a bit more like the package infra
Currently, to register a filesystem, one has to call:

    $(eval $(call ROOTFS_TARGET,blabla))

This is very unlike the package infrastructure, where the name of the
package is automatically guessed by the infra.

It turns out that we can now do that for the filesystem infra too.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-03 21:58:07 +01:00
Yann E. MORIN
423b01ecc2 fs: prepare temp directory before running PRE_GEN hooks
Some filesystems have PRE_GEN hooks that create a directory structure
under the temporary directory.

For example, iso9660 will create a sub-directory where it stores the
kernel (in case of initramfs or initrd).

So, we must run the PRE_GEN hooks after we cleanup/create the temporary
directory.

Fixes:
    https://gitlab.com/buildroot.org/buildroot/-/jobs/42835965
    https://gitlab.com/buildroot.org/buildroot/-/jobs/42835967

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-02 14:37:53 +01:00
Yann E. MORIN
787a31fed4 fs: don't remove intermediate files
Now that they are in their own directory and no longer pollute the build
dir, there is no point in removing them.

Furthermore, a follow-up patch will require that those files survive
when more than one filesystem image is generated.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 15:50:53 +01:00
Yann E. MORIN
917508e6bc fs: don't pollute $(BUILD_DIR) with temp files
Currently, we create a bunch of temporary files in $(BUILD_DIR), while
assembling the filesystem images.

Move those files to their own sub-directory.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:56:08 +01:00
Yann E. MORIN
b282f33cf8 fs: remove TARGET_DIR_WARNING_FILE late
Remove it just before generating the filesystem image.

This way, removing-and-recreating the file encloses the actual
image generation as tightly as possible.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:53:11 +01:00
Yann E. MORIN
207c596ad7 fs/common: get rid of post-target rules
The only users of post-target rules were ext2, cpio and initramfs.

Of those, ext2 and cpio were changed to use post-gen hooks, while
initramfs was not even using the generic rootfs infra and was fixed
to no longer reference post-target rules.

Besides, the comment in the infra was really misleading: it referenced
initramfs implying it was the sole user of that feature, even though
initramfs was not using the fs infra.

Furthermore, using post-target rules was inherently broken for top-level
parallel builds, because filesystems had to ensure the ordering by
themselves. Of the two real users of post-target rules (cpio and ext2),
one did enforce rules ordering (apparently correctly), while the other
forgot to do so.

We can get rid of post-target rules altogether, now.

Add a legacy check, to catch out-of-tree (e.g. br2-external) users of
post-target rules, and instruct them to switch to post-gen hooks instead.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:52:50 +01:00
Andrey Smirnov
3f9476df07 fs: add missing $$(sep) to pre- and post-command hooks code
When multiple hooks are registred, both pre-a and post-command hooks'
foreach loops need to have a separator at the end in order for the
code to work as intended. Without the separator all hooks end up as a
one single line command thus making all but the first hook into
no-ops.

Fixes: 4628b6f3b4 ("fs: add pre- and post-command hooks")
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-11-05 17:52:27 +01:00
Yann E. MORIN
4628b6f3b4 fs: add pre- and post-command hooks
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/

For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.

One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.

Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.

The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.

We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.

So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.

Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-02 20:58:27 +02:00
Arnout Vandecappelle
0f9c0bf3d5 Globally replace $(HOST_DIR)/usr/bin with $(HOST_DIR)/bin
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.

This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 15:19:29 +02:00
Yann E. MORIN
5b272e3e55 fs: set packages permissions even with no system device tables
Currently, when there is no syztem device table (permissions or static
devices) defined, then package permissions are not applied, because they
are guarded by the check on the system device tables being non empty.

Fix that by narrowing the guarding condition.

Note that the dependency on host-makedevs was not conditional; we always
build it even if we don't need it. Making it conditional is not
possible, because we don't know all the packages permissions by the time
the fs infra is parsed (packages from br2-external are parsed after it).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-14 23:45:48 +01:00
Peter Korsgaard
8852f08eed Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-01 22:29:56 +01:00
Peter Korsgaard
79061fe52f Revert "fs: use pseudo instead of fakeroot"
This reverts commit 8035ceb56c.

Moving to pseudo brought a number of issues (and longer compilation time),
so lets stick with fakeroot now that the reported ubi issue has been worked
around.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-27 21:24:10 +01:00
Peter Korsgaard
9b2f9b137c Revert "fs: call the pseudo wrapper"
This reverts commit 74a3e75bb8.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-27 21:12:23 +01:00
Jérôme Pouiller
68db7210fd fs/common: lock modification times in $TARGET_DIR
Make sure all files in $TARGET_DIR have a defined modification time
before to generate filesystems.

This work was sponsored by `BA Robotic Systems'.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 22:59:21 +01:00
Yann E. MORIN
74a3e75bb8 fs: call the pseudo wrapper
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Cc: Patrick Keroulas <patrick.keroulas@savoirfairelinux.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Cc: Julien BOIBESSOT <julien.boibessot@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-09 23:30:07 +01:00
Yann E. MORIN
8035ceb56c fs: use pseudo instead of fakeroot
fakeroot misbhaves when there are SElinux contexts, as reported in #9386,
and further detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1238802

A proposal in the FC bug is to use pseudo instead of fakeroot. Pseudo is
a from-scratch re-implementation of fakeroot; it is used in Poky/OE
instead of fakeroot.

Fixes #9386.

Reported-by: Andrey Yurovsky <yurovsky@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Andrey Yurovsky <yurovsky@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-03 21:36:15 +01:00
Yann E. MORIN
9cdb281fa6 fs/common: add option to execute custom scripts under fakeroot
Some users have the need to be able to tweak the content of the target
rootfs with root-like rights, that is, from inside the fakeroot script.

Add a new system option to allow those users to provide a list of
scripts, like the post-build and post-image scripts, that will be run
from our fakeroot script.

[Peter: pass TARGET_DIR to scripts, tweak help text]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Cam Hutchison <camh@xdna.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-03 20:58:46 +02:00
Yann E. MORIN
974e83a8f9 fs: add sha-bang to fakeroot script
We directly running this script, so it should start with a sha-bang (not
sure why/how it works today...).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-07 23:13:36 +02:00
Yann E. MORIN
6dd7bbb591 fs: properly escape commands when generating fakeroot script
Use the newly-introduced PRINTF macro to generate printf formats
that do "The Right Thing (TM)".

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-07 22:20:57 +02:00
Brian 'redbeard' Harrington
b2dcb2d6c4 fs/common: fix typo in comments
Fixed comment typos as to improve clarity.

Signed-off-by: Brian 'redbeard' Harrington <redbeard@coreos.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-08 15:34:52 +02:00
Yann E. MORIN
3f28a38e1a fs/common: generate users before setting permissions
We will need the users and groups to get defined before we can use them
from makedevs.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-02-01 07:25:36 +01:00
Thomas Petazzoni
15a53b93a0 fs/common: add <fs>_POST_GEN_HOOKS support
The common filesystem infrastructure already supports a
<fs>_PRE_GEN_HOOKS variable, which allows filesystem makefiles to
register some actions to be done before the root filesystem image is
generated.

This commit adds a similiar <fs>_POST_GEN_HOOKS variable, which will
allow filesystem makefiles to do some actions after the filesystem
image has been generated. It will initially be used by the iso9660
filesystem to delete the temporary directory it creates.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Noé Rubinstein <nrubinstein@aldebaran.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-08-10 20:56:05 +02:00
Thomas Petazzoni
95bd0ad351 fs: build host-mkpasswd when BR2_ROOTFS_USERS_TABLES is used
The code in fs/common.mk properly triggers the build of host-mkpasswd
when at least one package specifies a <pkg>_USERS variable. However,
when no selected package specifies a <pkg>_USERS variable but the user
uses a custom users table through BR2_ROOTFS_USERS_TABLES, then we
forget to build host-mkpasswd, leading to build failures if you don't
have mkpasswd installed in your machine.

This commit fixes that by taking into account both the package users
table and the custom users table when deciding to depend or not on
host-mkpasswd.

Reported-by: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2015-07-21 23:12:52 +02:00