Commit Graph

358 Commits

Author SHA1 Message Date
Thomas Petazzoni
d0f4f95e39 Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.

For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.

This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.

Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.

Thanks to those changes, the global "dirs" target is entirely removed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:09:46 +01:00
Yann E. MORIN
6d688e2132 fs/tar: add support for xattrs (thus capabilties)
By default, tar will not include any extended attribute (xattr) when
creating archives, and thus will not store capabilties either (as they
are stored in the xattr 'security.capability').

Using option --xattrs is enough to create a tarball with all the xattrs
attached to a file. However, extracting all xattrs from a tarball
requires that --xattrs-include='*' be used. This is not symetric (but on
purpose, as per the documentation), and so is confusing to some.

So, we use --xattrs-include='*' to create the archive, so as to be
explicit that we want all xattrs to be stored.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-20 23:28:07 +01:00
Serj Kalichev
305e4487e5 fs/common.mk: Fix show-build-order
The command "make show-build-order" doesn't show dependencies of rootfs-common target.

This patch adds $(ROOTFS_COMMON_DEPENDENCIES) to PACKAGES variable.

Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-18 09:46:52 +01:00
Yann E. MORIN
409d4c3fe9 fs: don't use an intermediate tarball
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.

However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.

Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.

Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:

    File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring

This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.

The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.

The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.

So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216

Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.

[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-13 00:47:36 +01:00
Grzegorz Blach
3946f74b00 fs/f2fs: simplify overprovision option handling
Setting overprovision to 0 and omitting this option has exactly
the same effect.

Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-08 21:17:49 +01:00
Grzegorz Blach
25076c94c0 fs/f2fs: add option to define list of hot file extensions
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-08 21:07:40 +01:00
Grzegorz Blach
c3159a2d14 fs/f2fs: clarify help messages
For BR2_TARGET_ROOTFS_F2FS_COLD_FILES clarify that extension list must
be a coma separated.

For BR2_TARGET_ROOTFS_F2FS_OVERPROVISION clarify that the default
overprovision ratio is autocalculated according to the partition size.

Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-08 21:06:27 +01:00
Thomas Petazzoni
9617ce5cf9 fs/f2fs/Config.in: remove consecutive empty lines
This fixes the following check-package warning:

fs/f2fs/Config.in:51: consecutive empty lines

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:49:57 +01:00
Grzegorz Blach
3c66b119ed fs/f2fs: add option to define list of filesystem features
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Thomas: split from the initial patch from Grzegorz]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:32:52 +01:00
Grzegorz Blach
d4e5c0925a fs/f2fs: add option to define discard policy
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Thomas: split from the initial patch from Grzegorz]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:29:32 +01:00
Grzegorz Blach
0d27baa16a fs/f2fs: add option to define overprovision ratio
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Thomas: split from the initial patch from Grzegorz, reworded
Config.in help text]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:28:00 +01:00
Grzegorz Blach
0446f34d97 fs/f2fs: add option to define list of cold file extensions
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Thomas: split from the initial patch from Grzegorz]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:17:02 +01:00
Grzegorz Blach
6f727ce1fe fs/f2fs: add support for creating a f2fs image
This patch makes possible to create rootfs image using f2fs
filesystem.

Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Thomas:
 - keep only the minimal functionality, as suggested by Yann E. Morin
 - use truncate -s instead of dd to create the initial empty image
   file, as suggested by Yann E. Morin]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-03 15:14:47 +01:00
Yann E. MORIN
89c92487cf fs/common: always depend on build host-tar if needed
Currently, the filesystems do not depend on building host-tar when it
is needed, even though all of them have to extract the intermediate
tarball.

However, in degenerate (but legally valid) configurations with no
user-selectable package selected, host-tar would not be built, so the
rootfs images would use whatever improper tar the system has.

Add the conditional dependency to host-tar to the rootfs-common
intermediate image. Since this is the internal step that all real rootfs
generators depend on, they now properly depend on host-tar when needed.

In practice, when host-tar is needed, it will always be built before the
rootfs images, because it is a dependency of all packages (except a very
few, like the skeleton), of which host-fakeroot, which is a mandatory
dependency of rootfs-comon anyway. But for consistency sake, let's
explicitly add host-tar as a dependency to rootfs-common too.

Note that rootfs-tar already had that dependency, and we leave it as-is
because it is semantically correct, even if superfluous.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-01 22:01:46 +01:00
Matt Weber
c3edec0018 fs/common: allow custom user table to override package-defined users
Currently, when a custom user table and a package define the same user,
the settings from the package takes precedence over the ones from the
custom user table.

However, it makes sense to allow the settings from the custom user table
take precedence. For example, it would allow redirecting the user's
home directory to an alternate location (e.g. away from tmp and into a
partition that is persistent).

The support/scripts/mkusers script will only retain settings from the
latest definition it finds.

Thus, by passing the custom user table after the package defined users,
it is possible to override the package provided user definitions.

Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-10-20 14:33:24 +01:00
Yann E. MORIN
f023810f04 fs/btrfs: remove destination file before generation
btrfs will happily use an existing destination file if it
already exists, increasing its size if needed. Hoever, it
will never decrease the size, even if the requested size
is smaller than the existing file.

So, remove any previously existing destination file before
generating the new filesystem.

Note: the original submission by Robert did that, but as
this case was not obvious, the removal was dropped by a
refactoring when the patch was initially applied.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Robert J. Heywood <robert.heywood@codethink.co.uk>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Thomas: use $@ instead of $(@), use $(RM) instead of rm.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-25 23:59:23 +02:00
Robert J. Heywood
0aade4df1f fs/btrfs: add support for generating a btrfs image
This patch makes it possible to format the rootfs using btrfs. It
introduces the option; BR2_TARGET_ROOTFS_BTRFS.

When selected, the user is able to specify the filesystem size, label,
options, and node and sector sizes. The new files are based on
fs/ext2/{Config.in,ext2.mk}

Signed-off-by: Robert J. Heywood <robert.heywood@codethink.co.uk>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas:
 - fix issues pointed by Yann (duplicated empty line, missing quotes
   around default values for string options)
 - use -f option so that we don't have to remove the image file before
   creating it again
 - use the --byte-count option to set the filesystem size, which
   avoids the need for doing a "truncate -s"
 - remove the possible explanation of a mkfs.btrfs error. Indeed,
   mkfs.btrfs automatically extends the size of the image as needed,
   so the size passed can never be "too small".
 - fix check-package warnings in Config.in file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-25 14:30:51 +02:00
Peter Korsgaard
065e7672e1 fs: ensure hard links in TARGET_DIR are correctly copied for filesystem input
Fixes #11046

Commit bb2a57a17a (fs: run packages' filesystem hooks in a copy of target/)
changed the file system logic to run file system hooks from packages on a
copy of TARGET_DIR, and finally use this copy as input for the file system
generation.

This copy was done with rsync, which by default does not preserve hard
links, leading to an expansion of the file system images when hard links are
present.

Fix it by passing the -H option to rsync (preserve hard links).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-05-27 23:46:29 +02:00
Peter Seiderer
124a149b7d fs/squashfs: add zstd support
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-25 23:39:04 +02:00
Chris Brandt
265aee8c51 fs/cramfs: add xip option
Cramfs now has the ability to XIP all ELF files.

Cc: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Chris Brandt <chris.brandt@renesas.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-09 11:29:46 +02:00
Chris Brandt
2cf4871808 package/cramfs: change to new site location
As of Linux-4.15, cramfs now has a official maintainer again.
Additionally, that person is hosting and maintaining a new version of
cramfs-tools.

Patches 0001-endian.patch and 0003-fix-missing-types.patch are no longer
needed because they have been upstreamed.

However, since patch 0002-cygwin_IO.patch is so old, it is being
removed until someone that needs it can reapply and test it. At
that point, they should submit the changes to the new cramfs-tools
maintainer instead of adding the patches back here.

Please note that cross-endian support for cramfsck is not supported
at this time.

Cc: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Chris Brandt <chris.brandt@renesas.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-09 11:29:43 +02:00
Ricardo Martincoski
4a417b7509 fs/*/Config.in: re-wrap help text
... to follow the convention <tab><2 spaces><62 chars>.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 08:52:45 +02:00
Ricardo Martincoski
b205420e6a fs/*/Config.in: fix attributes order
... to follow the convention: type, default, depends on, select, help.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 08:52:45 +02:00
Ricardo Martincoski
9b6b4e36b4 */Config.in*: remove consecutive empty lines
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 08:48:24 +02:00
Ricardo Martincoski
65e3743945 fs/*/*.mk: fix code style
The header of the initramfs.mk file fits in one line, so rearrange it.
Remove consecutive empty line.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 08:40:41 +02:00
Ricardo Martincoski
8f2b06f455 fs/ubi: use namespace for internal variable
UBINIZE_CONFIG_FILE_PATH is only used inside this file, so rename it to
start with UBI_, following the namespace convention already used by
common packages.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 08:40:29 +02:00
Ricardo Martincoski
0c90c109cb fs/yaffs2: fix code style
Rename the .mk to follow the convention. The variables inside it already
use the correct name.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 07:55:22 +02:00
Yann E. MORIN
543107d390 fs: remove intermediate artefacts
Each of the intermediate, per-rootfs target directories, as well as the
intermediate tarball, can take quite some place, and is mostly a
duplication of what's already in target/. The only delta, if any, would
be the tweaks made by the filesystem image generations, but those tweaks
are most probably only meaningful when seen as root.

We normally do not remove intermediate files, but those can be quite
large, and are not directly usable by, nor accessible to the user.
So, get rid of them once the filesystem has been generated.

This does not need to be done in fakeroot.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
fcdf58cad1 fs: get rid of package-provided post-fs hooks
Now that the pre-fs ones are run on a transient copy of target/, the
post-fs hooks are no longer needed because we no longer need to restore
the target/ directory as it is only a internal copy.

Remove support for the post-fs hooks, and update the sole package using
them.

We do not add a legacy check because this was mostly a purely-internal
detail that was never really exposed nor documented.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
bb2a57a17a fs: run packages' filesystem hooks in a copy of target/
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.

However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.

We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.

This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
118534fe54 fs: use a common tarball as base for the other filesystems
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:

  - the content of the target directory for a specific filesystems may
    depend on whether another filesystem is enabled or not; for example,
    cpio will create a /init script or symlink and a /dev/console node;

  - the filesystems can not be built in parallel, because they may change
    the content of the target directory while another is being assembled.

Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...

We introduce an intermediate tarball, for which we do the full fakeroot
shebang.

That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.

Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
31fd0b1cac fs: introduce a macro for reproducible command
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
98d1e7f928 fs: new intermediate rootfs-common to gather common dependencies
Before we can create an intermediate tarball for all filesystems, we
nedd to move the common dependencies needed to generate that
intermediate tarball, rather than leave those dependencies to each
filesystem.

So, we introduce rootfs-common, which gathers all those common
dependencies.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
c6e425729e fs: introduce per-rootfs TARGET_DIR variable
... which for now still points to the base target directory, but this is
a step forward.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
2a6c55de49 fs: split per-rootfs dependency line
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
4815240830 fs: use a per-rootfs fakeroot script
... and locate that script in a per-rootfs directory.

Just like for ROOTFS, this variable will leak down the dependency tree to
target-finalize and packages - But it doesn't matter as it isn't used
outside fs/.

[Peter: extend commit message]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
0c7adf75d2 fs: define per-rootfs directory
This will serve in future commits to store pre-rootfs files, like
fakeroot script...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:05 +02:00
Yann E. MORIN
2765973e01 fs: set per-rootfs variable name
Like we do for packages with the PKG variable, set ROOTFS to contain the
upper-case name of the rootfs currently being generated.

This will be useful in later patches, when we need more per-rootfs
variables, like a per-rootfs TARGET_DIR for example.

In Makefiles, per-rule variables trickle down the dependency chain, to
all dependencies of that rule, so we have to stop ROOTFS as soon as
we're not in a rootfs. This means we have to stop it at target-finalize
(which is a dependency of all filesystems), and for each package
individually, since some packages (host or target) can be direct
dependencies of filesystems as well.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:52:52 +02:00
Yann E. MORIN
8bfbf04f0c fs/iso9660: rename internal variable
We're soon to introduce a generic variable with that name, i.e.
$(FS_NAME)_TARGET_DIR, so our internal, temporary variable would
clash with it.

Just rename it.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:47:25 +02:00
Yann E. MORIN
dec061adce fs/cpio: don't extend packages' permissions table
Currently, when we build a cpio filesystem without static devices, we
shoehorn the /dev/console node as if it were specified by a package.

This means that this device is added for all filesystems as well, not
just the cpio. But if we disable cpio, that device is not created for
other filesystems.

This is not very clean, and may break expectations.

Instead, use an explicit mknod as part of the _CMD, as we know it's
going to run under fakeroot.

This is still visible to all filesystems built after cpio, and not to
those built before it, though.

[Peter: ensure /dev exists, simplify comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:47:25 +02:00
Yann E. MORIN
405f912004 fs: run filesystem hooks under fakeroot
Currently, the pre-gen hooks of the various filesystems are run before
we enter the fakeroot.

However, this precludes those hooks from doing actions that require
root, like creating a pseudo-device or the likes.

So, move those pre-gen hooks under fakeroot.

This has currently no side-effect, as they are still called before
everything else in the fakeroot script, even the system-wide chown call.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:47:25 +02:00
Thomas Petazzoni
e77013a3c4 package/pkg-generic: handle host-tar as an extract dependency
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.

In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.

Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:43:20 +02:00
Gaël PORTAY
4fa524144b fs/cpio/init: preserve arguments
Use "$@" instead of $* to preserve arguments containing spaces.

The shell expands "$@" as "$1" "$2" "$3"... while it expands $@ as $1 $2
$3. With the second form, we loses spaces in positional parameters.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Reviewed-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-26 20:49:40 +01:00
Yann E. MORIN
4b3d1b36a3 fs: pass EXTRA_ENV to post-fakeroot script
Like we do for post-build and post-image scripts, pass EXTRA_ENV to
post-fakeroot script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-15 21:37:39 +01:00
Peter Korsgaard
e7652635fa fs/tar: add option for lz4 compression
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:44:08 +01:00
Peter Korsgaard
0952579754 fs/ext2: add option for lz4 compression
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:44:04 +01:00
Peter Korsgaard
2c7c71825f fs/cpio: add option for lz4 compression
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:44:00 +01:00
Peter Korsgaard
51fa8d8e28 fs/common.mk: support lz4 compression
Similar to the other compressors.  Notice that we use the -l (legacy format)
for Linux kernel initrd compatibility.

Lz4 decompression is supported by the Linux kernel since 3.11.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:43:58 +01:00
Yann E. MORIN
b6842c4097 fs: make it behave a bit more like the package infra
Currently, to register a filesystem, one has to call:

    $(eval $(call ROOTFS_TARGET,blabla))

This is very unlike the package infrastructure, where the name of the
package is automatically guessed by the infra.

It turns out that we can now do that for the filesystem infra too.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-03 21:58:07 +01:00
Yann E. MORIN
9fc21664e8 fs/ubifs: spin-off ubi to be its own filesystem
Currently, the ubifs-in-ubi-image ("UBI image" thereafter) filesystem
is half an option of the ubifs filesystem, half a filesystem on its
own: the config options are options of the ubifs filesystem, but the
.mk code is in a separate .mk and registers a real filesystem.

Make it a full filesystem on its own, in its own directory tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-03 21:57:49 +01:00
Yann E. MORIN
158e832aac fs/iso9660: fix transparent (de)compression
It needs mkzftree from zisofs-tools, so we add a dependency to it, and
we call that one explicitly (to avoid using the one from the host in
PATH).

It also needs the the uncompressed kernel image, but because it is
already in target/ so it gets compressed by mkzftree. We have two
options:
  - compress everything but the kernel image,
  - compress everything, kernel included, and recopy it later.

We choose the latter, because it is the simplest solution. So, we always
define the kernel-copy hook, but only register it when needed.

Finally, it needs a kernel with support for transparent
(de)compression, so we update the existing test config.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-02 21:57:43 +01:00
Yann E. MORIN
90867d8fdf fs/iso9660: add option for transparent (de)compression
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: rename option to BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-31 18:13:36 +01:00
Yann E. MORIN
62a198c0db fs/ubifs: split long line, use simple asignment
This line has been sitting there unchanged for years now, but it does
not follow current best pratices, that is:
  - do not use imediate assignment,
  - split lines longer than ~80 chars.

Fix that.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-27 22:53:26 +01:00
Yann E. MORIN
423b01ecc2 fs: prepare temp directory before running PRE_GEN hooks
Some filesystems have PRE_GEN hooks that create a directory structure
under the temporary directory.

For example, iso9660 will create a sub-directory where it stores the
kernel (in case of initramfs or initrd).

So, we must run the PRE_GEN hooks after we cleanup/create the temporary
directory.

Fixes:
    https://gitlab.com/buildroot.org/buildroot/-/jobs/42835965
    https://gitlab.com/buildroot.org/buildroot/-/jobs/42835967

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-02 14:37:53 +01:00
Yann E. MORIN
787a31fed4 fs: don't remove intermediate files
Now that they are in their own directory and no longer pollute the build
dir, there is no point in removing them.

Furthermore, a follow-up patch will require that those files survive
when more than one filesystem image is generated.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 15:50:53 +01:00
Yann E. MORIN
76ff493009 fs/iso9660: don't pollute $(BUILD_DIR) with temp dir
Use the newly-introdued $(FS_DIR) location to store temporary files.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 15:50:38 +01:00
Yann E. MORIN
917508e6bc fs: don't pollute $(BUILD_DIR) with temp files
Currently, we create a bunch of temporary files in $(BUILD_DIR), while
assembling the filesystem images.

Move those files to their own sub-directory.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:56:08 +01:00
Yann E. MORIN
b282f33cf8 fs: remove TARGET_DIR_WARNING_FILE late
Remove it just before generating the filesystem image.

This way, removing-and-recreating the file encloses the actual
image generation as tightly as possible.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:53:11 +01:00
Yann E. MORIN
207c596ad7 fs/common: get rid of post-target rules
The only users of post-target rules were ext2, cpio and initramfs.

Of those, ext2 and cpio were changed to use post-gen hooks, while
initramfs was not even using the generic rootfs infra and was fixed
to no longer reference post-target rules.

Besides, the comment in the infra was really misleading: it referenced
initramfs implying it was the sole user of that feature, even though
initramfs was not using the fs infra.

Furthermore, using post-target rules was inherently broken for top-level
parallel builds, because filesystems had to ensure the ordering by
themselves. Of the two real users of post-target rules (cpio and ext2),
one did enforce rules ordering (apparently correctly), while the other
forgot to do so.

We can get rid of post-target rules altogether, now.

Add a legacy check, to catch out-of-tree (e.g. br2-external) users of
post-target rules, and instruct them to switch to post-gen hooks instead.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:52:50 +01:00
Yann E. MORIN
9f2ca35998 fs/cpio: use a post-gen hook rather than a post-target rule
We already have a mechanism for running stuff after the filesystem is
generated, and that's called post-gen hooks.

Use those hooks.

Note: for cpio (and unlike ext2 previously), the dependency chain was
correct, in that the post-target rule correctly depended on the image
rule. Nonetheless, we still want to fix it for consistency.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:51:42 +01:00
Yann E. MORIN
03b6b7952f fs/ext2: use a post-gen hook rather than a post-target rule
post-target rules are probably not resilient to parallel builds, given
that they do not depend on the image being generated first.

Beside, we already have a mechanism for running stuff after the
filesystem is generated, and that's called post-gen hooks.

Use those hooks.

Note: this basically reverts 75b6303 (rootfs-ext2: make the symlink as a
_POST_TARGET) since we've now re-introduced post-gen hooks.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:51:08 +01:00
Yann E. MORIN
0a49b49e0a fs/initramfs: cleanups, enhance comments
rootfs-initramfs is not using the generic fs infrastructure, because
there is virtually nothing to do to build the initramfs image: there is
no actual image to be built to begin with.

The only purpose of rootfs-initramfs is to ensure the rootfs.cpio image
is built and then that the Linux kernel is rebuilt with that rootfs.cpio
as initramfs source.

Using variables of the fs infra like if it were used is misleading. It
looked nice as long as there was the possibility that rootfs-initramfs
would one day use the fs infra. But there's no way that will happen any
time soon.

Furthermore, the linux' rule linux-rebuild-with-initramfs now already
depends on rootfs-cpio by itself, so we need not duplicate this
dependency in rootfs-initramfs.

Still, we want to advertise that the dependency is on rootfs-cpio, so
we get nice dependency graphs (and not expose the internal
linux-rebuild-with-initramfs rule to the users).

So, remove the variables and directly define the rules.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:50:42 +01:00
Andrey Smirnov
3f9476df07 fs: add missing $$(sep) to pre- and post-command hooks code
When multiple hooks are registred, both pre-a and post-command hooks'
foreach loops need to have a separator at the end in order for the
code to work as intended. Without the separator all hooks end up as a
one single line command thus making all but the first hook into
no-ops.

Fixes: 4628b6f3b4 ("fs: add pre- and post-command hooks")
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-11-05 17:52:27 +01:00
Yann E. MORIN
54a76f08fc fs/iso9660: remove support for grub
grub will be removed soon.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-09-23 20:23:12 +02:00
Thomas Petazzoni
29ace1e216 Merge branch 'next' 2017-09-02 15:10:48 +02:00
Yann E. MORIN
8a26adddde fs/iso9660: really create initrd temp dir
In case we're using an initrd, we create an empty "root" directory that
will contain only the bootloader stuff, not the actual root filesystem,
because it is in an initrd (standalone or initramfs).

We have to ensure that the directory is empty before assembling the
filesystem (to avoid any file lingering from a previous run, like the
sequence  "make; make"). So we first remove it before we create it, so
that on each build (especially not-from-scratch builds) we get the exact
expected content without any leftover.

However, the macro responsible for that, although defined since 7080eef9,
was never called.

Fix that by registering it as a pre-gen hook.

Note: the directory need not be created, as there are quite a few
"install -D" commands that ensure it is created. Yet, we prefer to
create it explicitly to avoid any confusion.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-08-15 17:50:11 +02:00
Yann E. MORIN
62e626b2ab fs/squashfs: honour the number of parallel jobs
By default, mksquashfs uses all the CPUs available, like us with
BR2_JLEVEL=0.

However, by setting BR21_JLEVEL to non-zero, a user volutarily limits
the number of jobbs allowed to run concurrently (for various reasons,
like this is not a priority task on the machine; or like the
autobuilders, to spread the load accross multiple runs without
overloading the machine).

So, propagate tthe parallelisn settings down to mksquashfs, to repect
the user's will.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
Tested-by: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-08-09 23:57:35 +02:00
Yann E. MORIN
4628b6f3b4 fs: add pre- and post-command hooks
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/

For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.

One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.

Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.

The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.

We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.

So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.

Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-02 20:58:27 +02:00
Marcus Hoffmann
ef47a48137 fs/ext2: fix typo
Signed-off-by: Marcus Hoffmann <m.hoffmann@cartelsol.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-07-29 00:46:13 +02:00
Yann E. MORIN
a67f4c3e29 fs/ext2: hint about setting the filesystem size on error
Since we now request that the user sets the exact size of the ext2/3/4
filesystem, we've had quite a few users wondering what was going on when
they enable too much and the default 60M are no longer enough.

When mkfs.ext2 fails, print a hint that the user should check the size
setting.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: Add *** and redirect to stderr]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-07-26 00:53:05 +02:00
Yann E. MORIN
815d78f8ba fs/squashfs: cleanup if-else cascade
And add myself to the DEVELOPPERS for squashfs.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-07-11 22:33:51 +02:00
Sébastien Szymanski
b404b27d41 fs/ext2: Add BR2_TARGET_ROOTFS_EXT2_MKFS_OPTIONS option
This option lets the user specify mke2fs options and/or ext2/3/4
features.

Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: improve wording a bit, fix typos.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-09 15:35:42 +02:00
Samuel Martin
bee9e881a2 fs/ext2: rename BR2_TARGET_ROOTFS_EXT2_BLOCKS to BR2_TARGET_ROOTFS_EXT2_SIZE
This change deprecates the ext2/3/4 rootfs size in blocks symbol in
favor of one that mimic the fs-size argument behavior of mkfs (i.e.
size in a human readable format accepting k, m, g or t suffix or their
upper-case variants).

This change also updates the defconfigs that used to set
BR2_TARGET_ROOTFS_EXT2_BLOCKS symbol.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-09 15:35:30 +02:00
Arnout Vandecappelle
3b91bd4791 Globally replace $(HOST_DIR)/usr/share with $(HOST_DIR)/share
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.

This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/share' | xargs sed -i 's%$(HOST_DIR)/usr/share%$(HOST_DIR)/share%g'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 15:21:31 +02:00
Arnout Vandecappelle
1eb57df2b2 Globally replace $(HOST_DIR)/usr/sbin with $(HOST_DIR)/sbin
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.

This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/sbin' | xargs sed -i 's%$(HOST_DIR)/usr/sbin%$(HOST_DIR)/sbin%g'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 15:19:44 +02:00
Arnout Vandecappelle
0f9c0bf3d5 Globally replace $(HOST_DIR)/usr/bin with $(HOST_DIR)/bin
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.

This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 15:19:29 +02:00
Sébastien Szymanski
67aaa90d3b fs/ext2: use mkfs to generate rootfs image
mkfs is now capable of generating rootfs images. Use mkfs instead of
genext2fs.

If not set, we now let mkfs calculate the block size and the number of
inodes needed.

This change also adjusts the options to meet those of mkfs.

Notes:
* Passing a null inode number to mkfs triggers its automatic calculation.
* Passing a fs-size with no unit suffix to mkfs is interpreted as using
  1K block size.

Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: don't pass PATH when calling mkfs.ext, just call it directly
from $(HOST_DIR)/usr/sbin, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 00:39:38 +02:00
Samuel Martin
235b6f17a0 fs/ext2: deprecate BR2_TARGET_ROOTFS_EXT2_EXTRA_INODES option
Soon, the ext{2,3,4} rootfs image will no longer be generated with
genext2fs, but using mke2fs instead which has no support for the extra
inode number option.

So, deprecate the BR2_TARGET_ROOTFS_EXT2_EXTRA_INODES option and
recommend, in lieu, to set the total inode number, taking account of
the extra ones if needed.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 00:35:00 +02:00
Yann E. MORIN
221808810d package/mke2img: add option to specify block size
ext filesystems can use a block size of 1024, 2048, or 4096 bytes, the
former being interesting to store small files, while the latter being
more intersting to store bigger files.

So far, we were using the default, which was to use a 1024-byte block.
Continue doing so (for now...).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 08:50:40 +02:00
Yann E. MORIN
ddecc7dcb2 fs/ext2: simplify code
No option is conditional anymore, so just collate them into a single
assignment.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 08:49:15 +02:00
Yann E. MORIN
3286a4d6aa fs/ext2: allow reserving zero block for root
The previous default, zero, just meant "use the default value of the
filesystem generator", which happened to be 5% (the traditional value
for all ext-creating tools we've ever seen).

So, change the new default accordingly to 5%.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 08:47:57 +02:00
Yann E. MORIN
64cfa50789 fs/ext2: always pass the label option
... since passing an empty string is equivalent to not setting a label.

And fix the syntax highlighting in some editors...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 08:47:07 +02:00
Yann E. MORIN
ef975b5063 fs/ext2: always pass the number of inodes
... since zero also means auto for the mke2img script, now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-04 08:46:23 +02:00
J Evans
426fd787fa ext2: add help text for BR2_TARGET_ROOTFS_EXT2_BLOCKS
Signed-off-by: J Evans <g4@novadsp.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-04 17:15:13 +02:00
J Evans
84e1e38812 fs/ext2: ensure -b option is always set for mke2img
Signed-off-by: J Evans <g4@novadsp.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-30 19:31:52 +02:00
Sébastien Szymanski
c6bca8cef0 fs/ext2: Remove support for auto-calculation of rootfs size
Auto-calculation of the rootfs size cannot be done in a reliable way
as it depends on the host filesystem and is broken on non ext4 host
(see bugs [1] [2]). So let the user specify the size he wants for his
rootfs.

[1] https://bugs.busybox.net/show_bug.cgi?id=8831
[2] https://bugs.busybox.net/show_bug.cgi?id=9496

Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: as suggested by Arnout, use 60 MB as the default size instead
of 64 MB.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-25 14:41:05 +01:00
Arnout Vandecappelle
3e57fa370c fs/iso9660: doesn't support (grub2) EFI
The iso9660 generation for grub2 assumes that grub-eltorito.img is
available. However, this image is only available for the i386-pc target
(i.e. legacy BIOS). An EFI-bootable iso9660 requires a different layout.

Since we currently can't generate the EFI-bootable iso9660 layout,
require the i386-pc target to be selected in grub.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reported-by: arnaud.miche@orange.com
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-01 21:52:58 +01:00
Baruch Siach
d91ce1aa08 fs/tar: make --no-recursion effective
The tar --no-recursion option is position sensitive. It only affects following
file listing options. Move --no-recursion before the -T option to make it
effective. This fixes duplication of entries in the generated rootfs.tar
archive.

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-28 10:13:21 +01:00
Yann E. MORIN
5b272e3e55 fs: set packages permissions even with no system device tables
Currently, when there is no syztem device table (permissions or static
devices) defined, then package permissions are not applied, because they
are guarded by the check on the system device tables being non empty.

Fix that by narrowing the guarding condition.

Note that the dependency on host-makedevs was not conditional; we always
build it even if we don't need it. Making it conditional is not
possible, because we don't know all the packages permissions by the time
the fs infra is parsed (packages from br2-external are parsed after it).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-14 23:45:48 +01:00
Peter Korsgaard
219d0b5db2 fs/ext2: default to ext2 rev1
Rev0 is very old (E.G. from before Linux was maintained in git), the kernel
prints a scary warning when used:

EXT4-fs warning (device sda): ext4_update_dynamic_rev:746: updating to rev 1
because of new feature flag, running e2fsck is recommended

And rev0 support is broken in u-boot 2016.11:
http://lists.denx.de/pipermail/u-boot/2016-December/275916.html

So default to rev1 instead.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-14 15:59:38 +01:00
Peter Korsgaard
8852f08eed Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-01 22:29:56 +01:00
Peter Korsgaard
79061fe52f Revert "fs: use pseudo instead of fakeroot"
This reverts commit 8035ceb56c.

Moving to pseudo brought a number of issues (and longer compilation time),
so lets stick with fakeroot now that the reported ubi issue has been worked
around.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-27 21:24:10 +01:00
Peter Korsgaard
9b2f9b137c Revert "fs: call the pseudo wrapper"
This reverts commit 74a3e75bb8.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-27 21:12:23 +01:00
Peter Korsgaard
01354e1a0f fs/ubifs: work around fakeroot / sed -i issue on systems with SELinux enabled
Fixes #9386

GNU sed -i misbehaves on systems with SELinux enabled, as it tries to copy
to the SELinux security context (xattr) from the source file to to the new
destination file, which fails under fakeroot and leaves the file with 000
permissions, causing ubinize to fail when it cannot read to configuration
file.

So as a workaround, combine the install and tweak steps in a single
sed with a redirect to the destination file instead.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-27 20:31:37 +01:00
Jérôme Pouiller
68db7210fd fs/common: lock modification times in $TARGET_DIR
Make sure all files in $TARGET_DIR have a defined modification time
before to generate filesystems.

This work was sponsored by `BA Robotic Systems'.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 22:59:21 +01:00
Jérôme Pouiller
59f9c4ca9f fs/tar: make results reproducible
In order to make tar images reproducible, force files order in tarball.

This work was sponsored by `BA Robotic Systems'.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 22:55:34 +01:00
Gaël PORTAY
fe13efe19d fs/ext2: fix double quoted label
Since the commit 6dd7bbb591, the label does
not need anymore to be quoted. Even worse it *must* not be simple-quoted,
unless the label will contain the double-quotes from the config variable
BR2_TARGET_ROOTFS_EXT2_LABEL.

The commit mentionned above has replaced echo by printf:
-	echo "$$(ROOTFS_$(2)_CMD)" >> $$(FAKEROOT_SCRIPT)
+	$$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)

Since this commit the rootfs label contains extra double-quotes.
$ blkid
/dev/mmcblk0: LABEL=""BR 2016.08"" UUID="xxx"
                     ^          ^

With this fix, the extra double-quotes have disappeared:
/dev/mmcblk0: LABEL="BR 2016.11-rc2" UUID="yyy"

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-21 21:16:02 +01:00
Yann E. MORIN
74a3e75bb8 fs: call the pseudo wrapper
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Cc: Patrick Keroulas <patrick.keroulas@savoirfairelinux.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Cc: Julien BOIBESSOT <julien.boibessot@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-09 23:30:07 +01:00
Thomas Petazzoni
d5ed7fcad1 fs/iso9660: drop reference to host-fakeroot
Since commit 41f0688d91 ("fs/iso9660:
convert to the filesystem infrastructure"), the is9660 logic uses the
common filesystem infrastructure, so the dependency on host-fakeroot is
no longer needed, and has actually become bogus since we switched to
host-pseudo.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-03 23:28:01 +01:00
Yann E. MORIN
8035ceb56c fs: use pseudo instead of fakeroot
fakeroot misbhaves when there are SElinux contexts, as reported in #9386,
and further detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1238802

A proposal in the FC bug is to use pseudo instead of fakeroot. Pseudo is
a from-scratch re-implementation of fakeroot; it is used in Poky/OE
instead of fakeroot.

Fixes #9386.

Reported-by: Andrey Yurovsky <yurovsky@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Andrey Yurovsky <yurovsky@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-03 21:36:15 +01:00