Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.
However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.
We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.
This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, some filesystems may want to tweak the content of the target
directory, create special device nodes etc... This all means that:
- the content of the target directory for a specific filesystems may
depend on whether another filesystem is enabled or not; for example,
cpio will create a /init script or symlink and a /dev/console node;
- the filesystems can not be built in parallel, because they may change
the content of the target directory while another is being assembled.
Furthermore, the same fakeroot script is executed over-and-over-again
for each filesystem, to create the device nodes, the users and their
homes and files, and setting permissions...
We introduce an intermediate tarball, for which we do the full fakeroot
shebang.
That tarball then serves as the base for the other filesystems, with a
very simple fakeroot script that untars the common tarball, and calls
the actual filesystem image generator on that.
Note that we use a very simple tar command to generate the intermediate
tarball, because we are not concerned with reproducibility of the
archive itself (only of the archived files).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Before we can create an intermediate tarball for all filesystems, we
nedd to move the common dependencies needed to generate that
intermediate tarball, rather than leave those dependencies to each
filesystem.
So, we introduce rootfs-common, which gathers all those common
dependencies.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
... which for now still points to the base target directory, but this is
a step forward.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
... and locate that script in a per-rootfs directory.
Just like for ROOTFS, this variable will leak down the dependency tree to
target-finalize and packages - But it doesn't matter as it isn't used
outside fs/.
[Peter: extend commit message]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This will serve in future commits to store pre-rootfs files, like
fakeroot script...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Like we do for packages with the PKG variable, set ROOTFS to contain the
upper-case name of the rootfs currently being generated.
This will be useful in later patches, when we need more per-rootfs
variables, like a per-rootfs TARGET_DIR for example.
In Makefiles, per-rule variables trickle down the dependency chain, to
all dependencies of that rule, so we have to stop ROOTFS as soon as
we're not in a rootfs. This means we have to stop it at target-finalize
(which is a dependency of all filesystems), and for each package
individually, since some packages (host or target) can be direct
dependencies of filesystems as well.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We're soon to introduce a generic variable with that name, i.e.
$(FS_NAME)_TARGET_DIR, so our internal, temporary variable would
clash with it.
Just rename it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, when we build a cpio filesystem without static devices, we
shoehorn the /dev/console node as if it were specified by a package.
This means that this device is added for all filesystems as well, not
just the cpio. But if we disable cpio, that device is not created for
other filesystems.
This is not very clean, and may break expectations.
Instead, use an explicit mknod as part of the _CMD, as we know it's
going to run under fakeroot.
This is still visible to all filesystems built after cpio, and not to
those built before it, though.
[Peter: ensure /dev exists, simplify comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, the pre-gen hooks of the various filesystems are run before
we enter the fakeroot.
However, this precludes those hooks from doing actions that require
root, like creating a pseudo-device or the likes.
So, move those pre-gen hooks under fakeroot.
This has currently no side-effect, as they are still called before
everything else in the fakeroot script, even the system-wide chown call.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.
In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.
Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Use "$@" instead of $* to preserve arguments containing spaces.
The shell expands "$@" as "$1" "$2" "$3"... while it expands $@ as $1 $2
$3. With the second form, we loses spaces in positional parameters.
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Reviewed-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Like we do for post-build and post-image scripts, pass EXTRA_ENV to
post-fakeroot script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Similar to the other compressors. Notice that we use the -l (legacy format)
for Linux kernel initrd compatibility.
Lz4 decompression is supported by the Linux kernel since 3.11.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, to register a filesystem, one has to call:
$(eval $(call ROOTFS_TARGET,blabla))
This is very unlike the package infrastructure, where the name of the
package is automatically guessed by the infra.
It turns out that we can now do that for the filesystem infra too.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the ubifs-in-ubi-image ("UBI image" thereafter) filesystem
is half an option of the ubifs filesystem, half a filesystem on its
own: the config options are options of the ubifs filesystem, but the
.mk code is in a separate .mk and registers a real filesystem.
Make it a full filesystem on its own, in its own directory tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It needs mkzftree from zisofs-tools, so we add a dependency to it, and
we call that one explicitly (to avoid using the one from the host in
PATH).
It also needs the the uncompressed kernel image, but because it is
already in target/ so it gets compressed by mkzftree. We have two
options:
- compress everything but the kernel image,
- compress everything, kernel included, and recopy it later.
We choose the latter, because it is the simplest solution. So, we always
define the kernel-copy hook, but only register it when needed.
Finally, it needs a kernel with support for transparent
(de)compression, so we update the existing test config.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: rename option to BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This line has been sitting there unchanged for years now, but it does
not follow current best pratices, that is:
- do not use imediate assignment,
- split lines longer than ~80 chars.
Fix that.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some filesystems have PRE_GEN hooks that create a directory structure
under the temporary directory.
For example, iso9660 will create a sub-directory where it stores the
kernel (in case of initramfs or initrd).
So, we must run the PRE_GEN hooks after we cleanup/create the temporary
directory.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/42835965https://gitlab.com/buildroot.org/buildroot/-/jobs/42835967
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that they are in their own directory and no longer pollute the build
dir, there is no point in removing them.
Furthermore, a follow-up patch will require that those files survive
when more than one filesystem image is generated.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use the newly-introdued $(FS_DIR) location to store temporary files.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we create a bunch of temporary files in $(BUILD_DIR), while
assembling the filesystem images.
Move those files to their own sub-directory.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Remove it just before generating the filesystem image.
This way, removing-and-recreating the file encloses the actual
image generation as tightly as possible.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The only users of post-target rules were ext2, cpio and initramfs.
Of those, ext2 and cpio were changed to use post-gen hooks, while
initramfs was not even using the generic rootfs infra and was fixed
to no longer reference post-target rules.
Besides, the comment in the infra was really misleading: it referenced
initramfs implying it was the sole user of that feature, even though
initramfs was not using the fs infra.
Furthermore, using post-target rules was inherently broken for top-level
parallel builds, because filesystems had to ensure the ordering by
themselves. Of the two real users of post-target rules (cpio and ext2),
one did enforce rules ordering (apparently correctly), while the other
forgot to do so.
We can get rid of post-target rules altogether, now.
Add a legacy check, to catch out-of-tree (e.g. br2-external) users of
post-target rules, and instruct them to switch to post-gen hooks instead.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We already have a mechanism for running stuff after the filesystem is
generated, and that's called post-gen hooks.
Use those hooks.
Note: for cpio (and unlike ext2 previously), the dependency chain was
correct, in that the post-target rule correctly depended on the image
rule. Nonetheless, we still want to fix it for consistency.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
post-target rules are probably not resilient to parallel builds, given
that they do not depend on the image being generated first.
Beside, we already have a mechanism for running stuff after the
filesystem is generated, and that's called post-gen hooks.
Use those hooks.
Note: this basically reverts 75b6303 (rootfs-ext2: make the symlink as a
_POST_TARGET) since we've now re-introduced post-gen hooks.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
rootfs-initramfs is not using the generic fs infrastructure, because
there is virtually nothing to do to build the initramfs image: there is
no actual image to be built to begin with.
The only purpose of rootfs-initramfs is to ensure the rootfs.cpio image
is built and then that the Linux kernel is rebuilt with that rootfs.cpio
as initramfs source.
Using variables of the fs infra like if it were used is misleading. It
looked nice as long as there was the possibility that rootfs-initramfs
would one day use the fs infra. But there's no way that will happen any
time soon.
Furthermore, the linux' rule linux-rebuild-with-initramfs now already
depends on rootfs-cpio by itself, so we need not duplicate this
dependency in rootfs-initramfs.
Still, we want to advertise that the dependency is on rootfs-cpio, so
we get nice dependency graphs (and not expose the internal
linux-rebuild-with-initramfs rule to the users).
So, remove the variables and directly define the rules.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When multiple hooks are registred, both pre-a and post-command hooks'
foreach loops need to have a separator at the end in order for the
code to work as intended. Without the separator all hooks end up as a
one single line command thus making all but the first hook into
no-ops.
Fixes: 4628b6f3b4 ("fs: add pre- and post-command hooks")
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In case we're using an initrd, we create an empty "root" directory that
will contain only the bootloader stuff, not the actual root filesystem,
because it is in an initrd (standalone or initramfs).
We have to ensure that the directory is empty before assembling the
filesystem (to avoid any file lingering from a previous run, like the
sequence "make; make"). So we first remove it before we create it, so
that on each build (especially not-from-scratch builds) we get the exact
expected content without any leftover.
However, the macro responsible for that, although defined since 7080eef9,
was never called.
Fix that by registering it as a pre-gen hook.
Note: the directory need not be created, as there are quite a few
"install -D" commands that ensure it is created. Yet, we prefer to
create it explicitly to avoid any confusion.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
By default, mksquashfs uses all the CPUs available, like us with
BR2_JLEVEL=0.
However, by setting BR21_JLEVEL to non-zero, a user volutarily limits
the number of jobbs allowed to run concurrently (for various reasons,
like this is not a priority task on the machine; or like the
autobuilders, to spread the load accross multiple runs without
overloading the machine).
So, propagate tthe parallelisn settings down to mksquashfs, to repect
the user's will.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
Tested-by: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/
For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.
One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.
Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.
The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.
We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.
So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.
Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since we now request that the user sets the exact size of the ext2/3/4
filesystem, we've had quite a few users wondering what was going on when
they enable too much and the default 60M are no longer enough.
When mkfs.ext2 fails, print a hint that the user should check the size
setting.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: Add *** and redirect to stderr]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
And add myself to the DEVELOPPERS for squashfs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This change deprecates the ext2/3/4 rootfs size in blocks symbol in
favor of one that mimic the fs-size argument behavior of mkfs (i.e.
size in a human readable format accepting k, m, g or t suffix or their
upper-case variants).
This change also updates the defconfigs that used to set
BR2_TARGET_ROOTFS_EXT2_BLOCKS symbol.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/share' | xargs sed -i 's%$(HOST_DIR)/usr/share%$(HOST_DIR)/share%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/sbin' | xargs sed -i 's%$(HOST_DIR)/usr/sbin%$(HOST_DIR)/sbin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
mkfs is now capable of generating rootfs images. Use mkfs instead of
genext2fs.
If not set, we now let mkfs calculate the block size and the number of
inodes needed.
This change also adjusts the options to meet those of mkfs.
Notes:
* Passing a null inode number to mkfs triggers its automatic calculation.
* Passing a fs-size with no unit suffix to mkfs is interpreted as using
1K block size.
Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: don't pass PATH when calling mkfs.ext, just call it directly
from $(HOST_DIR)/usr/sbin, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Soon, the ext{2,3,4} rootfs image will no longer be generated with
genext2fs, but using mke2fs instead which has no support for the extra
inode number option.
So, deprecate the BR2_TARGET_ROOTFS_EXT2_EXTRA_INODES option and
recommend, in lieu, to set the total inode number, taking account of
the extra ones if needed.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
ext filesystems can use a block size of 1024, 2048, or 4096 bytes, the
former being interesting to store small files, while the latter being
more intersting to store bigger files.
So far, we were using the default, which was to use a 1024-byte block.
Continue doing so (for now...).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
No option is conditional anymore, so just collate them into a single
assignment.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The previous default, zero, just meant "use the default value of the
filesystem generator", which happened to be 5% (the traditional value
for all ext-creating tools we've ever seen).
So, change the new default accordingly to 5%.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... since passing an empty string is equivalent to not setting a label.
And fix the syntax highlighting in some editors...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... since zero also means auto for the mke2img script, now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Auto-calculation of the rootfs size cannot be done in a reliable way
as it depends on the host filesystem and is broken on non ext4 host
(see bugs [1] [2]). So let the user specify the size he wants for his
rootfs.
[1] https://bugs.busybox.net/show_bug.cgi?id=8831
[2] https://bugs.busybox.net/show_bug.cgi?id=9496
Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: as suggested by Arnout, use 60 MB as the default size instead
of 64 MB.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The iso9660 generation for grub2 assumes that grub-eltorito.img is
available. However, this image is only available for the i386-pc target
(i.e. legacy BIOS). An EFI-bootable iso9660 requires a different layout.
Since we currently can't generate the EFI-bootable iso9660 layout,
require the i386-pc target to be selected in grub.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reported-by: arnaud.miche@orange.com
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The tar --no-recursion option is position sensitive. It only affects following
file listing options. Move --no-recursion before the -T option to make it
effective. This fixes duplication of entries in the generated rootfs.tar
archive.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, when there is no syztem device table (permissions or static
devices) defined, then package permissions are not applied, because they
are guarded by the check on the system device tables being non empty.
Fix that by narrowing the guarding condition.
Note that the dependency on host-makedevs was not conditional; we always
build it even if we don't need it. Making it conditional is not
possible, because we don't know all the packages permissions by the time
the fs infra is parsed (packages from br2-external are parsed after it).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Rev0 is very old (E.G. from before Linux was maintained in git), the kernel
prints a scary warning when used:
EXT4-fs warning (device sda): ext4_update_dynamic_rev:746: updating to rev 1
because of new feature flag, running e2fsck is recommended
And rev0 support is broken in u-boot 2016.11:
http://lists.denx.de/pipermail/u-boot/2016-December/275916.html
So default to rev1 instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This reverts commit 8035ceb56c.
Moving to pseudo brought a number of issues (and longer compilation time),
so lets stick with fakeroot now that the reported ubi issue has been worked
around.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes#9386
GNU sed -i misbehaves on systems with SELinux enabled, as it tries to copy
to the SELinux security context (xattr) from the source file to to the new
destination file, which fails under fakeroot and leaves the file with 000
permissions, causing ubinize to fail when it cannot read to configuration
file.
So as a workaround, combine the install and tweak steps in a single
sed with a redirect to the destination file instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Make sure all files in $TARGET_DIR have a defined modification time
before to generate filesystems.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In order to make tar images reproducible, force files order in tarball.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since the commit 6dd7bbb591, the label does
not need anymore to be quoted. Even worse it *must* not be simple-quoted,
unless the label will contain the double-quotes from the config variable
BR2_TARGET_ROOTFS_EXT2_LABEL.
The commit mentionned above has replaced echo by printf:
- echo "$$(ROOTFS_$(2)_CMD)" >> $$(FAKEROOT_SCRIPT)
+ $$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)
Since this commit the rootfs label contains extra double-quotes.
$ blkid
/dev/mmcblk0: LABEL=""BR 2016.08"" UUID="xxx"
^ ^
With this fix, the extra double-quotes have disappeared:
/dev/mmcblk0: LABEL="BR 2016.11-rc2" UUID="yyy"
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit 41f0688d91 ("fs/iso9660:
convert to the filesystem infrastructure"), the is9660 logic uses the
common filesystem infrastructure, so the dependency on host-fakeroot is
no longer needed, and has actually become bogus since we switched to
host-pseudo.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
fakeroot misbhaves when there are SElinux contexts, as reported in #9386,
and further detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1238802
A proposal in the FC bug is to use pseudo instead of fakeroot. Pseudo is
a from-scratch re-implementation of fakeroot; it is used in Poky/OE
instead of fakeroot.
Fixes#9386.
Reported-by: Andrey Yurovsky <yurovsky@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Andrey Yurovsky <yurovsky@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some users have the need to be able to tweak the content of the target
rootfs with root-like rights, that is, from inside the fakeroot script.
Add a new system option to allow those users to provide a list of
scripts, like the post-build and post-image scripts, that will be run
from our fakeroot script.
[Peter: pass TARGET_DIR to scripts, tweak help text]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Cam Hutchison <camh@xdna.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Move TAR_OPTS so that long options (or any option with an initial '-')
may be passed to tar. Since TAR_OPTS is at the front of the list, single
letter options still work.
Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We directly running this script, so it should start with a sha-bang (not
sure why/how it works today...).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When that was added (in 975e30b, fs/squashfs: fix image file
permissions), the reasons were not quite explicit.
We are now forcing the umask, and various tests have shown that the mode
on the generated image file are correct without the chmod.
Remove it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Acked-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We can now properly use multi-line commands, so use that.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we can properly printf a multi-line command, there is no need
to use a single command to gnerate the squashfs image.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we can properly printf a multi-line command, there is no need
to use a single command to gnerate the jffs2 image.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use the newly-introduced PRINTF macro to generate printf formats
that do "The Right Thing (TM)".
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.. and add the missing newline-at-end-of-file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixed comment typos as to improve clarity.
Signed-off-by: Brian 'redbeard' Harrington <redbeard@coreos.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We will need the users and groups to get defined before we can use them
from makedevs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Add two options to the ext2 filesystem, one to add extra free space, one
to add extra free inodes.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Martin Bark <martin@barkynet.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Advanced XIP File System is a Linux kernel filesystem driver that
enables files to be executed directly from flash or ROM memory rather
than being copied into RAM. It has the ability to store individual
*pages* in a file uncompressed/XIP or compressed/Demand Paged.
This commit only adds support for 'XIP all' mode, so all the files that
have the execute attribute set will be XIP'ed.
At the moment, the FS is not supported in Linux mainline (v4.3-rc5), so
the kernel has to be built with the axfs patches to be able to read it.
Patches can be found here: https://github.com/jaredeh/axfs
[Thomas: rewrap Config.in help text.]
Signed-off-by: Ariel D'Alessandro <ariel@vanguardiasur.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Probably a leftover from our last fs infra reworking a couple of years
ago.
Incidentally, also added a newline at the end of the file.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
lz4 offers a high compression mode. A minimalistic file system shrinks by about 15%.
[Peter: Unconditionally use hc mode for lz4 instead of introducing another option]
Signed-off-by: Matthias Weisser <m.weisser.m@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Replace Grub-specific "menu.lst" with "menu config" in the
BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU Kconfig entry text, and mention
missing grub.cfg for Grub 2.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The common filesystem infrastructure already supports a
<fs>_PRE_GEN_HOOKS variable, which allows filesystem makefiles to
register some actions to be done before the root filesystem image is
generated.
This commit adds a similiar <fs>_POST_GEN_HOOKS variable, which will
allow filesystem makefiles to do some actions after the filesystem
image has been generated. It will initially be used by the iso9660
filesystem to delete the temporary directory it creates.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Noé Rubinstein <nrubinstein@aldebaran.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This installs every module selected by the BR2_TARGET_SYSLINUX_C32.
This is useful when using a custom Isolinux configuration file that may use
comboot modules.
Signed-off-by: Noé Rubinstein <nrubinstein@aldebaran.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The code in fs/common.mk properly triggers the build of host-mkpasswd
when at least one package specifies a <pkg>_USERS variable. However,
when no selected package specifies a <pkg>_USERS variable but the user
uses a custom users table through BR2_ROOTFS_USERS_TABLES, then we
forget to build host-mkpasswd, leading to build failures if you don't
have mkpasswd installed in your machine.
This commit fixes that by taking into account both the package users
table and the custom users table when deciding to depend or not on
host-mkpasswd.
Reported-by: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Auke Willem Oosterhoff <oosterhoff@baopt.nl>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Currently, the generated rootfs will be called rootfs.iso9660, but the
temporary directory we create while building it is called
rootfs-iso9660.tmp. They are in different directories so it's not so
obvious, but still to be consistent it's better to call the temp dir
rootfs.iso9660.tmp.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
After having extended the iso9660 support to allow isolinux as an
alternative to grub, this commit adds grub2 as a third
alternative. With the previous work done to support isolinux, adding
support for grub2 is fairly trivial.
[Thomas: set timeout to 10 seconds and not 5 seconds, in order to
match the configuration used for grub.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
This commit adds a new option, which allows, when isolinux is used as
the bootloader, to generate an "hybrid" ISO image. Such images can
either be booted from CD-ROM or from USB keys. It simply uses the
isohybrid tool provided by syslinux.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
After all the preparation commits, this commit finally adds the
iso9660 support itself. Besides adding a new Config.in entry, a little
bit of .mk code and the isolinux.cfg default configuration, not much
is needed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
In preparation to the introduction of the support for other
bootloaders than Grub, this commit switches from having the iso9660
support "select" the necessary bootloader to using a "depends on".
The main motivation is that the isolinux bootloader support will
really need to do a "depends on", due to the need of having a certain
option enabled, but this option being a kconfig "choice".
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
In preparation to the introduction for the support of other
bootloaders than Grub, this commit adds a "choice ... endchoice" block
with just the Grub option, and adds some conditionals in the
iso9660.mk code for the Grub specific parts.
Of course, for now those conditionals are a bit useless with just this
commit, but they become useful with the followup commits.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
In preparation to the introduction of the support for other
bootloaders that Grub, this commit introduces two new variables:
- ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH, which gives the full path to
the bootloader configuration file
- ROOTFS_ISO9660_BOOT_IMAGE, which gives the relative path of the
main bootloader image, as needed by genisoimage's -b option.
There are no functional changes made, as the variables are for now
always set to the same value.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Until now, the iso9660 filesystem handling only supported using an
initrd/initramfs to store the root filesystem, which is very different
from what we do with the other filesystems.
This commit changes the iso9660 logic to also allow using directly an
iso9660 filesystem to store the root filesystem. A new option,
BR2_TARGET_ROOTFS_ISO9660_INITRD, is created to tell the iso9660 that
we want to use an initrd and not directly the root filesystem in
iso9660 format. This option defaults to 'y' to preserve the existing
behavior.
After this commit, we therefore have three possibilities:
* BR2_TARGET_ROOTFS_ISO9660=y, with BR2_TARGET_ROOTFS_INITRAMFS and
BR2_TARGET_ROOTFS_ISO9660_INITRD disabled. In this case, the
iso9660 filesystem is directly the contents of the root filesystem
(since is possible thanks to the Rockridge extensions that were
already enabled using the -R option of genisoimage). Obviously, it
means that the root filesystem is read-only.
* BR2_TARGET_ROOTFS_ISO9660=y and BR2_TARGET_ROOTFS_INITRAMFS=y (the
value of BR2_TARGET_ROOTFS_ISO9660_INITRD doesn't matter). In this
case, the root filesystem is already linked into the kernel image
itself, as an initramfs. So the iso9660 filesystem doesn't contain
the root filesystem as is, but just the bootloader and the kernel
image.
* BR2_TARGET_ROOTFS_ISO9660=y, BR2_TARGET_ROOTFS_ISO9660_INITRD=y and
BR2_TARGET_ROOTFS_INITRAMFS disabled. In this case, a separate
initrd is used. The iso9660 filesystem only contains the
bootloader, the kernel and the initrd.
In order to support the first case out of the box, root=/dev/sr0 is
added on the kernel command line in the example Grub configuration
file, so that the kernel knows where the root filesystem is
located. This argument is ignored when initrd/initramfs are used.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
The iso9660.mk modifies the menu.lst file from Grub to set the correct
initrd/kernel image locations. However, with the upcoming support of
other bootloaders for iso9660 filesystems, we need to modify a bit
this logic.
Instead of relying on the specific details of the grub menu.lst
syntax, we introduce the __KERNEL_PATH__ and __INITRD_PATH__ magic
keywords, which iso9660.mk will replace by the appropriate
values. They can therefore be used where needed in grub menu.lst, and
in similar configuration files of other bootloaders, as will be
supported in the following commits.
Also, in order to be consistent with the soon to be introduced support
for having the root filesystem itself as iso9660, this commit changes
the installation location of the initrd and kernel. Instead of being
/initrd and /kernel, they become /boot/initrd and
/boot/$(LINUX_IMAGE_NAME).
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
