mkfs is now capable of generating rootfs images. Use mkfs instead of
genext2fs.
If not set, we now let mkfs calculate the block size and the number of
inodes needed.
This change also adjusts the options to meet those of mkfs.
Notes:
* Passing a null inode number to mkfs triggers its automatic calculation.
* Passing a fs-size with no unit suffix to mkfs is interpreted as using
1K block size.
Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: don't pass PATH when calling mkfs.ext, just call it directly
from $(HOST_DIR)/usr/sbin, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Soon, the ext{2,3,4} rootfs image will no longer be generated with
genext2fs, but using mke2fs instead which has no support for the extra
inode number option.
So, deprecate the BR2_TARGET_ROOTFS_EXT2_EXTRA_INODES option and
recommend, in lieu, to set the total inode number, taking account of
the extra ones if needed.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
ext filesystems can use a block size of 1024, 2048, or 4096 bytes, the
former being interesting to store small files, while the latter being
more intersting to store bigger files.
So far, we were using the default, which was to use a 1024-byte block.
Continue doing so (for now...).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
No option is conditional anymore, so just collate them into a single
assignment.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The previous default, zero, just meant "use the default value of the
filesystem generator", which happened to be 5% (the traditional value
for all ext-creating tools we've ever seen).
So, change the new default accordingly to 5%.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... since passing an empty string is equivalent to not setting a label.
And fix the syntax highlighting in some editors...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... since zero also means auto for the mke2img script, now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Auto-calculation of the rootfs size cannot be done in a reliable way
as it depends on the host filesystem and is broken on non ext4 host
(see bugs [1] [2]). So let the user specify the size he wants for his
rootfs.
[1] https://bugs.busybox.net/show_bug.cgi?id=8831
[2] https://bugs.busybox.net/show_bug.cgi?id=9496
Signed-off-by: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: as suggested by Arnout, use 60 MB as the default size instead
of 64 MB.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Rev0 is very old (E.G. from before Linux was maintained in git), the kernel
prints a scary warning when used:
EXT4-fs warning (device sda): ext4_update_dynamic_rev:746: updating to rev 1
because of new feature flag, running e2fsck is recommended
And rev0 support is broken in u-boot 2016.11:
http://lists.denx.de/pipermail/u-boot/2016-December/275916.html
So default to rev1 instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since the commit 6dd7bbb591, the label does
not need anymore to be quoted. Even worse it *must* not be simple-quoted,
unless the label will contain the double-quotes from the config variable
BR2_TARGET_ROOTFS_EXT2_LABEL.
The commit mentionned above has replaced echo by printf:
- echo "$$(ROOTFS_$(2)_CMD)" >> $$(FAKEROOT_SCRIPT)
+ $$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)
Since this commit the rootfs label contains extra double-quotes.
$ blkid
/dev/mmcblk0: LABEL=""BR 2016.08"" UUID="xxx"
^ ^
With this fix, the extra double-quotes have disappeared:
/dev/mmcblk0: LABEL="BR 2016.11-rc2" UUID="yyy"
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add two options to the ext2 filesystem, one to add extra free space, one
to add extra free inodes.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Martin Bark <martin@barkynet.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit improves the filesystem handling code to declare its
various targets as PHONY when appropriate.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Filesystems of the ext familly can carry a filesystem label.
Add an option for the user to specify such a label.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Károly Kasza <kaszak@gmail.com>
Tested-by: Karoly Kasza <kaszak@gmail.com>
Reviewed-by: Karoly Kasza <kaszak@gmail.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
And get rid of the old genext2fs.sh wrapper.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Karoly Kasza <kaszak@gmail.com>
Reviewed-by: Karoly Kasza <kaszak@gmail.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we are using a shell script called genext2fs, that
impersonates the real genext2fs. But that script does much more than
just call genextfs: it also calls tune2fs and e2fsck.
Because it impersonates genext2fs, we can not easily add new options,
and are constrained by the genext2fs options.
But it turns out that, of all the options supported by the real
genext2fs, we only really care for a subset, namely:
- number of blocks
- number of inodes
- percentage of blocks reeserved to root
- the root directory which to generate the image from
So, we introduce a new host package, mke2img, that is intended to
eventually replace genext2fs.sh.
This new script is highly modeled from the existing genext2fs.sh, but
was slightly refreshed, and a new, supposedly sane set of options has
been choosen for the features we need (see above), and some new options
were added, too, rather than relying on the arguments order or
environment variables:
-b <nb-blocks> number of blocks in the filesystem
-i <nb-inodes> number of inodes in the filesystem
-r <pc-reserved> percentage of reserved blocks
* -d <root-dir> directory containing the root of the filesystem
* -o <img-file> output image file
-G <ext-gen> extfs generation: 2, 3, or 4 (default: 2)
-R <ext-rev> ext2 revision: 0 or 1 (default 1)
-l <label> filesystem label
-u <uid> filesystem UUID; if not specified, a random one is used
* Mandatory options
Since the upstream e2fsprogs are expected to release a new mke2fs that
will be able to generate a filesystem image from a directory, we then
will be able to replace all the logic in mke2img, to use mke2fs instead
of the (relatively fragile) combination of the three tools we currently
use.
An entry is added for it in the "Host utilities" menu, so it can be
selected for use by post-{build,image} scripts. The ext2 filesystem
selection is changed to select that now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Karoly Kasza <kaszak@gmail.com>
Reviewed-by: Karoly Kasza <kaszak@gmail.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Thanks to the 2 previous patches of the series, BR_PATH contains
all locations in which host-packages may install programs.
This patch replaces the occurrences TARGET_PATH and HOST_PATH with
BR_PATH, everywhere these variables are used in the *.mk files.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
cset 7a58a4e (e2fsprogs: bump to version 1.42.9) broke the generation
of ext4 filesystems.
This is because, in ext4, some metadata are dependent on the UUID.
If changing the UUID of an ext4 filesystem, tune2fs now exits with
exit-code 1, and prints a message to run fsck, to avoid trashing the
filesystem.
This condition is of utmost importance on a mounted filesysten (which
is not our case) to avoid corruption (yes, it is possible to change
the UUID of a mounted filesystem).
But the error is not valid for us, since we are working on an unmonted
filesystem image in the first place.
Since we change the UUID after we convert the filesystem (to ext4),
tune2fs just bails out.
We can not just ignore the exit code of tune2fs, since we still want
to catch any other failure.
It turns out that, changing the UUID before converting the filesystem
is just the way to go.
Fixes#6752.
Reported-by: Daniel Mentz <daniel@exxm.de>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Daniel Mentz <daniel@exxm.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Tab instead of spaces
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Previously, a symlink was created to the uncompressed filesystem,
which made it a bit useless in case compression was chosen.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This will allow us to remove the unused ROOTFS_$(FSTYPE)_POST_GEN_HOOKS.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
genext2fs is built only if the user selected an ext2 root filesystem.
However, some use-cases can't live with the full target/ dir on the
root filesystem, and requires separate partitions (eg. for /usr).
In this case, the user would not select an ext2 root fs in the
Buildrooot menu, and would only generate a tarball of the rootfs.
This tarball would then be used from a post-image script to build
the actual required FSes.
But then, genext2fs is not built, since the ext2 root FS was not
selected.
As for the other filesystem generators, provide a host variant of
genext2fs (genext2fs is already host-package aware, so only needs
adding a Kconfig entry).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Turned out that setting a nil-UUID is no better than clearing it.
What currently happens is as follows:
- first, genext2fs does not generate a UUID
- then we tune2fs to upgrade the filesystem
- then we run fsck, which generates a random UUID
- then we re-run tune2fs to set a nil-UUID
So, on the target, if the file system is improperly unmounted (eg.
with a power failure), on next boot, fsck may be run, and a new
random UUID will be generated.
*However*, fsck improperly updates the filesystem when it adds the
UUID, and there are a few group descriptor checksum errors.
Those errors will go undetected until the next fsck, which will then
block for user input (bad on embedded systems, bad).
Fix that by systematically generating a random UUID _before_ we call
to fsck.
A random UUID is not so bad, after all, since there are already so
many sources of unpredictability in the filesystem: files date and
ordering, files content (date, paths...) which renders a fixed UUID
unneeded.
And it is still possible to set the UUID in a post-image script if
needed, anyway.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
"tune2fs -U clear" creates an invalid filesystem, that fsck.ext2
whines about later:
$ make rootfs-ext2
[--SNIP--]
$ ./host/usr/sbin/fsck.ext4 images/rootfs.ext2
e2fsck 1.42.7 (21-Jan-2013)
Filesystem did not have a UUID; generating one.
images/rootfs.ext2: clean, 4616/5120 files, 53171/131072 blocks
$ ./host/usr/sbin/fsck.ext4 -f images/rootfs.ext2
e2fsck 1.42.7 (21-Jan-2013)
One or more block group descriptor checksums are invalid. Fix<y>? yes
Group descriptor 0 checksum is 0x4131, should be 0x8bdb. FIXED.
[--SNIP--]
So we set an explicitly NULL UUID instead.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Add support for LZO and XZ compression methods to cpio, ext2, tar and
ubifs filesystem targets.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
A quick test with a ~3.5MB ext4 filesystem shows that 1081 blocks isn't
enough:
tune2fs 1.42.7 (21-Jan-2013)
Please run e2fsck on the filesystem.
Creating journal inode:
Journal size too big for filesystem.
So bump it a bit. Overestimating the journal size is probably not really
a big deal for the kind of systems using ext3/4 anyway.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Set the count- and time-based checks intervals to 0, thus effectively
disabling automatic checks at boot (after a suggestion by Arnout).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Some bootloaders have a buggy ext2 support, and require ext2 rev1
instead of the traditional ext2 rev0 that genext2fs produces.
tune2fs accepts only one '-O list' at a time, so we need to construct
a list of -O options.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Use the host-e2fsprogs to tune2fs the generated rootfs.ext2 image,
and upgrade it to either one of ext2, ext3 or ext4.
Since calling tune2fs may require running e2fsck (tune2fs will warn
to do so when certain FS options are changed), we systematically call
e2fsck. This makes the code path simpler, and as a side-effect checks
that genext2fs did not generate garbage.
In turn, e2fsck will unconditionally add a UUID to the filesystem,
which is bad for reproducibility, so we call tune2fs again to remove
the UUID. This does not require checking the filesystem.
To ensure compatibility of Buildroot's .config, leave ext2 as the
default. Boards' .config can override this at will.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Use a 'if...endif' construct instead of repeating the
'depends on' for each symbols.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This makes things easier to understand and more consistent with the pkg-infra.
For example, it removes the need for '$$@' in the CMD variables of fs/*/*.mk.
It also makes it possible to update the variables later, e.g. in the package
override file.
It also makes sure that the date will be recorded correctly in Yann E. Morin's
patch that logs the MESSAGE macros to a file.
The fs/*/*.mk must be updated as well because the '$@' shouldn't be quoted
anymore in the CMD variables or the hooks.
The $(eval ...) for the dependencies is redundant, because the $(ROOTFS_TARGET)
variable is already eval'd. Note that it is only redundant if the evaluation of
the uses of the variable is also delayed.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
The generic fs handling does a chmod -R 0:0 $(TARGET_DIR), so there's no
need for a specific option to enforce this when making an ext2fs image.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
It's pretty uncommon to use ext2fs on embedded systems, so don't enable
it by default.
Adjust defconfigs to match.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Closes#2929
Instead of just adding a fixed amount to the blocks used, try to
estimate the real space needed according to the filesystem structure
(bitmaps, inodes, blocks).
The side effect of this is that we no longer significantly overestimate
the size needed for small file systems.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
With the ROOTFS_TARGET conversion, EXT2_OPTS gets evaluated very early
(before TARGET_DIR is populated with files), so the calculated
blocks/inodes numbers are wrong.
Fix it by moving the calculation to a shell script wrapper around
genext2fs, so it only gets executed just before genext2fs runs.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
BR2_TARGET_ROOTFS_EXT2_RESBLKS is an int, so test against 0 rather than
the empty string - Otherwise the test is always true and a -m option
without arguments is added to the argument list, causing genext2fs to
get confused.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
The dependency on gzip, bzip2 and lzma are properly handled
automatically. No need to tell the user about this.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
