Commit Graph

628 Commits

Author SHA1 Message Date
Gaël PORTAY
cf9bf7d28b support/download: keep files downloaded without hash
In the situation where the hash is missing from the hash file, the
dl-wrapper downloads the file again and again until the developer
specifies the hash to complete the download step.

To avoid this situation, the freshly-downloaded file is not removed
anymore after a successful download.

After this change, the behaviour is as follows:

- Hash file doesn't exist, or file is in BR_NO_CHECK_HASH_FOR
  => always succeeds.

- Hash file exists, but file is not present
  => file is NOT removed, build is terminated immediately (i.e.
     secondary site is not tried).

- Hash file exists, file is present, but hash mismatch
  => file is removed, secondary site is tried.
  => If all primary/secondary site downloads or hash checks fail, the
     build is terminated.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
[Arnout: extend commit log]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-04-01 15:02:53 +02:00
Yann E. MORIN
bb2a57a17a fs: run packages' filesystem hooks in a copy of target/
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.

However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.

We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.

This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
5563a1c6a4 support/check-uniq-files: support weird locales and filenames
Currently, when a filename contains characters not representable in the
user's locale, we fail hard, especially when the host python is python3.

This is because python2 and python3 handle encoding/decoding strings
differently, with python3 presumable doing the right thing, but it
breaks on some systems, while python2 presumable does the wrong thing,
but it works everywhere. (Just joking, obviously...)

Part of the issue being that the csv reader in python2 is broken with
UTF8.

We fix the issue by ditching the csv reader, and simply read the file in
binary mode, manually partitioning the lines on the first comma.

Then, we use the binary-encoded (really, un-encoded) package names and
filenames as values and keys, respectively.

Finally, for each filename or package we need to print, we try to decode
them with the defaults for the user settings, but catch any decoding
exception and fall back to dumping the raw, binary values. Which codec
is used by default differs between Python version, but in all cases
something sane is printed at least.

Thanks a lot to Arnout for the live help doing this patch. :-)

Reported-by: Jaap Crezee <jaap@jcz.nl>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Jaap Crezee <jaap@jcz.nl>
[Arnout: commit log improvement]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-03-31 15:32:00 +02:00
Thomas Petazzoni
0e4de0f2db support/config-fragments/autobuild: fix SSP in br-nios2-glibc
Commit c868095681 ("toolchain: fix
detection of SSP support") fixed the SSP check so that it does the
correct thing for nios2 toolchains. While this commit fixed the
description of the Sourcery NIOSII toolchain, it didn't fix the
description for the autobuilders of the br-nios2-glibc toolchain,
causing some build failures. This commit adjusts br-nios2-glibc.config
to indicate that the toolchain doesn't have SSP support.

Fixes:

  http://autobuild.buildroot.net/results/6c44e328b7bffd8474d29d5bdf1ea109ec15f4ad/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-31 08:49:14 +02:00
Thomas Petazzoni
ea9669fffa core: kill DEPENDENCIES_HOST_PREREQ
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:52:06 +02:00
Thomas Petazzoni
0283d7ee39 package/pkg-generic: handle host-ccache as a regular dependency
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:

 - The extractor packages host-tar, host-xz and host-lzip

 - host-ccache itself

 - host-skeleton, because all packages depend on it

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:49:05 +02:00
Thomas Petazzoni
15815e43dd package/pkg-generic: handle host-lzip as an extract dependency
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:

 - host-lzip, because we would otherwise depend on ourself.

 - host-tar, because lzip itself is delivered as a tarball, so we need
   to have host-lzip depend on host-tar, and not host-tar depend on
   host-lzip

 - host-skeleton, because we need to have host-lzip depend on
   host-skeleton, and not the opposite.

We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.

In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:45:58 +02:00
Thomas Petazzoni
f13fdc45e8 package/pkg-generic: handle host-xz as an extract dependency
This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:

 - host-xz, because we would otherwise depend on ourself.

 - host-tar, because xz itself is delivered as a tarball, so we need
   to have host-xz depend on host-tar, and not host-tar depend on
   host-xz

 - host-skeleton, because we need to have host-xz depend on
   host-skeleton, and not the opposite.

In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-xz/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:44:04 +02:00
Thomas Petazzoni
e77013a3c4 package/pkg-generic: handle host-tar as an extract dependency
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.

In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.

Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:43:20 +02:00
Ricardo Martincoski
14aa15a5a5 support/dockerfile: install flake8
Use the latest version of the tool because it is actively maintained.
But use a fixed version of the tool and its dependencies to get stable
results. It can be manually bumped from time to time.

Before installing any Python packages, ensure pip, setuptools, and wheel
are up to date as recommended in the docs [1].

[1] https://packaging.python.org/tutorials/installing-packages/

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-13 22:32:36 +01:00
Ricardo Martincoski
d44816f611 test_python_cryptography: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E301 expected 1 blank line, found 0
E302 expected 2 blank lines, found 1
F401 'os' imported but unused

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-13 22:28:05 +01:00
Yann E. MORIN
1b9a57442d support/check-bin-arch: exclude kernel modules for merged /usr
When using a merged /usr, the kernel module path is really
/usr/lib/modules, as /lib is a symlink to usr/lib .

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-07 23:03:52 +01:00
Yann E. MORIN
01d90f0d09 spport/check-bin-arch: accept arbitrary per-package ignore paths
Some packages (mostly, out-of-tree) may want to install binary blobs for
another architecture,  outside the locations we currently exclude, like
in /opt or whatever...

Add support in check-bin-arch to accept any arbitrary location, that
individual package can each request to excude from the check, when they
are installed.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-07 23:03:27 +01:00
Peter Korsgaard
92b8bd0879 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-05 21:04:14 +01:00
Peter Korsgaard
8a94ff12d2 Update for 2018.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 22:28:34 +01:00
Yann E. MORIN
69781ebb50 support/tests: enhance the runtime systemd tests
Recent systemd bump has broken DBus dameon and DBus applications can no
longer find the daemon. So we want to catch those kind of failures
early.

We also want to check that the system as a whole is stable: no unit
should be failed.

Finally, ensure that we can read the jounrnal, even when we are doing our
tricks on read-only systems.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 21:10:23 +01:00
Yann E. MORIN
b1aa2148b0 core: drop no-longer used C.UTF-8 locale option
Its use has been globbed into the more generic
BR2_NEEDS_HOST_UTF8_LOCALE option now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Julius Kriukas <julius@kriukas.lt>
Cc: Christian Stewart <kidovate@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:59:16 +01:00
Yann E. MORIN
9f8c7766e4 support/dependencies: unbreak check for UTF-8 locale
Although the UTF-8 locales in mainstream distributions all are suffixed
with just 'utf8', the nomenclature is a bit ambiguous with the way they
are to be specified with the various LC_* variables, suffixed there with
'UTF-8'.

Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_*
variables:
    http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02

    """
    If the locale value has the form:
        language[_territory][.codeset]

    it refers to an implementation-provided locale, where settings of
    language, territory, and codeset are implementation-defined.
    """

To avoid any confusion, use a regexp that is a bit more lax when
matching locales.

Also, quote the regexp, so that the '?' and '$' are not interpreted by
the shell.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:46:48 +01:00
Thomas Petazzoni
8cde7e6af9 support/dependencies/dependencies.sh: check for Python argparse module
The script support/scripts/check-uniq-files uses the argparse Python
module. In most recent Python versions (starting with 2.7), the
argparse module is part of the standard library, and we already check
for the availability of Python in
support/dependencies/dependencies.sh.

However, when running on an ancient distribution with Python 2.6, the
argparse module is not part of the Python standard library, but
available as an external module. Without this module, the build fails,
because check-uniq-files, which is used in target-finalize, fails to
run.

To avoid this failure, this commit adds a check in
support/dependencies/dependencies.sh to verify that the argparse
module is available.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-03 17:18:18 +01:00
Thomas Petazzoni
62fa5e17cb support/scripts/check-uniq-files: add indices in format string
Using {} in format strings is only supported in sufficiently recent
Python versions. Python 2.6 doesn't support this, and only format
strings with numbered arguments: {0}, {1}, etc.

Python 2.7:

$ python -c 'print("foo {}".format(12))'
foo 12
$ python -c 'print("foo {0}".format(12))'
foo 12

Python 2.6:

$ python -c 'print("foo {}".format(12))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: zero length field name in format
$ python -c 'print("foo {0}".format(12))'
foo 12

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-03 17:17:41 +01:00
Julius Kriukas
41569717c3 systemd: switch to C.UTF-8 locale when building
When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale
building systemd fails with an error:

[1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command.
FAILED: catalog/systemd.bg.catalog
/buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128)
package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed
make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1

This patch changes default host system locale from C to C.UTF-8 when
building systemd package to fix this issue. It also introduces
BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on
the host system. If locale is not available error message is show and build
process is stopped.

Signed-off-by: Julius Kriukas <julius@kriukas.lt>
[Thomas: use C.UTF-8 instead of en_US.UTF-8.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-02 22:44:34 +01:00
Peter Korsgaard
b8fa273d50 check-host-tar.sh: blacklist tar 1.30+
Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git.  This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.

For more details, see:
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html

To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:02:26 +01:00
Peter Korsgaard
7c09cb82b7 dependencies.mk: check for valid host-tar before other host dependencies
host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.

With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:01:47 +01:00
Thomas De Schampheleire
78ec4bd9e6 check-bin-arch: skip /lib/modules to allow 32-bit userland on 64-bit arch
The script check-bin-arch fails as follows on a config for PowerPC e6500
(64-bit CPU) with BR2_ARCH="powerpc" (32-bit userland desired):

ERROR: architecture for "/lib/modules/..../lib/libcrc32c.ko"
            is "PowerPC64", should be "PowerPC"

This situation is perfectly acceptable: the kernel is 64-bit and so are its
modules, even though userland is 32-bit.

To keep check-bin-arch and its caller simple, just skip /lib/modules/
entirely, like is done for /lib/firmware and some others.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-25 23:16:05 +01:00
Ricardo Martincoski
2b0ca5749c support/testing: fix job check-gitlab-ci.yml
Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method
with the name starting with "test_" otherwise nose2 assumes it is a test
case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-13 23:12:00 +01:00
Gaël PORTAY
27d2229692 support/dependencies: add check for c++ multilib
Some packages build C++ 32bits host-tools and need the g++-multilib to
be installed on the build machine. As example, qt5webengine builds a C++
host-tool when target is 32bits.

Add the check for g++-multilib to the dependencies script; and update
the Dockerfile to install g++-multilib package.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-06 17:21:09 +01:00
Yegor Yefremov
ecfde09985 support/testing: add python-cryptography tests
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-06 15:10:39 +01:00
Thomas Petazzoni
14fdb63804 support/config-fragments: remove bogus BR2_JLEVEL option
There is no reason to have a BR2_JLEVEL option in such toolchain
defconfigs.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-05 22:46:59 +01:00
Thomas Petazzoni
6030986311 support/config-fragments: add internal defconfigs for glibc and musl
While we have several defconfigs building internal toolchains with
uClibc, we don't have any building internal toolchain with glibc and
musl. However, having such defconfigs is nice when we bump the C
library version, in order to immediately get feedback on build
failures.

Note that while the ARC internal defconfig uses glibc, it uses the
special ARC glibc version, so it doesn't test version bumps of the
upstream glibc C library.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-05 22:45:17 +01:00
Eric Le Bihan
e9429539ec support/testing: add tests for Rust
To test the support for the Rust language, the following tests are added:

- building Rust compiler and Cargo from source.
- installing a pre-built Rust compiler and building Cargo from source.

For each test, a Rust test program is built and installed in the root file
system of a ARM vexpress QEMU system. The test is declared OK if the program can
be run properly from the test system.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-05 14:57:48 +01:00
Yann E. MORIN
a874fc151a support/dockker: create and push tagged images
Currently, we refer to the latest version of the image, which means we
can't guarantee any reproducibility. Also, it measn we can't have a
separate images for the maintenance branches (especially the LTS) and
master.

Update the comment in the Dockerfile to create and push tagged images.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:58:59 +01:00
Yann E. MORIN
5fdf5e4099 support/docker: use a fixed Debian snapshot
Since we're now using a specific base image tag, we need to also use a
specific, stable repository to get additional packages from for this
image.

As such, use the Debian snapshot that matches the base image.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:57:53 +01:00
Yann E. MORIN
3c24b8bd40 support/docker: use a known base distribution
Currently, we are using debian:stable, which is subject to change with
time, as new stable versions of Debian are released/updated.

Use the latest tagged stable release, stretch-20171210 as of today, as
the base distribution to use.

This will ease reproducible builds in the future.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:57:37 +01:00
Yann E. MORIN
efa8744655 support/docker: don't be silent when setting up
This image is not built very often, and when it is, it is important to
see what's going on, so don't be silent when installing packages from
the distro, and since that can take a bit of time it thus serves as
progress report...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:53:02 +01:00
Yann E. MORIN
7517aef4dc support/docker: limit the number of layers
The official documentation [0] suggests limiting the number of layers
generated from a dockerfile. A layer is created for each RUN (and COPY
and ADD) command. But we are only ever interested in the final image,
so the intermediate layers are useless to us.

Limit the number of RUN commands to limit the number of generated
layers.

[0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:52:03 +01:00
Ricardo Martincoski
a3bb315e42 testing/tests/boot/test_atf: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E302 expected 2 blank lines, found 1
F401 'os' imported but unused

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
493a86a7b7 size-stats: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
253e016616 pycompile: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
cd04833926 graph-build-time: fix code style
Fix these warnings:
E201 whitespace after '['
E202 whitespace before ']'
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Ignore these warnings:
E402 module level import not at top of file

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
236bede631 check-uniq-files: fix code style
Fix these warnings:
E128 continuation line under-indented for visual indent
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
9ff44852c4 graph-depends: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
E202 whitespace before ']'
E221 multiple spaces before operator
E225 missing whitespace around operator
E231 missing whitespace after ','
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E502 the backslash is redundant between brackets
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ilya Kuzmich
28fac3973b merge_config.sh: add br2-external support
Pass BR2_EXTERNAL value via -e option.
This will prevent merge_config.sh from silently eating any symbols defined in
external trees on a clean buildroot tree invocation.

Signed-off-by: Ilya Kuzmich <ilya.kuzmich@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-29 14:54:42 +01:00
Yann E. MORIN
02fc52fe59 tests: add test for post-fakeroot script
Currently, only post-build and post-image scripts were tested, each with
their own test-script.

The two test-scripts only differ in the name of the log file they
create, and it is based on the name of the script, so it is easy to
share the script.

This allows us to easily re-use it for testing post-fakeroot scripts.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-15 21:37:45 +01:00
Guillermo A. Amaral
8aa4ee2b02 support/kconfig: ppply upstream nconfig ncurses/ncursesw fix
Buildroot's "make nconfig" command stopped working a while ago on
Gentoo systems. Running the command would result in a crash.

The issue is caused by lxdialog's cflags which are also used to build
nconfig; It would detect *ncursesw* and turn on WIDECHAR support --
but the Makefile would still link to plain *ncurses* while building
nconfig (which was built without WIDECHAR support).

This would cause a crash after using *wattrset* on a WINDOW instance.
WIDECHAR *wattrset* would try to set the _color member in the WINDOW
struct which does not exist in the NON-WIDECHAR ncurses instance. It
would end up clobbering data outside the struct (usually _line entries).

An upstream patch fixes the issue, so we're applying it to Buildroot's
kconfig.

Signed-off-by: Guillermo A. Amaral <g@maral.me>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-15 21:35:38 +01:00
Yann E. MORIN
ff0ebec8a9 support/dockerfile: add directives to run as non-root
Currently, our jobs on the gitlab-ci infra are running as root, which is
problematic for two reasons:

  - this is not the usual way Buildroot is built;
  - it may miss issues where running as non-root is problematic.

So, complement our Dockerfile with directives to add a new user and run
everything as that user, as demonstrated by this build job:
    https://gitlab.com/ymorin/buildroot-ci/-/jobs/46929562

Additional, enforce an UTF-8 locale while running.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-12 22:53:29 +01:00
Romain Naour
b6d39f8c03 support/config-fragments: br-powerpc-internal-full: remove old option
BR2_TOOLCHAIN_BUILDROOT_INET_IPV6 has been removed with the commit [1].

Since this option is still in br-powerpc-internal-full config-fragment,
the powerpc configuration is droped by autobuild-run script:

WARN: toolchain can't be used
      Missing: BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
[Sat, 06 Jan 2018 03:03:43] WARN: failed to generate configuration

[1] 4bcacfd2c0

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-12 22:25:52 +01:00
Arnout Vandecappelle (Essensium/Mind)
390780d562 dependencies: correct dependencies for top-level parallel build
Commit 4932c8a7cc introduced the
core-dependencies target to make sure that the dependencies.sh script
runs before we attempt to compile any host tool, so that the absence of
a compiler is properly detected. However, this relied on the
left-to-right evaluation of dependencies. This will no longer be true
when we enable top-level parallel build.

Fix this by letting DEPENDENCIES_HOST_PREREQ depend on
core-dependencies.

Note that it is not possible to remove the
dependencies <- core-dependencies. Indeed, it is possible that
DEPENDENCIES_HOST_PREREQ is completely empty, and in that case we still
need to check core-dependencies.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:37:59 +01:00
Arnout Vandecappelle (Essensium/Mind)
42decc07ce core-dependencies: remove unneeded HOSTCC
dependencies.sh uses HOSTCC_NOCCACHE directly, and this variable is
exported from the top-level Makefile, so there is no need to pass
HOSTCC to it. HOSTCC is not used at all in dependencies.sh.

Thus, we also no longer need to apply the HOSTCC override for
core-dependencies. The core-depencies rule doesn't use HOSTCC or
HOSTCXX.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 23:37:54 +01:00
Thomas De Schampheleire
a3f37c53d5 support/scripts: sort pie charts
Make sure that the pie charts produced by 'graph-build' and 'graph-size'
targets are sorted on the size of each piece of the pie. Otherwise, making
visual analysis is difficult, as one needs to look at the legends of each
piece and do the sorting manually in their head.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-02 22:40:09 +01:00
Yann E. MORIN
158e832aac fs/iso9660: fix transparent (de)compression
It needs mkzftree from zisofs-tools, so we add a dependency to it, and
we call that one explicitly (to avoid using the one from the host in
PATH).

It also needs the the uncompressed kernel image, but because it is
already in target/ so it gets compressed by mkzftree. We have two
options:
  - compress everything but the kernel image,
  - compress everything, kernel included, and recopy it later.

We choose the latter, because it is the simplest solution. So, we always
define the kernel-copy hook, but only register it when needed.

Finally, it needs a kernel with support for transparent
(de)compression, so we update the existing test config.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-02 21:57:43 +01:00