Commit Graph

636 Commits

Author SHA1 Message Date
George Redivo
3146ba7633 support/scripts/graph-depends: add --flat-list option
graph-depends currently spits out a graph in .dot format. However, as
part of the upcoming introduction of <pkg>-show-recursive-depends and
<pkg>-show-recursive-rdepends, we need graph-depends to be able to
display a flat list.

Signed-off-by: George Redivo <george.redivo@datacom.ind.br>
[Thomas:
 - Rebase on top of graph-depends changes
 - Do not display the package name itself in the list, only its
   dependencies (or reverse dependencies)
 - Display the result on a single line, instead of one package per
   line, in order to match what <pkg>-show-depends and
   <pkg>-show-rdepends are doing today.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 22:25:57 +02:00
Peter Korsgaard
dcfcb777f4 support/scripts/graph-depends: add --quiet option
This will be useful for the upcoming recursive show-depends and
show-rdepends features.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 22:25:57 +02:00
Peter Korsgaard
3543d17441 support/scripts/graph-depends: use the standard python logging module
Instead of hardcoded sys.stderr.write() calls.  No functional change, but
allows us to easily implement a quiet option.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 22:25:57 +02:00
Thomas Petazzoni
3cd92ac711 support/scripts/graph-depends: use colors instead of colours
The graph-depends was not very consistent in colors vs. colours: some
parts were using colours, some parts were using colors.

Let's settle on the US spelling, colors.

This change the user-visble option --colours to --colors, but it is
unlikely that a lot of users customize the colors through
BR2_GRAPH_DEPS_OPTS, so this user interface change is considered
reasonable.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 20:01:38 +02:00
Thomas Petazzoni
f179394621 support/scripts/graph-depends: remove global code and most global variables
The graph-depends script had no main() function, and the main code was
actually spread between the function definitions, which was a real
mess.

This commit moves the global code into a main() function, which allows
to more easily follow the flow of the script. The argument parsing
code is moved into a parse_args() function.

Most of the global variables are removed, and are instead passed as
argument when appropriate. This has the side-effect that the
print_pkg_deps() function takes a lot of argument, but this is
considered better than tons of global variables.

The global variables that are removed are: max_depth, transitive,
mode, root_colour, target_colour, host_colour, outfile, dict_deps,
dict_version, stop_list, exclude_list, arrow_dir.

The root_colour/target_colour/host_colour variables are entirely
removed, and instead a single colours array is passed, and it's the
function using the colors that actually uses the different entries in
the array.

The way the print_attrs() function determines if we're display the
root node is not is changed. Instead of relying on the package name
and the mode (which requires passing the root package name, and the
mode), it relies on the depth: when the depth is 0, we're at the root
node.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 20:01:38 +02:00
Ricardo Martincoski
998300c5f5 testing/infra/builder: split configure() from build()
Some test cases don't use a full build as setup, so split the build()
method into configure() and build().
It allows a test case to perform configuration at the setup stage and
the build inside the test itself.

Call this new method just before build in the BRTest base class, to keep
the current behavior for existing test cases.

This change will be needed when adding a common class to test the git
download infra.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 19:59:36 +02:00
Ricardo Martincoski
b19cb8143b testing/infra/builder: call make with empty env
Use an empty environment when calling make, but import PATH so the
buildroot tree under test can find binaries from the host machine.

Since environment variables are now ignored, move the handling of
BR2_DL_DIR to the defconfig to keep the current precedence of -d:

    BR2_DL_DIR  | -d DIR   | test downloads   | BR downloads
    ------------+----------+------------------+--------------
    unset       | unset    | [error]          | [error]
    unset       | set      | in $(DIR)        | in $(DIR)
    set         | unset    | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
    set         | set      | in $(DIR)        | in $(DIR)

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 19:59:36 +02:00
Thomas Petazzoni
1fdd63ad37 toolchain-configs.csv: re-organize for test-pkg
This commit reorganizes the toolchain-configs.csv so that the first
toolchains are a subset of "useful" toolchains to be tested by
contributors to validate a package. This subset is the one that will
be used by default by test-pkg.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 16:30:30 +02:00
Gaël PORTAY
cf9bf7d28b support/download: keep files downloaded without hash
In the situation where the hash is missing from the hash file, the
dl-wrapper downloads the file again and again until the developer
specifies the hash to complete the download step.

To avoid this situation, the freshly-downloaded file is not removed
anymore after a successful download.

After this change, the behaviour is as follows:

- Hash file doesn't exist, or file is in BR_NO_CHECK_HASH_FOR
  => always succeeds.

- Hash file exists, but file is not present
  => file is NOT removed, build is terminated immediately (i.e.
     secondary site is not tried).

- Hash file exists, file is present, but hash mismatch
  => file is removed, secondary site is tried.
  => If all primary/secondary site downloads or hash checks fail, the
     build is terminated.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
[Arnout: extend commit log]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-04-01 15:02:53 +02:00
Yann E. MORIN
bb2a57a17a fs: run packages' filesystem hooks in a copy of target/
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.

However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.

We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.

This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
5563a1c6a4 support/check-uniq-files: support weird locales and filenames
Currently, when a filename contains characters not representable in the
user's locale, we fail hard, especially when the host python is python3.

This is because python2 and python3 handle encoding/decoding strings
differently, with python3 presumable doing the right thing, but it
breaks on some systems, while python2 presumable does the wrong thing,
but it works everywhere. (Just joking, obviously...)

Part of the issue being that the csv reader in python2 is broken with
UTF8.

We fix the issue by ditching the csv reader, and simply read the file in
binary mode, manually partitioning the lines on the first comma.

Then, we use the binary-encoded (really, un-encoded) package names and
filenames as values and keys, respectively.

Finally, for each filename or package we need to print, we try to decode
them with the defaults for the user settings, but catch any decoding
exception and fall back to dumping the raw, binary values. Which codec
is used by default differs between Python version, but in all cases
something sane is printed at least.

Thanks a lot to Arnout for the live help doing this patch. :-)

Reported-by: Jaap Crezee <jaap@jcz.nl>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Jaap Crezee <jaap@jcz.nl>
[Arnout: commit log improvement]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-03-31 15:32:00 +02:00
Thomas Petazzoni
0e4de0f2db support/config-fragments/autobuild: fix SSP in br-nios2-glibc
Commit c868095681 ("toolchain: fix
detection of SSP support") fixed the SSP check so that it does the
correct thing for nios2 toolchains. While this commit fixed the
description of the Sourcery NIOSII toolchain, it didn't fix the
description for the autobuilders of the br-nios2-glibc toolchain,
causing some build failures. This commit adjusts br-nios2-glibc.config
to indicate that the toolchain doesn't have SSP support.

Fixes:

  http://autobuild.buildroot.net/results/6c44e328b7bffd8474d29d5bdf1ea109ec15f4ad/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-31 08:49:14 +02:00
Thomas Petazzoni
ea9669fffa core: kill DEPENDENCIES_HOST_PREREQ
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:52:06 +02:00
Thomas Petazzoni
0283d7ee39 package/pkg-generic: handle host-ccache as a regular dependency
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:

 - The extractor packages host-tar, host-xz and host-lzip

 - host-ccache itself

 - host-skeleton, because all packages depend on it

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:49:05 +02:00
Thomas Petazzoni
15815e43dd package/pkg-generic: handle host-lzip as an extract dependency
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:

 - host-lzip, because we would otherwise depend on ourself.

 - host-tar, because lzip itself is delivered as a tarball, so we need
   to have host-lzip depend on host-tar, and not host-tar depend on
   host-lzip

 - host-skeleton, because we need to have host-lzip depend on
   host-skeleton, and not the opposite.

We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.

In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:45:58 +02:00
Thomas Petazzoni
f13fdc45e8 package/pkg-generic: handle host-xz as an extract dependency
This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:

 - host-xz, because we would otherwise depend on ourself.

 - host-tar, because xz itself is delivered as a tarball, so we need
   to have host-xz depend on host-tar, and not host-tar depend on
   host-xz

 - host-skeleton, because we need to have host-xz depend on
   host-skeleton, and not the opposite.

In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

[Peter: fix s/host-tar/host-xz/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:44:04 +02:00
Thomas Petazzoni
e77013a3c4 package/pkg-generic: handle host-tar as an extract dependency
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.

In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.

Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:43:20 +02:00
Ricardo Martincoski
14aa15a5a5 support/dockerfile: install flake8
Use the latest version of the tool because it is actively maintained.
But use a fixed version of the tool and its dependencies to get stable
results. It can be manually bumped from time to time.

Before installing any Python packages, ensure pip, setuptools, and wheel
are up to date as recommended in the docs [1].

[1] https://packaging.python.org/tutorials/installing-packages/

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-13 22:32:36 +01:00
Ricardo Martincoski
d44816f611 test_python_cryptography: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E301 expected 1 blank line, found 0
E302 expected 2 blank lines, found 1
F401 'os' imported but unused

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-13 22:28:05 +01:00
Yann E. MORIN
1b9a57442d support/check-bin-arch: exclude kernel modules for merged /usr
When using a merged /usr, the kernel module path is really
/usr/lib/modules, as /lib is a symlink to usr/lib .

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-07 23:03:52 +01:00
Yann E. MORIN
01d90f0d09 spport/check-bin-arch: accept arbitrary per-package ignore paths
Some packages (mostly, out-of-tree) may want to install binary blobs for
another architecture,  outside the locations we currently exclude, like
in /opt or whatever...

Add support in check-bin-arch to accept any arbitrary location, that
individual package can each request to excude from the check, when they
are installed.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-07 23:03:27 +01:00
Peter Korsgaard
92b8bd0879 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-05 21:04:14 +01:00
Peter Korsgaard
8a94ff12d2 Update for 2018.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 22:28:34 +01:00
Yann E. MORIN
69781ebb50 support/tests: enhance the runtime systemd tests
Recent systemd bump has broken DBus dameon and DBus applications can no
longer find the daemon. So we want to catch those kind of failures
early.

We also want to check that the system as a whole is stable: no unit
should be failed.

Finally, ensure that we can read the jounrnal, even when we are doing our
tricks on read-only systems.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 21:10:23 +01:00
Yann E. MORIN
b1aa2148b0 core: drop no-longer used C.UTF-8 locale option
Its use has been globbed into the more generic
BR2_NEEDS_HOST_UTF8_LOCALE option now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Julius Kriukas <julius@kriukas.lt>
Cc: Christian Stewart <kidovate@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:59:16 +01:00
Yann E. MORIN
9f8c7766e4 support/dependencies: unbreak check for UTF-8 locale
Although the UTF-8 locales in mainstream distributions all are suffixed
with just 'utf8', the nomenclature is a bit ambiguous with the way they
are to be specified with the various LC_* variables, suffixed there with
'UTF-8'.

Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_*
variables:
    http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02

    """
    If the locale value has the form:
        language[_territory][.codeset]

    it refers to an implementation-provided locale, where settings of
    language, territory, and codeset are implementation-defined.
    """

To avoid any confusion, use a regexp that is a bit more lax when
matching locales.

Also, quote the regexp, so that the '?' and '$' are not interpreted by
the shell.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:46:48 +01:00
Thomas Petazzoni
8cde7e6af9 support/dependencies/dependencies.sh: check for Python argparse module
The script support/scripts/check-uniq-files uses the argparse Python
module. In most recent Python versions (starting with 2.7), the
argparse module is part of the standard library, and we already check
for the availability of Python in
support/dependencies/dependencies.sh.

However, when running on an ancient distribution with Python 2.6, the
argparse module is not part of the Python standard library, but
available as an external module. Without this module, the build fails,
because check-uniq-files, which is used in target-finalize, fails to
run.

To avoid this failure, this commit adds a check in
support/dependencies/dependencies.sh to verify that the argparse
module is available.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-03 17:18:18 +01:00
Thomas Petazzoni
62fa5e17cb support/scripts/check-uniq-files: add indices in format string
Using {} in format strings is only supported in sufficiently recent
Python versions. Python 2.6 doesn't support this, and only format
strings with numbered arguments: {0}, {1}, etc.

Python 2.7:

$ python -c 'print("foo {}".format(12))'
foo 12
$ python -c 'print("foo {0}".format(12))'
foo 12

Python 2.6:

$ python -c 'print("foo {}".format(12))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: zero length field name in format
$ python -c 'print("foo {0}".format(12))'
foo 12

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-03 17:17:41 +01:00
Julius Kriukas
41569717c3 systemd: switch to C.UTF-8 locale when building
When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale
building systemd fails with an error:

[1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command.
FAILED: catalog/systemd.bg.catalog
/buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128)
package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed
make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1

This patch changes default host system locale from C to C.UTF-8 when
building systemd package to fix this issue. It also introduces
BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on
the host system. If locale is not available error message is show and build
process is stopped.

Signed-off-by: Julius Kriukas <julius@kriukas.lt>
[Thomas: use C.UTF-8 instead of en_US.UTF-8.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-02 22:44:34 +01:00
Peter Korsgaard
b8fa273d50 check-host-tar.sh: blacklist tar 1.30+
Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git.  This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.

For more details, see:
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html

To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:02:26 +01:00
Peter Korsgaard
7c09cb82b7 dependencies.mk: check for valid host-tar before other host dependencies
host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.

With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 17:01:47 +01:00
Thomas De Schampheleire
78ec4bd9e6 check-bin-arch: skip /lib/modules to allow 32-bit userland on 64-bit arch
The script check-bin-arch fails as follows on a config for PowerPC e6500
(64-bit CPU) with BR2_ARCH="powerpc" (32-bit userland desired):

ERROR: architecture for "/lib/modules/..../lib/libcrc32c.ko"
            is "PowerPC64", should be "PowerPC"

This situation is perfectly acceptable: the kernel is 64-bit and so are its
modules, even though userland is 32-bit.

To keep check-bin-arch and its caller simple, just skip /lib/modules/
entirely, like is done for /lib/firmware and some others.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-25 23:16:05 +01:00
Ricardo Martincoski
2b0ca5749c support/testing: fix job check-gitlab-ci.yml
Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method
with the name starting with "test_" otherwise nose2 assumes it is a test
case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-13 23:12:00 +01:00
Gaël PORTAY
27d2229692 support/dependencies: add check for c++ multilib
Some packages build C++ 32bits host-tools and need the g++-multilib to
be installed on the build machine. As example, qt5webengine builds a C++
host-tool when target is 32bits.

Add the check for g++-multilib to the dependencies script; and update
the Dockerfile to install g++-multilib package.

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-06 17:21:09 +01:00
Yegor Yefremov
ecfde09985 support/testing: add python-cryptography tests
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-06 15:10:39 +01:00
Thomas Petazzoni
14fdb63804 support/config-fragments: remove bogus BR2_JLEVEL option
There is no reason to have a BR2_JLEVEL option in such toolchain
defconfigs.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-05 22:46:59 +01:00
Thomas Petazzoni
6030986311 support/config-fragments: add internal defconfigs for glibc and musl
While we have several defconfigs building internal toolchains with
uClibc, we don't have any building internal toolchain with glibc and
musl. However, having such defconfigs is nice when we bump the C
library version, in order to immediately get feedback on build
failures.

Note that while the ARC internal defconfig uses glibc, it uses the
special ARC glibc version, so it doesn't test version bumps of the
upstream glibc C library.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-05 22:45:17 +01:00
Eric Le Bihan
e9429539ec support/testing: add tests for Rust
To test the support for the Rust language, the following tests are added:

- building Rust compiler and Cargo from source.
- installing a pre-built Rust compiler and building Cargo from source.

For each test, a Rust test program is built and installed in the root file
system of a ARM vexpress QEMU system. The test is declared OK if the program can
be run properly from the test system.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-05 14:57:48 +01:00
Yann E. MORIN
a874fc151a support/dockker: create and push tagged images
Currently, we refer to the latest version of the image, which means we
can't guarantee any reproducibility. Also, it measn we can't have a
separate images for the maintenance branches (especially the LTS) and
master.

Update the comment in the Dockerfile to create and push tagged images.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:58:59 +01:00
Yann E. MORIN
5fdf5e4099 support/docker: use a fixed Debian snapshot
Since we're now using a specific base image tag, we need to also use a
specific, stable repository to get additional packages from for this
image.

As such, use the Debian snapshot that matches the base image.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:57:53 +01:00
Yann E. MORIN
3c24b8bd40 support/docker: use a known base distribution
Currently, we are using debian:stable, which is subject to change with
time, as new stable versions of Debian are released/updated.

Use the latest tagged stable release, stretch-20171210 as of today, as
the base distribution to use.

This will ease reproducible builds in the future.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:57:37 +01:00
Yann E. MORIN
efa8744655 support/docker: don't be silent when setting up
This image is not built very often, and when it is, it is important to
see what's going on, so don't be silent when installing packages from
the distro, and since that can take a bit of time it thus serves as
progress report...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:53:02 +01:00
Yann E. MORIN
7517aef4dc support/docker: limit the number of layers
The official documentation [0] suggests limiting the number of layers
generated from a dockerfile. A layer is created for each RUN (and COPY
and ADD) command. But we are only ever interested in the final image,
so the intermediate layers are useless to us.

Limit the number of RUN commands to limit the number of generated
layers.

[0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-04 21:52:03 +01:00
Ricardo Martincoski
a3bb315e42 testing/tests/boot/test_atf: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E302 expected 2 blank lines, found 1
F401 'os' imported but unused

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
493a86a7b7 size-stats: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
253e016616 pycompile: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
cd04833926 graph-build-time: fix code style
Fix these warnings:
E201 whitespace after '['
E202 whitespace before ']'
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Ignore these warnings:
E402 module level import not at top of file

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
236bede631 check-uniq-files: fix code style
Fix these warnings:
E128 continuation line under-indented for visual indent
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
9ff44852c4 graph-depends: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
E202 whitespace before ']'
E221 multiple spaces before operator
E225 missing whitespace around operator
E231 missing whitespace after ','
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E502 the backslash is redundant between brackets
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ilya Kuzmich
28fac3973b merge_config.sh: add br2-external support
Pass BR2_EXTERNAL value via -e option.
This will prevent merge_config.sh from silently eating any symbols defined in
external trees on a clean buildroot tree invocation.

Signed-off-by: Ilya Kuzmich <ilya.kuzmich@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-29 14:54:42 +01:00