Commit Graph

919 Commits

Author SHA1 Message Date
Adrian Perez de Castro
e0c879509d support/dependencies/dependencies.sh: check for JSON:PP Perl module
The JSON::PP Perl module is used at build time by the webkitgtk and
wpewebkit packages.

Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-17 22:36:42 +02:00
Thomas Petazzoni
ffcd34af07 support/scripts/pkg-stats: simplify Git commit id retrieval
As suggested by Baruch Siach, using "git rev-parse HEAD" is a lot
simpler than playing around with "git log" to just retrieve the commit
id corresponding to the current HEAD.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-15 16:10:22 +02:00
Thomas Petazzoni
3f08ffa423 support/scripts/pkg-stats: extract current commit id, not master
pkg-stats extracts the Buildroot commit id from which the package
information was collected. However, when doing so, it always assumes
we're using the master branch, by running "git log master".

But in fact, pkg-stats can be run from any branch/tag, so it makes a
lot more sense to use "git log HEAD".

Cc: victor.huesca@bootlin.com
Cc: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-09-12 18:53:14 +02:00
Peter Korsgaard
d8e6851f11 Merge branch 'next' 2019-09-03 15:03:02 +02:00
Peter Korsgaard
1fcdfbfb8a Update for 2019.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-09-01 23:06:01 +02:00
Yann E. MORIN
20cbf17e0a support/graph-size: reorder colours assigned to sizes
Now that we can order packages from biggest to smallest, it makes sense
to assign the most aggressive colours to the biggest packages.

As such, reorder the current colours so that we have, in order:
  - red-ish
  - orange-ish
  - yellow-ish
  - purple-ish
  - eggplant-ish (is that even a colour? :-] )
  - some-indeterminate-blue-ish
  - dark-green-ish
  - light-green-ish

For the previous, smallest-first ordering, it does not matter much what
the ordering is: the actual colours are still somewhat-unpredictably
assigned to packages, depending on the cut-off limit...

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:51:47 +02:00
Yann E. MORIN
33c1ef88f8 support/graph-size: add option to sort packages in reverse size order
Currently, the packages are sorted smallest first, and biggest last
(with unknown and others second-to-last and last, resp.).

Add an option to invert the ordering (but keeping unknown and others at
their current positions).

This has the nice side effect that we can now control the colours
assigned to the biggest package(s), as the colours are cycled from the
first to the last. Currently, the biggest packages gets a redish colour,
which is appropriate, but the second gets a greenish one, which is not
as appropriate (but changing that can come later).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:50:05 +02:00
Yann E. MORIN
1dbce133db support/graph-size: add option to report size with IEC prefixes
When dealing with embedded devices, storage is more often than not some
kind of flash device, on which the memory is usually counted as powers
of 1024 instead of powers of 1000. As such, people may prefer reports
using IEC prefixes [0] instead of the SI prefixes.

Add an option to that effect.

We use argparse's ability to use custom actions [1] [2], to provide a
set of options that act on a boolean, but has a single help entry and
internally ensures consistency of the settings. We could have been using
the more conventional store_true/store_false actions instead, but that
would have meant either two help entries, one for each set of options,
and/or some logic after parse_args() to check the validity of the
settings.

[0] https://en.wikipedia.org/wiki/Binary_prefix
[1] https://docs.python.org/2/library/argparse.html#action
[2] https://docs.python.org/2/library/argparse.html#argparse.Action

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:49:22 +02:00
Yann E. MORIN
e9cdabee71 support/graph-size: add option to change percentage to group in Others
Currently, we group packages that contribute less then 1%, into the
"Other" category.

However, in some cases, there can be a lot of very comparatively small
packages, and they may not exceed this limit, and so only the "Others"
category would be displayed, which is not nice.

Conversely, if there are a lot of packages, most of which only so
slightly exceeding this limit, then we get all of them in the graph,
which is not nice either.

Add a way for the developers to pass a different cut-off limit. As for
the dependency graph which has BR2_GRAPH_DEPS_OPTS, add the environment
variable BR2_GRAPH_SIZE_OPTS to carry those extra option (in preparation
for more to come, later).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Arnout:
 - remove empty base class definition from Config;
 - use parser.error instead of ValueError for invalid argument.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:44:27 +02:00
Yann E. MORIN
3fc3c4ac99 support/graph-size: display human-readable size
Currently, we forcibly report sizes in multiple of Kilobytes. In some
big configurations, the sizes of the system as a whole, as well as that
of individual packages, may exceed megabytes, and when some artistic
assets get used, even the gigabyte may get exceed.

These big sizes are not easy to read when expressed in kilobytes.

Additionally, some very small packages might have sizes below the
kilobyte (and when we can specify the cut-off grouping size, they may
get reported), and thus the size displayed for those would be 0 kB.

Add a helper function that can format a floating-point size into a
string with all the appropriate formatting:

  - there are at least 3 meaningfull digits visible, i.e. we display
    "3.14" or "10.4" instead of just "3" or "10", but for big number we
    don't care about too many precision either, so we report "100" or
    "1000", not "100.42" or "1000.27";

  - the proper SI prefix is appended, if needed.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:15:35 +02:00
Yann E. MORIN
e8de561436 support/graph-size: report 'Unknown" after all packages, but before "Others"
Currently, the "unknown" category may be reported anywhere, so it does
not really stand out when there are a lot of packages in the graph.

Move it towards the end, but right before the "other" category, so that
it is a bit more visible. Like for Others, don't report it if its size
is zero.

Also, make it title case (i.e. "Unknown" instead of "unknown").

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:12:37 +02:00
Yann E. MORIN
c68ee73924 support/graph-size: don't report "Others" if size is zero
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:08:18 +02:00
Yann E. MORIN
a2d20ca613 support/graph-size: introduce main()
It is nicer overall to have a main() function, like all our other
scripts tend to have too.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:08:01 +02:00
Yann E. MORIN
cecaf7001f support/graph-size: fix flake8 warnings
There are three E501 warnings returned by flake8, when run locally,
because we enforce a local 80-char limit, but that are not reported by
the gitlab-ci jobs because only a 132-char limit is required there.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-26 22:07:50 +02:00
Victor Huesca
152776b420 support/testing: new test for root password
Add support to test that the root passowrd is working as expected.
- Buildtime test: Check the hash present in the generated '/etc/shadow'.
- Runtime test: Build an armv7 image and try to login with a password.

Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-13 22:30:41 +02:00
Ricardo Martincoski
620c98a214 support/testing: factor out run_cmd_on_host
Currently many test cases call subprocess.check_output on their own.
Factor out that code to an infra method so the call get standardized.

This will be handful when switching the test infra to use Python 3.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:58 +02:00
Ricardo Martincoski
1b2928201f support/testing: do not print non-utf-8 chars on cbor test
The python-cbor sample script to be run on the target dumps a binary
variable to the target stdout. This is done for debug purposes-only, in
the case the test on test infra fails.
This non-utf-8 is currently silently ignored by the Emulator class /
pexpect.spawn from the test infra because the infra uses Python 2, that
in turn do not differentiate between string and byte data.

Make the code Python3-friendly (Python 3 in the host) by doing the right
thing and encoding the data before printing it.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:55 +02:00
Ricardo Martincoski
8de4291bd2 support/testing: check ubi image before booting
Do the same as other fs tests and minimally check the ubi image before
booting.
The call to 'file' was already there, but the output wasn't tested for
some unknown reason. Add the assert for the output of the command.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:53 +02:00
Ricardo Martincoski
a565917046 support/testing: provide entropy to perl tests
Newer versions of perl-io-socket-ssl require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
perl tests.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139402

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
eb6b0fd87a support/testing: provide entropy to lua tests
Newer versions of lua-http require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
lua tests.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139374
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139376

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
d23056ffe1 support/testing: make crossbar to use umsgpack
Since version 19.3.3, python-autobahn assumes that when CPython is used
msgpack will be used as well [1]. But it still allows the user to
override this behavior by setting an environment variable [2] to use
umsgpack.

Make the test to explicitly use umsgpack since it is part of the minimal
config (python-crossbar selects python-u-msgpack).

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139430

[1] ea019b8042
[2] fe70ceebe0

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
ef5e87bb92 support/testing: make treq tests to ignore warning
Newer versions of python-treq display a warning at runtime when
service_identity is not installed:
"Without the service_identity module, Twisted can perform only
rudimentary TLS client hostname verification."

This warning message confuses the test that looks for another string in
stdout. Make the test ignore other messages while still expecting
"Connection refused".

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139449
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139450

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 14:17:28 +02:00
Ricardo Martincoski
5c39c90ae1 support/testing: fix hardening tests
Since "2467822c85 package/checksec: bump to version 2.1.0" the hardening
tests fail because upstream slightly changed the way the script is
called.
According to README.md: "- All options now require `--$option=$value`
instead of `--$option $value`"

Instead of just replacing '--output json' with '--output=json' take into
account that upstream also changed the usage example to show --format
instead of --output. Both options do exactly the same, but following the
usage example seems to be more future-proof.

Upstream also improved the json output. Now when a file is passed as
parameter, the json has the file name as the main key, instead of the
string "file". Adjust the test cases accordingly.

Fixes:
tests.core.test_hardening.TestFortifyConserv
tests.core.test_hardening.TestFortifyNone
tests.core.test_hardening.TestRelro
tests.core.test_hardening.TestRelroPartial
tests.core.test_hardening.TestSspNone
tests.core.test_hardening.TestSspStrong

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-05 22:15:18 +02:00
Thomas Petazzoni
dc938c221d support/config-fragments/autobuild: set gcc version for RISC-V toolchains
Prior to b3ba26150d
("toolchain/toolchain-external/toolchain-external-custom: be more
flexible on gcc version"), the default gcc version selected by
Buildroot for custom external toolchain was affected by the
BR2_ARCH_NEEDS_GCC_AT_LEAST_xyz definitions.

Since BR2_riscv selects BR2_ARCH_NEEDS_GCC_AT_LEAST_7, gcc 7.x was the
default gcc version assumed to be used in a custom RISC-V external
toolchain, so our config snippets for RISC-V toolchains were correct.

With b3ba26150d applied, the default gcc
version assumed for custom external toolchains is the latest one
(currently gcc 9.x), while our RISC-V toolchains use gcc 7.x. So we
now need to explicitly give the gcc version used by our RISC-V
toolchains, otherwise the build fails with:

  Incorrect selection of gcc version: expected 9.x, got 7.4.0

Fixes:

  http://autobuild.buildroot.net/results/b872befe1adec2633b9cbcc49bc0eb7619f606c2/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-04 08:55:10 +02:00
Yann E. MORIN
cfb929fbfa core: allow br2-external trees to provide opensl
Similar to toolchains and jpeg, we now offer a way for br2-external
trees to provide their openssl implementation, which gets included in
the openssl choice.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-04 00:13:37 +02:00
Yann E. MORIN
3b67e8e664 core: allow br2-external trees to provide libjpeg
Similar to toolchains, we now offer a way for br2-external trees to
provide their libjpeg implementation, which gets included in the jpeg
choice.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-04 00:13:37 +02:00
Yann E. MORIN
fa037acee0 core: allow br2-external trees to provide pre-configured toolchains
Since we have a choice for the pre-configured pre-built toolchains,
there is no possbility for a br2-external to provide its own. The
only solution so far for defconfigs in br2-external trees is to use
BR2_TOOLCHAIN_EXTERNAL_CUSTOM and define all the bits by itself...

This is not so convemient, so offer a way for br2-external trees to
provide such pre-configured toolchains.

To allow for this, we now scan each br2-external tree and look for a
specific file, provides.toolchains.in. We generate a kconfig file that
sources each such file, and that generated file is sourced from within
the toolchain choice, thus making the toolchains from a br2-external
tree possible and available in the same location as the ones known to
Buildroot:

    Toolchain  --->
        Toolchain type (External toolchain)  --->
        Toolchain  --->
            (X) Arm ARM 2019.03
            ( ) Linaro ARM 2018.05
            ( ) Custom toolchain
                *** Toolchains from my-br2-ext-tree: ***
            ( ) My custom ARM toolchain
                *** Toolchains from another-br2-ext-tree: ***
            ( ) Another custom ARM toolchain
            ( ) A third custom ARM toolchain

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-04 00:13:37 +02:00
Yann E. MORIN
edf32b021c core: split generated kconfig file
Currently, the kconfig part contains two things: the kconfig option
with the paths to br2-external trees, and the kconfig menus for the
br2-external trees.

When we want to include more kconfig files from the br2-external tree
(e.g. to get definitions for pre-built toolchains), we will need to
have the paths defined earlier, so they can be used from the br2-external
tree to include files earlier than the existing menus.

Split the generated kconfig file in two: one to define the paths, which
gets included early in our main Config.in, and one to actually define
the existing menus, which still gets included at the same place they
currently are.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-04 00:13:37 +02:00
Yann E. MORIN
0797dae894 core: prepare for generating multiple kconfig fragments
We currently redirect the output of each helper function. This was nice
as long as we were generating single .mk and .in fragments.

But we are soon to need more .in fragments.

So, do the redirection inside the .in helpers.

We do not (currently) need to generate more than one .mk fragment, but
for consistency, do the redirection in the .mk helper too.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 21:51:40 +02:00
Yann E. MORIN
d027cd75d0 core: generate all br2-external files in one go
When we introduced support for multiple br2-external trees, we
introduced two files, one on the Makefile side, needed very early,
and one on the kconfig side, needed later in the configuration
process. We naturally introduced a two-step generation, as it looked
like the simplest and most obvious way.

But now, we are on the verge of generating more files on the kconfig
side, and it does not make sense to add even more steps to generate
them.

And even better yet, we can generate both the Makefile-side and
kconfig-side files at the same time, in fact.

Make it so.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 21:51:40 +02:00
Yann E. MORIN
2130903347 support/scripts/br2-external: drop help for internal helper script
We do not usually provide help for our internal scripts. Besides, such
help has a tendency to bitrot pretty quickly anyway.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 19:58:46 +02:00
Yann E. MORIN
3617f1350a support/scripts/br2-external: declare missing local variables
Commit b14b02698 (core/br2-external: restore compatibility with old
distros) switched to using 'eval' to emulate associative arrays, for
those distros too old to have bash-4+.

In so doing, it forgot to declare the new local variables in the
respective helper functions.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 19:58:17 +02:00
Frank Vanbever
07f31ee263 support/cmake: Explicitly set CMAKE_SYSTEM
Some packages test for CMAKE_SYSTEM explicitly[1]

CMAKE_SYSTEM is comprised of CMAKE_SYSTEM_NAME and CMAKE_SYSTEM_VERSION.
It defaults to CMAKE_SYSTEM_NAME if CMAKE_SYSTEM_VERSION is not set[2]

At the point CMAKE_SYSTEM_NAME is set to "Linux" CMAKE_SYSTEM is already
constructed. Setting it explicitly ensures that it is the correct value.

This is because we do set CMAKE_SYSTEM_NAME twice, in fact:

  - first in toolchainfile.cmake, so that we tell cmake to use the
    "Buildroot" platform,

  - second, in the Buildroot.cmake platform definition itself, so that
    we eventually behave like the Linux platform.

We also set CMAKE_SYSTEM_VERSION to 1, and so the real CMAKE_SYSTEM
value should be set to Linux-1 if we were to follow the documentation to
the letter.

However, for Linux, the version does not matter, and in some situations
may even be harmful (that was reported in one of the commits that
introduce Buildroot.cmake and toolchainfile.cmake).

[1] Fluidsynth 0cd44d00e1/CMakeLists.txt (L80)
[2] https://cmake.org/cmake/help/git-master/variable/CMAKE_SYSTEM.html#variable:CMAKE_SYSTEM

Signed-off-by: Frank Vanbever <frank.vanbever@mind.be>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
[Peter: update commit message with description from Yann]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 19:17:32 +02:00
Ricardo Martincoski
1ad5e0a6a6 support/testing: test check-package ignores external.mk
Check external.mk is ignored only when in the root path of a
br2-external.

Add a file called external.mk as a fixture to be used by the test case.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: wrap at 80 columns]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
4323fbaf8d support/testing: add test for check-package
Check the basic usage for check-package.

It can be called using either absolute path, relative path or from PATH.
Files to be checked can be passed with either absolute path or relative
path (also including files in the current directory).

Also check it ignores some special files when checking intree files,
i.e. package/pkg-generic.mk, while still generating warnings for out-of-tree
files when called with -b.
In order to allow the later, add an empty line to the Config.in in the
br2-external being tested so the script does generate a warning.

Catches bug #11271.

More tests can be added later, for example compatibility to Python 3.

Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: wrap at 80 columns where appropriate; merge into a single
class.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
8669243da3 support/testing: allow run-tests to be called from anywhere
Currently run-tests must be called from the Buildroot top directory.

Derive the top directory from the script path, so run-tests can be called from
any path.
As a consequence the test infra will always test the repo it belongs to.

Suggested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-03 17:17:29 +02:00
Ricardo Martincoski
9609a7c2c2 support/testing: test atop with builtin kernel
Use the new builtin armv5 4.19 kernel to test atop.
The atop package cannot be tested using BASIC_TOOLCHAIN_CONFIG because
it needs kernel headers >= 3.14. So use an updated version of it,
copying the config fragment from
support/config-fragments/autobuild/br-arm-full.config

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
650e9903a9 support/testing: drop haveged for python tests
The new builtin armv5 kernel 4.19 uses VirtIORNG to provide entropy, so we
no longer need haveged.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508536
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508537
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508542
https://gitlab.com/buildroot.org/buildroot/-/jobs/264508543

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Peter: reword]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
56e38a705e support/testing: test syslog-ng with builtin kernel
Use the new builtin kernel 4.19 with VirtIORNG to provide entropy to
test syslog-ng.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Ricardo Martincoski
7acb32dabb support/testing: add builtin armv5 kernel 4.19 with entropy
More and more packages being tested by the test infra, e.g. syslog-ng,
need entropy at startup, usually reading from /dev/random.

Some test cases can also depend on a kernel version newer than the
builtin ones already provided by the test infra:
 - 3.11.0 for armv5;
 - 4.0.0 for armv7.

Add a new builtin kernel to be used by such test cases.
Add it for armv5 so most test cases that switch to use this kernel can
keep using BASIC_TOOLCHAIN_CONFIG.
Use the same kernel version and kernel config as qemu_arm_versatile plus
HW_RANDOM_VIRTIO for VirtIORNG to be usable.
Copy the actual binary file from the syslog-ng runtime test at current
master @ 29e1cb8884.

Since there is already a 'kernel-versatile' file on autobuild.buildroot.net
and we must keep it with this name for reproducibility purposes, create a
simple naming convention for newer builtin kernel images and dtb files:
kernel-<defconfig>-<kernel_series_version>
<dtb_name>-<kernel_series_version>.dtb
Pass '-device virtio-rng-pci' to qemu when this kernel is used.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Peter: use this new kernel instead of the old builtin/armv5 kernel]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-03 14:53:05 +02:00
Thomas Petazzoni
228b0b1f00 support/config-fragments/autobuild: update CSV after musl toolchain rename
In commit a589325405
("support/config-fragments/autobuild: rename br-riscv64-musl config"),
the RISC-V 64-bit musl toolchain config snippet was renamed, but the
toolchain.csv file was not updated accordingly.

Due to this, utils/genrandconfig was no longer able to generate any
configuration.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-03 12:57:46 +02:00
Thomas Petazzoni
b96c343b2d support/config-fragments/autobuild: make br-arm-internal-glibc bleeding edge
We do not have any configuration that tests the very bleeding edge gcc
and binutils versions, so let's change br-arm-internal-glibc to use
the latest version of gcc (9.x right now) and binutils (2.32 right
now). The idea is that this defconfig should be updated to the latest
version of gcc and binutils when their version is bumped.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-02 23:33:00 +02:00
Thomas Petazzoni
84f356e375 support/config-fragments/autobuild: update toolchains to 2019.05.1
All toolchains have been rebuilt with Buildroot 2019.05.1. A number of
toolchains are now using Linux headers 5.1 instead of 4.19, because
5.1 is now the default version.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-02 23:32:40 +02:00
Thomas Petazzoni
a589325405 support/config-fragments/autobuild: rename br-riscv64-musl config
For internal toolchains, we have a policy of naming the files with
"internal", to clearly distinguish them from external toolchain
configurations.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-02 23:32:19 +02:00
Francois Perrad
1a49dcb528 support/testing: improve test_luasyslog
Lua has a builtin lsyslog module, so let's test this one as well.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-08-02 17:35:44 +02:00
Mark Corbin
1003af2ebe support/config-fragments: add RISC-V 64-bit musl autobuild config
Add a RISC-V 64-bit autobuild configuration for the internal
toolchain with musl.

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-01 23:55:40 +02:00
Ricardo Martincoski
450863097f support/testing: use virtio-rng-pci to test syslog-ng
Recent versions of syslog-ng need some entropy on startup.
So use VirtIORNG to provide it. In order to accomplish this:
 - build the kernel containing the driver;
 - pass '-device virtio-rng-pci' to qemu.

Use the same kernel version and kernel config as qemu_arm_versatile.
It already has PCI enabled but it does not have HW_RANDOM_VIRTIO, so add
a defconfig fragment to enable the drivers.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/259856394

At the same time, fix a typo (missing '#') that resulted in the
generation of root.tar. This file is not used in the test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-08-01 20:31:54 +02:00
Francois Perrad
0b32cd888e package/luaposix: bump to version 34.1.1
the module std.normalize is no longer a dependency

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-01 18:11:33 +02:00
Victor Huesca
294fc3218c support/scripts/pkg-stats: retrieve packages latest version using processes
The major bottleneck in pkg-stats is the time spent waiting for
answers from remote servers. Two functions involve such communication
with remote servers:

- 'check_package_urls' which checks that each package upstream website
  is up, it is efficient due to the use of process-pools thanks to
  Matt Weber.

- 'check_package_latest_version' which fetches the latest package
  version from release-monitoring, it uses a http-pool but runs
  sequentially.

This patch extends the use of process-pools to 'check_latest_version'.
Due to some limitations of multiprocess callbacks, this patch loses
the overall progress of packages in favour of just the current package
name.

Runtimes for this function are ~3m vs ~25m for the linear version.
Tested on an i7 7500U (2/4 cores/threads @3.5GHz) with 15ms ping.

Note: There have already been work trying to parallelize this function
using threads but there were a failure on some configurations [1].
This implementation rely on a dedicated module already in use on this
script, so it's unlikely to see failure with this version.

[1] http://lists.busybox.net/pipermail/buildroot/2018-March/215368.html

Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-01 18:04:09 +02:00
Victor Huesca
3938afe1b5 support/scripts/pkg-stats: fix flake8 issues
Fixes:
 - blank space before ':'
 - unused 'o' variable left from a previous patch
 - bad continuous alignment

Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-01 18:04:09 +02:00