Commit Graph

847 Commits

Author SHA1 Message Date
Thomas Petazzoni
7dd2f4f3e1 support/testing/tests/download/gitremote: remove trailing whitespace
Fixes the following flake8 warnings:

support/testing/tests/download/gitremote.py:29:67: W291 trailing whitespace
support/testing/tests/download/gitremote.py:30:60: W291 trailing whitespace

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-02-04 17:10:35 +01:00
Ricardo Martincoski
f284b5e7a1 testing/tests/download: add git hash test
Add one test case to ensure the hash is checked for git packages:
 - correct hash;
 - wrong hash;
 - no hash file.

Add required infra:
 - a GitRemote class, that can start a git server in the host machine to
   emulate a remote git server under the control of the test;
 - a new base class, called GitTestBase, that inherits from BRTest and
   must be subclassed by all git test cases.
   Its setUp() method takes care of configuring the build with a
   br2-external, avoiding to hit http://sources.buildroot.net by using
   an empty BR2_BACKUP_SITE. It also avoids downloading not
   pre-installed dependencies (i.e. lzip) every time by calling 'make
   dependencies' using the common dl directory, and it instantiates the
   GitRemote object.

Besides the Python scripts, add some fixtures used during the tests:
 - a br2-external (git-hash) with one package for each part of the test
   case;
 - a static git bare repo (repo.git) to be served using GitRemote class.

Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.

Thanks to Arnout for the hint about how to add a bare repo to test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Arnout: split long line; reorder imports to satisfy flake8]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-04 16:52:58 +01:00
Bryce Ferguson
3f1b965bc3 infra: add force build flag for host dependencies
This commit adds a config option which will force buildroot to
build all host dependencies even if they are already present on the
host system. This may be a desirable option if different hosts are
used to build the same source. In this case, some packages will be
built on one host that are not built on another. This is problematic
if build source archives are cached afterwards for offline builds.

Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Bryce Ferguson <bryce.ferguson@rockwellcollins.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Peter: reword, drop exit 1, reshuffle]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-04 15:52:44 +01:00
Thomas De Schampheleire
8e3c632f7d support/scripts/graph-build-time: replace confusing colors
The color for 'extract' is very similar to the one for 'install-images'.
Both are cyan-like.

Replace the former by a pale blue to make all colors sufficiently distinct.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-04 15:23:27 +01:00
Mathias De Mare
ec757a813b support/scripts/graph-build-time: add download times
Total build time also involves download. Getting a visibility on the impact
of that step can be important for users/admins, e.g. to evaluate different
methods of BR2_PRIMARY_SITE.

Colors used are some kind of purple (primary scheme) and light orange
(alternate scheme).

Signed-off-by: Mathias De Maré <mathias.de_mare@nokia.com>
[ThomasDS: rebase and update colors to avoid confusion]
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-04 15:22:59 +01:00
Adam Duskett
6809f38351 dependencies.sh: Check for a host python version >= 2.7
Older distributions such as CentOS6 come with python2.6, which causes build
failures in packages such as host-libglib2 because they require python2.7 and
above.

host-libglib2 will produce the error message:
/bin/sh: python2.7: command not found

Python2.7 is a hard-coded value in configure.ac. If one changes the value to
just "python," the following stack trace is produced:

Traceback (most recent call last):
  File "./gdbus-2.0/codegen/gdbus-codegen.in", line 55, in <module>
    self.outfile.write(LICENSE_STR.format(config.VERSION))
ValueError    : sys.exit(codegen_main.codegen_main())
zero length field name in format

Instead of supporting an ancient version of Python that had its support ended
in October os 2013, it would be more pragmatic only to support Python2.7 and
above.

Luckily; CentOS6 has the centos-release-scl repository, which allows users to
install python2.7, and Debian 8 comes with Python2.7 already, making this patch
relatively low impact.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[Peter: only look at major.minor to handle x.y.z with z < 10]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-04 13:22:55 +01:00
Ricardo Martincoski
4d5d0124c7 support/testing/infra/basetest: support br2-external
Some upcoming test cases can use one or more br2-external trees as
fixtures that provide packages used only in runtime tests.

Add support for br2-external into the BRTest class. Any test case can
then provide a list of paths for being used as br2-external trees
during the build of the image to test.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Thomas: use named argument for make_extra_opts.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-29 22:16:33 +01:00
Ricardo Martincoski
07bbf1b4dd support/testing/infra/builder: configure and build with make target and environment
Make the builder able to call 'VAR1=1 make VAR2=2 target'.

Allow sending extra parameters to be added to the end of make command
line. Uses for these purposes:
 - to configure a br2-external, using the 'BR2_EXTERNAL="dir" variable.
 - to specify a make target, such as 'foo-source.'

Allow adding variables to the environment when calling make.
These added variables allow a user to override default values from BuildRoot,
such as 'BR2_DL_DIR="dl"'.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-29 22:14:55 +01:00
Ricardo Martincoski
e2b98dfede support/testing: add atop test
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-28 10:00:45 +01:00
Thomas Petazzoni
b1e294cc15 support/scripts/check-host-rpath: document existing functions
As suggested by Arnout Vandecappelle, let's document the
elf_needs_rpath() and check_elf_has_rpath() functions, before we make
them a bit more complicated with per-package directory support.

Suggested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-17 22:38:34 +01:00
Yann E. MORIN
fc8a5f56b9 infra/pkg-cmake: use an obviously-invalid value for CMAKE_SYSTEM_VERSION
In 36568732e4, we expanded toolchain.cmake to also define the value for
CMAKE_SYSTEM_VERSION, as the cmake documentation states that it must be
manually defined when doing cross-compilation [0]:

    When the CMAKE_SYSTEM_NAME variable is set explicitly to enable
    cross compiling then the value of CMAKE_SYSTEM_VERSION must also
    be set explicitly to specify the target system version.

However, the fix in 36568732e4 uses the version of the kernel headers,
assuming that would be the oldest kernel we could run on. Yet, this is
not the case, because glibc (for example) has fallbacks to support
running on kernels older than the headers it was built against.

The cmake official wiki [1] additionally states:

  * CMAKE_SYSTEM_VERSION : optional, version of your target system, not
    used very much.

Folllowed a little bit below, by:

  * CMAKE_TOOLCHAIN_FILE : absolute or relative path to a cmake script
    which sets up all the toolchain related variables mentioned above

    For instance for crosscompiling from Linux to Embedded Linux on PowerPC
    this file could look like this:

        # this one is important
        SET(CMAKE_SYSTEM_NAME Linux)
        #this one not so much
        SET(CMAKE_SYSTEM_VERSION 1)

    [...]

Furthermore, using the kernel headers version can be a bit misleading (as
it really looks like is is the correct version to use when it is not),
while it is obvious that 1 is not really the output of `uname -r` and
thus is definitely not misleading.

Finally, random searches [2] about CMAKE_SYSTEM_VERSION, mostly only
turns up issues related with Windows, Mac-OS, and to a lesser extent,
Android (where it is forcibly set to 1), with issues realted to running
under just Linux (as opposed to Adnroid) mostly non-existent.

Consequently, we revert to using the value that is suggested in the
cmake WiKi, i.e. 1, and which is basically what we also used as a
workaround in the azure-iot-sdk-c paclkage up until d300b1d3b1.

A case were we will need to have a real kernel version, is if we one day
have a cmake-based pacakge that builds and installs a kernel module [3],
because it will need the _running_ kernel version to install it in
/lib/modules/VERSION/, but in that case it will anyway most probably
not be the headers version.

[0] https://cmake.org/cmake/help/v3.8/variable/CMAKE_SYSTEM_VERSION.html
[1] https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling
[2] https://duckduckgo.com/?q=CMAKE_SYSTEM_VERSION
[3] https://stackoverflow.com/questions/38205745/cmake-system-version-not-updated-for-new-kernel

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-16 23:19:09 +01:00
Arnout Vandecappelle (Essensium/Mind)
36568732e4 package/pkg-cmake: add CMAKE_SYSTEM_VERSION to toolchainfile.cmake
Quoting the CMake documentation:

  When the CMAKE_SYSTEM_NAME variable is set explicitly to enable cross
  compiling then the value of CMAKE_SYSTEM_VERSION must also be set
  explicitly to specify the target system version.

Thus, we should also set CMAKE_SYSTEM_VERSION in toolchainfile.cmake. It
is supposed to be set to the value of `uname -r` on the target. We don't
have that exact value available (unless we build the kernel), but the
value of BR2_TOOLCHAIN_HEADERS_AT_LEAST contains the (minimum) version
of the kernel it will run on, so it should be OK for all practical
purposes.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-12 16:06:51 +01:00
Mark Corbin
7a389a6ec8 support/config-fragments: add RISC-V 32-bit to autobuild configs
Add a minimal RISC-V 32-bit autobuild configuration for the
internal toolchain with glibc.

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-10 21:30:21 +01:00
Mark Corbin
ae99fa6d64 arch: add support for RISC-V 32-bit (riscv32) architecture
This enables a riscv32 system to be built with a Buildroot generated
toolchain (gcc >= 7.x, binutils >= 2.30, glibc only).

This requires a custom version of glibc 2.26 from the riscv-glibc
repository. Note that there are no tags in this repository, so the
glibc version just consists of the 40 character commit id string.

Thanks to Fabrice Bellard for pointing me towards the 32-bit glibc
repository and for providing the necessary patch to get it to build.

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-06 14:09:31 +01:00
Thomas De Schampheleire
4d6e20ff46 support/download: fix scp downloads
scp download is broken, because scp is called without filename argument and
only the server is specified. The call is:
    scp <server> <outputfile>

but should be:
    scp <server>/<filename> <outputfile>

Instead of assuming '-u' lists a full URL including filename (which it is
not), align with the wget helper where -u is the server URL and -f gives the
filename.

With this commit, an scp download can work if FOO_SITE_METHOD is explicitly
set to 'scp' and the server does not have a scheme prefix 'scp://'.
The next commit will handle the case where a scheme prefix is present.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: s/URL/URI/, as noticed by Yann.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-03 22:05:43 +01:00
Ricardo Martincoski
e7e30455ef Makefile: offload .gitlab-ci.yml generation
GitLab has severe limitations imposed to triggers.
Using a variable in a regexp is not allowed:
|    only:
|        - /-$CI_JOB_NAME$/
|        - /-\$CI_JOB_NAME$/
|        - /-%CI_JOB_NAME%$/
Using the key 'variables' always lead to an AND with 'refs', so:
|    only:
|        refs:
|            - branches
|            - tags
|        variables:
|            - $CI_JOB_NAME == $CI_COMMIT_REF_NAME
would make the push of a tag not to trigger all jobs anymore.
Inheritance is used only for the second level of keys, so:
|.runtime_test: &runtime_test
|    only:
|        - tags
|tests.package.test_python_txaio.TestPythonPy2Txaio:
|    <<: *runtime_test
|    only:
|        - /-TestPythonPy2Txaio$/
would override the entire key 'only', making the push of a tag not to
trigger all jobs anymore.

So, in order to have a trigger per job and still allow the push of a tag
to trigger all jobs (all this in a follow up patch), the regexp for each
job must be hardcoded in the .gitlab-ci.yml and also the inherited
values for key 'only' must be repeated for every job.
This is not a big issue, .gitlab-ci.yml is already automatically
generated from a template and there will be no need to hand-editing it
when jobs are added or removed.

Since the logic to generate the yaml file from the template will become
more complex, move the commands from the main Makefile to a script.

Using Python or other advanced scripting language for that script would
be the most versatile solution, but that would bring another dependency
on the host machine, pyyaml if Python is used. So every developer that
needs to run 'make .gitlab-ci.yml' and also the docker image used in the
GitLab pipelines would need to have pyyaml pre-installed.
Instead of adding the mentioned dependency, keep using a bash script.

While moving the commands to the script:
 - mimic the behavior of the previous make target and fail on any
   command that fails, by using 'set -e';
 - break the original lines in one command per line, making the diff for
   any patch to be applied to this file to look nicer;
 - keep the script as simple as possible, without functions, just a
   script that executes from the top to bottom;
 - do not perform validations on the input parameters, any command that
   fails already makes the script to fail;
 - do not add an usage message, the script is not intended to be called
   directly.

This patch does not change functionality.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Thomas: make the script output on stdout rather than take the output
file name as second argument.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-09 21:30:24 +01:00
Damien Thébault
bb7c13273f support/download/dl-wrapper: fix urlencode option never being sent to backend
Since commit 38de434123 ("download: fix file:// BR2_PRIMARY_SITE
(download cache)"), the urlencode option is no longer passed to the
download backend, because we use ${backend} instead of
${backend_urlencode}.

We must get the urlencode information from backend_urlencode.

Signed-off-by: Damien Thébault <damien.thebault@vitec.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: rework commit log]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-09 17:16:04 +01:00
Thomas Petazzoni
ba99ccaae4 support/testing: drop docstring in TestPythonPackageBase::test_run()
Having a docstring in the test_run() method clutters the "run-tests
-l" output:

test_run (tests.package.test_python_crossbar.TestPythonPy3Crossbar)
Test a python package. ... ok
[...]
test_run (tests.package.test_python_pexpect.TestPythonPy2Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_pexpect.TestPythonPy3Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy2Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy3Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy2Pynacl)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy3Pynacl)
Test a python package. ... ok

So let's simply drop this docstring that is not particularly useful.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-08 20:50:08 +01:00
Francois Perrad
9d08630128 support/testing: add prosody test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 23:08:59 +01:00
Francois Perrad
bcfbf6437a support/testing: add lua test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:37:10 +01:00
Francois Perrad
2d1f32f084 support/testing: add perl-io-socket-ssl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:46 +01:00
Francois Perrad
35c858bd82 support/testing: add perl-io-socket-multicast test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:40 +01:00
Francois Perrad
d5196b8ee9 support/testing: add perl-encode-decode test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:36 +01:00
Francois Perrad
7046c20476 support/testing: add perl-dbd-mysql test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:30:30 +01:00
Francois Perrad
fe4b91db57 support/testing/tests/packages/test_perl_*: regeneration
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:19:59 +01:00
Thomas Petazzoni
3e19b837f4 support/scripts/graph-depends: introduce MANDATORY_DEPS array
This array will be re-used in another function in a follow-up commit,
so it makes sense to factor it out.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:17:07 +01:00
Thomas Petazzoni
659d45adc4 support/scripts/graph-depends: use proper rootpkg in remove_extra_deps()
The remove_extra_deps() function removes dependencies that we are not
interested in seeing in the dependency graph. It does this for all
packages, except the 'all' package, which on full dependency graphs is
the root of the tree.

However, this doesn't take into account package-specific dependency
graphs (i.e make <pkg>-graph-depends) where the root is not 'all', but
'<pkg>'. Due to this, dependencies on "mandatory deps" were not
visible at all, i.e the toolchain package (and its dependencies) and
the skeleton package (and its dependencies) were not displayed in
package-specific dependency graphs.

To fix this, we use the existing rootpkg variable instead of
hardcoding 'all'.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:16:37 +01:00
Yann E. MORIN
7ee54bd603 support/graph-depends: drop legacy target-exceptions
Currently, we avoid drawing the dependencies that we call 'target
exceptions', becasue they initially were returned by 'show-targets',
when they in fact were not really packages and thus should not be on
the graph.

However, those two exceptions have no longer been reported in the output
of show-targets since we merged very old initial top-level parallel
build way back in 2014, with commit a24877586a (Makefile: add support
for top-level parallel make), where they had been converted into purely
internal rules.

4 years have passed, we can now drop those exceptions from the
graph-depends script.

This concludes the cleanup initiated three years ago with commit
0b32791f00 (graph-depends: remove absent targets from
TARGET_EXCEPTIONS).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-06 22:15:32 +01:00
Francois Perrad
a03f66f431 support/testing: add perl-xml-libxml test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872335]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:52:15 +01:00
Francois Perrad
d9f5748a07 support/testing: add perl-x10 test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872334]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:52:10 +01:00
Francois Perrad
d48dc64c7d support/testing: add perl-mail-dkim test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872333]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:52:05 +01:00
Francois Perrad
1ac4178f2f support/testing: add perl-libwww-perl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872332]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:50:40 +01:00
Francois Perrad
80196db1f4 support/testing: add perl-gdgraph test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872330]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:50:25 +01:00
Francois Perrad
78f4fad3cf support/testing: add perl-class-load test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/124872329]
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:50:21 +01:00
Francois Perrad
e729bf722b support/testing: add perl test
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-03 20:48:49 +01:00
Peter Korsgaard
13c43455a0 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-02 08:16:10 +01:00
Peter Korsgaard
9089a9ff30 Update for 2018.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-01 23:06:49 +01:00
Erico Nunes
2d9566b7b7 boot/grub2: add option to install tools
Add an option to install grub2 support tools to the target.

In the context of Buildroot, some useful target tools provided are
grub2-editenv, grub2-reboot, which provide means to manage the grub2,
environment, boot order, and others.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-26 20:59:33 +01:00
Thomas Petazzoni
eeb6341410 support/scripts/check-host-rpath: split condition on two statements
Inside the check_elf_has_rpath(), we check if the host binary has a
correct RPATH, which should be either an absolute path to
$(HOST_DIR)/lib, or a relative path using $ORIGIN. Those two
conditions are checked in a single statements, but as we are going to
add a third condition, let's split this up a bit:

 - If we have a RPATH to $(HOST_DIR)/lib -> we're good, return 0
 - If we have a RPATH to $ORIGIN/../lib -> we're good, return 0
 - Otherwise, we will exit the loop, and return 1

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:08:57 +01:00
Yann E. MORIN
2218dc85be support/dependencies: add a check for a suitable gzip
Recently, some hash mismatch have been reported, both by users as well
as autobuilder failures, about tarballs generated from git repositories.

This turned out to be caused by users having the 'gzip' command somehow
aliased to 'pigz' (which stand for: parallel implementation of gzip,
which takes advantage of multi-processor system to parallelise the
compression).

Unfortunately, the output of pigz-compressed archives differ from that
of gzip (even though they *are* valid gzip-compressed streams).

Add a dependency check that ensures that gzip is not pigz. If that is
the case, define a conditional dependency to host-gzip, that is used as
a download dependency for packages that will generate compressed files,
i.e. cvs, git, and svn.

Fixes:
    http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Marcin Niestrój <m.niestroj@grinn-global.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 15:14:58 +01:00
Yann E. MORIN
c3af086395 support/dependencies: treat BSD-tar like the other cases
Currently, when we detect that tar is BSD-tar, we fake an unsupported
version (major, minor) and rely on the version check to reject BSD-tar.

There is no reason to use such shenanigans, when we can simply reject it
from the onset.

Simplify the logic:
  - use positive logic in the condition
  - directly exit in error

Also, comment that case like the other cases are commented.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 15:14:50 +01:00
Yann E. MORIN
020206ca57 support/graph-depends: fix package names starting with a non-alpha
Graphviz' dot utility does not like nodes which names does not start
with an ^[[:alpha:]], i.e. 18xx-ti-utils would cause grievance:

    Warning: syntax ambiguity - badly delimited number '18x' in line 4 [...]/graph-depends.dot splits into two tokens
    Warning: syntax ambiguity - badly delimited number '18x' in line 5 [...]/graph-depends.dot splits into two tokens
    Warning: syntax ambiguity - badly delimited number '18x' in line 6 [...]/graph-depends.dot splits into two tokens
    Warning: syntax ambiguity - badly delimited number '18x' in line 7 [...]/graph-depends.dot splits into two tokens

Prefix nodes with an underscore to fix that.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 10:58:40 +01:00
Nasser Afshin
7b87c2ae81 support/kconfig/merge_config.sh: avoid false positive matches from comment lines
We are using empty CONFIG_PREFIX_. This results in false positive match
for comment lines when merging config fragments.

To avoid false positive reports, we use separate sed expressions and
address comment lines explicitly.

This is actually is in the Linux kernel mainline (v4.20-rc2):
6bbe4385d035c6fac56f840a59861a0310ce137b
("kconfig: merge_config: avoid false positive matches from comment lines")

Signed-off-by: Nasser Afshin <Afshin.Nasser@gmail.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 10:11:15 +01:00
Nasser Afshin
8f069a665c support/kconfig/merge_config.sh: fix merging buildroot config files
This patch allows us to define config prefix with CONFIG_ environment
variable.

By setting the proper config prefix, we will have proper 'redundant
configuration warnings' when we use '-r -m' options.

This is actually already in mainline for v4.20-rc1:
2cd3faf87d2d8f6123adf34741b9a7b98828a76f
("merge_config.sh: Allow to define config prefix")

Signed-off-by: Nasser Afshin <afshin.nasser@gmail.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-24 10:11:11 +01:00
Ricardo Martincoski
bac9a78646 support/testing: add python-ubjson tests
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
2979ab9bd5 support/testing: add python-treq tests
Use a simple script to check the basic usage. The target has no https
server, so a connection from in the target to localhost must not
succeed.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
ac010beec5 support/testing: add python-subprocess32 test
Add a simple test case to check the basic usage by calling 'ls' and
checking the output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
d144edb21d support/testing: add python-service-identity tests
Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
74d61681f1 support/testing: add python-pyyaml tests
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
7e69ddc9e7 support/testing: add python-pynacl tests
Add a simple test case that minimally uses the module.
Add haveged to the target to generate enough entropy so pynacl ->
libsodium don't hang waiting for /dev/random.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
cdad4bf6aa support/testing: add python-pexpect tests
Add a simple test case to check the basic usage. Call 'login' and try
wrong user/password, expecting the 'Login incorrect' message.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
756dd5f132 support/testing: add python-passlib tests
Add a simple test case that creates a hash for a password and verifies
it against an incorrect and a correct password.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
9ee9566640 support/testing: add python-constantly tests
Add a simple test case to check the basic usage by creating a class with
two constants.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
887248d354 support/testing: add python-click tests
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
61b4b81c44 support/testing: add python-cbor tests
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
38557a8c63 support/testing: add python-bitstring tests
Add a simple test case to check the basic usage by checking the
corresponding representation of a 12-bit decimal number in hex, binary
and integer.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
66f99555ed support/testing: add python-automat tests
Use a minimal script to check the basic usage by creating and using a
small state machine.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
367c045512 support/testing: add python-attrs tests
Use a minimal script to check the basic usage creating a class with 2
attributes.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
1bd9e9c199 support/testing: add python-argh tests
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-23 22:05:46 +01:00
Ricardo Martincoski
adec30e4f0 support/testing/test_rust: use standard defconfig fragment style
Since commit "2927f412be support/testing: standardize defconfig
fragments style" all other test cases use the same style for defconfig
fragments:
 - start after a backslash;
 - be declared as a multi-line string literal;
 - be indented one level more than the variable that contains it.

Do the same here for consistency.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-20 09:01:08 +01:00
Peter Korsgaard
446a6f91b2 make: configure host-make with host- prefix
Fixes:
http://autobuild.buildroot.net/results/e29/e293aadc692d2ed337881ef2172ddf66a60bc05c/

And many more.

Install as 'host-make' rather than just 'make', as that otherwise confuses a
number of packages when they invoke recursive / sub-make.  The internal job
control logic of GNU make is version dependant, so mixing versions may lead
to issues like:

make[1]: Entering directory `/home/peko/autobuild/instance-0/output/build/boa-0.94.14rc21'
(cd src && make -w --jobserver-fds=5,6 -j)
make: unrecognized option '--jobserver-fds=5,6'

With this rename, only packages explicitly opting in for our host-make
(using the BR2_MAKE / BR2_MAKE_HOST_DEPENDENCY logic) will use it.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-20 08:17:43 +01:00
Yegor Yefremov
ad361c69f6 support/testing: add python-crossbar tests
This test invokes "crossbar version" command, that checks all
dependencies found in setup.py files and prints some system related
information.
Add haveged to the target to generate enough entropy so crossbar ->
pynacl -> libsodium don't hang waiting for /dev/random.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Ricardo: move test script to a separate file, remove Python 2 variant,
 add haveged to target to add entropy and avoid hanging]
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:49 +01:00
Ricardo Martincoski
6a24761c72 support/testing: use TestPythonPackageBase for python-txtorcon
Move the test script to be run on the target from inline in the test
case to a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
ac4a052968 support/testing: use TestPythonPackageBase for python-txaio
Move the test scripts to be run on the target from inline in the test
case to a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
0f98e58cac support/testing: use TestPythonPackageBase for python-twisted
Move the test script to be run on the target from inline in the test
case to a separate file.

Get the base defconfig fragment from the immediate parent class and not
directly from TestPythonBase because it is the correct way of doing
this. This way the base class TestPythonTwisted could even be placed in
a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
ee6b37cf87 support/testing: use TestPythonPackageBase for python-incremental
Move the test script to be run on the target from inline in the test
case to a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
f07994f7d6 support/testing: use TestPythonPackageBase for python-cryptography
Move the test script to be run on the target from inline in the test
case to a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
b5dd9364ee support/testing: use TestPythonPackageBase for python-autobahn
Move the test script to be run on the target from inline in the test
case to a separate file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Ricardo Martincoski
13e3c14ed2 support/testing: create default test case for python packages
Test cases for python packages are very similar among each other: run a
simple script in the target that minimally tests the package.
So create a new helper class named TestPythonPackageBase that holds all
the logic to run a script on the target.

TestPythonPackageBase adds in build time one or more sample scripts to
be run on the target. The test case for the python package must
explicitly list them in the "sample_scripts" property. The test case
then automatically logins to the target, checks the scripts are really
in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt
to make the logfile more friendly, since someone analysing a failure can
easily check the expected script was executed) and then calls the python
interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script
needs more time to run than the default timeout from the test infra
(currently 5 seconds).

A simple test case for a package that only supports Python 2 will look
like this:

|from tests.package.test_python import TestPythonPackageBase
|
|
|class TestPythonPy2<Package>(TestPythonPackageBase):
|    __test__ = True
|    config = TestPythonPackageBase.config + \
|        """
|        BR2_PACKAGE_PYTHON=y
|        BR2_PACKAGE_PYTHON_<PACKAGE>=y
|        """
|    sample_scripts = ["tests/package/sample_python_<package>.py"]
|    timeout = 15

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-13 20:55:45 +01:00
Yann E. MORIN
409d4c3fe9 fs: don't use an intermediate tarball
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.

However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.

Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.

Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:

    File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring

This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.

The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.

The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.

So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216

Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.

[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-13 00:47:36 +01:00
Thomas Petazzoni
bff6b61adf support/testing/tests/fs/test_f2fs: fix flake8 warning
This commit fixes the following flake8 warnings:

support/testing/tests/fs/test_f2fs.py:6:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:12:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:38:23: E225 missing whitespace around operator

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-08 22:41:53 +01:00
Grzegorz Blach
5244fd29a1 support/testing: add test for the f2fs filesystem support
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-08 21:17:51 +01:00
Serj Kalichev
c77cd17082 support/scripts/mkmakefile: make wrapper silent by default
Suppose we use Makefile wrapper and build some project out of
buildroot tree (O=...). A command like "make
busybox-all-external-deps" will output the string "uname 022 && make
..." to stdout before the usefull information. It pollutes stdout. At
the same time if we use the same command in the buildroot source-tree
then we don't get the additional output. This patch makes wrapper
silent by default. People who prefer to see more verbose output can
use V=1.

Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-02 21:23:02 +01:00
Petr Vorel
0d61e067eb kconfig: Refresh patches
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-30 20:58:54 +01:00
Petr Vorel
a3366b2705 kconfig: Restore missing custom changes + add patches
Commit 6eacea5ae0 accidentally removed these changes in merge_config.sh:
0f56304521 ("merge_config.sh: create temporary files in /tmp")
28fac3973b ("merge_config.sh: add br2-external support")

Changes were lost because commits just changed files, but didn't add patches.
Therefore not only restore our changes, but also add (updated) patches.

Missing 0f56304521 caused breaking merge_config.sh when used in out of
tree build:
$ make -C buildroot O=$PWD/output defconfig
...
$ cd output
$ echo 'BR2_TARGET_GENERIC_HOSTNAME="test"' > test.frag
$ ../buildroot/support/kconfig/merge_config.sh .config test.frag
Using .config as base
Merging test.frag
umask 0022 && make -C /home/test/buildroot O=/home/test/output/. alldefconfig
  GEN     /home/test/output/Makefile
*** Can't read seed configuration "./.tmp.config.qIcpASpUyh"!
make[1]: *** [Makefile:925: alldefconfig] Error 1
make: *** [Makefile:16: _all] Error 2

Fixes: 6eacea5ae0 support/kconfig: bump to kconfig from Linux 4.17-rc2

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Reported-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-30 20:58:09 +01:00
Petr Vorel
775087fab0 kconfig: Add missing patch
which was specified, but not added during last update.

Fixes: 6eacea5ae0 ("support/kconfig: bump to kconfig from Linux 4.17-rc2")

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-30 20:57:04 +01:00
Matt Weber
aa2eb4bd64 support/testing: test_hardening fix flake8 whitespace
Resolves:
support/testing/tests/core/test_hardening.py:25:42: E231 missing whitespace after ','

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-20 17:27:28 +02:00
Thomas Petazzoni
b6b766527e support/config-fragments/autobuild: test the ARM AArch64 toolchain
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-20 17:12:27 +02:00
Mark Corbin
be43be070f support/config-fragments: add RISC-V 64-bit to autobuild configs
Add a minimal RISC-V 64-bit autobuild configuration for the
internal toolchain with glibc.

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-20 14:17:23 +02:00
Matt Weber
963f824511 support/testing/tests/core: SSP & hardening flags
Catch the commonly used options of SSP, Relro, and fortify.
Using the package targets of busybox and lighttpd.  This
can easily be expanded to a larger list.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-20 13:25:27 +02:00
Thomas Petazzoni
22c2b27249 support/testing: fix TestATFMarvell test case
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. We update U-Boot and ATF to
what is used in the current version of
solidrun_macchiatobin_mainline_defconfig, for which the problem no
longer exists.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/107860312

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-14 19:38:32 +02:00
Thomas Petazzoni
2a63cea587 support/testing: fix TestATFAllwinner test case
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. Let's update to U-Boot 2017.11,
which is used by our current bananapi_m64_defconfig that was the
inspiration for this test case.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/107860310

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-14 19:38:20 +02:00
Ricardo Martincoski
1009590870 support/testing: add python-twisted tests
Use a minimal script to listen to a port and check using netstat.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: increase the delay after starting the Twisted server, as 5
seconds was not enough for Python 3.x configurations.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 17:28:06 +02:00
Ricardo Martincoski
35bcb54e1d support/testing: add python-incremental tests
Add a simple test case that imports the module and asserts a version
string for a fake package is generated.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 17:27:23 +02:00
Ricardo Martincoski
ddc0ebdb7c support/testing: add python-txtorcon tests
Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 17:27:15 +02:00
Ricardo Martincoski
f389f26691 support/testing: add python-txaio tests
Add a simple test case that imports the module to use with twisted in
Python 2 and with asyncio in Python 3.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 17:27:12 +02:00
Ricardo Martincoski
7e02070914 support/testing: add python-autobahn tests
Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 17:27:04 +02:00
Thomas Petazzoni
b2c24f584d package/pkg-generic.mk: increase precision of timestamps
Currently, the timestamps that we keep in build-time.log use a
second-level precision. However, as we are going to introduce a new
type of graph to draw the time line of a build, this precision is
going to be insufficient, as a number of steps are so short that they
are not even one second long, and generally the rounding to the second
gives a not so great looking graph.

Therefore, we add to the timestamps the nanoseconds using the %N date
specifier. A milli-second precision would have been sufficient, but %N
is all what date(1) provides at the sub-second level.

Since this is changing the format of the build-time.log file, this
commit adjusts the support/scripts/graph-build-time script
accordingly, to account for the floating point numbers that we have as
timestamps.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-10 21:30:51 +02:00
Matt Weber
3b657e264c support/scripts/pkg-stats: URL check using threads
Adds a pool of worker threads to accelerate connection testing.

~7.5MB and 2% CPU per thread on a Intel i5-3230M CPU @ 2.60GHz.

Runtime is ~3min in parallel vs ~15min.

CC: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-09 10:11:47 +02:00
Matt Weber
b4da3b2603 support/scripts/pkg-stats: URL checking support
- Adds support to check if a package has a URL and if that URL
  is valid by doing a header request.
- Reports this information as part of the generated html output

The URL data is currently gathered from the URL string provided
in the Kconfig help sections for each package.

This check helps ensure the URLs are valid and can be used
for other scripting purposes as the product's home site/URL.
CPE XML generation is an example of a case that could use this
product URL as part of an automated update generation script.

CC: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-09 10:05:46 +02:00
Petr Vorel
6eacea5ae0 support/kconfig: bump to kconfig from Linux 4.17-rc2
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-20 23:14:38 +02:00
Johan Oudinet
497f7134fc support/scripts/mkusers: preserve group members
When the function add_one_group is called on an existing group,
make sure the members of this group are not removed in the process of
deleting then re-adding the group.

Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: add curly braces when referencing ${members}, as suggested by
Yann.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-12 21:22:25 +02:00
Yann E. MORIN
63a0593566 download/git: fix code-style
This file uses leading spaces, not TABs.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-10 22:44:26 +02:00
Romain Naour
f6db3df799 dependencies: host-make version check
The host make program is already checked by dependencies.sh but we
want to check the version number even if Buildroot is able to use
GNU make >= 3.81 but some packages may require a more recent version.

For example, since version 2.28 [1], glibc requires GNU Make >= 4.0.

For packages requiring make >= 4.0, the package makefile must use:
<PKG>_DEPENDENCIES = $(BR2_MAKE_HOST_DEPENDENCY) ...
<PKG>_MAKE = $(BR2_MAKE)

BR2_MAKE1 is also available if needed.

[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Thomas: remove extraction of "bugfix" part of the version, since it's
not used anywhere.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-08 23:36:36 +02:00
Peter Korsgaard
721e4cbb52 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-07 13:13:17 +02:00
Peter Korsgaard
339d550e92 Update for 2018.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-06 22:11:06 +02:00
Hollis Blanchard
38de434123 download: fix file:// BR2_PRIMARY_SITE (download cache)
wget is the only downloader currently usable with BR2_PRIMARY_SITE, and that
doesn't work at all for file:// URLs. The symptoms are these:

	support/download/dl-wrapper -c '2.4.47' -d '/PATH/build/sw/source/attr' -D '/PATH/build/sw/source' -f 'attr-2.4.47.src.tar.gz' -H 'package/attr//attr.hash' -n 'attr-2.4.47' -N 'attr' -o '/PATH/build/sw/source/attr/attr-2.4.47.src.tar.gz'  -u file\|urlencode+file:///NFS/buildroot_dl_cache/attr -u file\|urlencode+file:///NFS/buildroot_dl_cache -u http+http://download.savannah.gnu.org/releases/attr -u http\|urlencode+http://sources.buildroot.net/attr -u http\|urlencode+http://sources.buildroot.net  --
	file:///NFS/buildroot_dl_cache/attr/attr-2.4.47.src.tar.gz: Unsupported scheme `file'.
	ERROR: attr-2.4.47.src.tar.gz has wrong sha256 hash:
	ERROR: expected: 25772f653ac5b2e3ceeb89df50e4688891e21f723c460636548971652af0a859
	ERROR: got     : e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
	ERROR: Incomplete download, or man-in-the-middle (MITM) attack

In the case of custom Linux kernel versions, this is fatal, because there isn't
necessarily a hash file to indicate that wget's empty tarball is wrong.

This seems to have been broken by commit c8ef0c03b0, because:
1. BR2_PRIMARY_SITE always appends "urlencode" (package/pkg-download.mk)
2. Anything with the "|urlencode" suffix in $uri will end up using wget due to
   the backend case wildcarding.
3. The wget backend rejects file:/// URLs ("unsupported scheme"), and we end up
   with an empty .tar.gz file in the downloads directory.

Fix that by shell-extracting the backend name from the left of "|". I'm not
positive if all URLs will have a "|", so this code only looks for a "|" left of
the "+".

Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-21 00:24:29 +02:00
Yann E. MORIN
3ab56f8a63 support/dependencies: check for system-provided bison and flex
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-20 16:01:42 +02:00
Angelo Compagnucci
8511a66fc1 support/misc: updating Vagrantfile to ubuntu 18.04
This patch updates the vagrant box to ubuntu bionic 64 and switches back
to the official ubuntu image cause the issues with the official image
are now solved.

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-13 11:45:48 +02:00
Matt Weber
b25c39ebd9 support/testing/infra: rm extra log flush in builder.py
While integrating proxy support in builder.py, a log flush
was left in the code.  This commit cleans/removes that code.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-11 12:38:45 +02:00