Add a branch to the static repo and check on the git refs test case the
download of a git package:
- with a sha1 reachable by a branch name, but not pointed by it, as
version. This is the most common use case for git refs in the tree;
- with a partial sha1 of a commit reachable by a branch as version;
- with a sha1 of the commit head of a branch as version;
- with a partial sha1 of the commit head of a branch as version;
Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
setlocalversion will use 'hg id' to determine whether or not the current
revision is tagged. If there is no tag, the Mercurial revision is printed,
otherwise nothing is printed.
The problem is that the user may have custom configuration settings (in
their ~/.hgrc file or similar) that changes the output of 'hg id' in a way
that the script does not expect. In such cases, the Mercurial revision may
not be printed or printed incorrectly.
It is good practice to ignore the user environment when calling Mercurial
commands from a well-defined script, by setting the environment variable
HGRCPATH to the empty string. See also 'hg help environment'.
In the particular case of Nokia, a custom extension adds dynamic tags in the
repository, i.e. tags that are stored in a file external to the repository
and only visible when the extension is active. These tags should not
influence the behavior of setlocalversion as they are not official Buildroot
tags, i.e. even if a revision is tagged, the Mercurial revision should still
be printed.
Note that this still does not solve the problem where an organization adds
_real_ tags in their Buildroot repository. For example, there might be a
moving tag 'last-validated' or tags indicating in which product release that
Buildroot revision was used. In these cases, setlocalversion will still not
behave as expected, i.e. show the Mercurial revision.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When Buildroot is stored in a Mercurial repository on a branch other than
'default' ('master' in git terms), setlocalversion (used to populate
/etc/os-release) will incorrectly think that this is a tagged version and
will NOT print out the revision hash.
This is due to the fact that the output of 'hg id' is assumed to be
"<revision> <tags-if-any>"
but when on a branch it actually is:
"<revision> (<branch>) <tags-if-any>"
To let setlocalversion receive the output it expects, explicitly ask 'hg id'
to retrieve only the revision hash and any tags, ommitting any branch
information.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.
Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
- trying to download an invalid sha1 generates an error;
- downloading a valid sha1 that contains unexpected content generates
an error.
In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add one test case to ensure the hash is checked for git packages:
- correct hash;
- wrong hash;
- no hash file.
Add required infra:
- a GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test;
- a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using
an empty BR2_BACKUP_SITE. It also avoids downloading not
pre-installed dependencies (i.e. lzip) every time by calling 'make
dependencies' using the common dl directory, and it instantiates the
GitRemote object.
Besides the Python scripts, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each part of the test
case;
- a static git bare repo (repo.git) to be served using GitRemote class.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Arnout: split long line; reorder imports to satisfy flake8]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds a config option which will force buildroot to
build all host dependencies even if they are already present on the
host system. This may be a desirable option if different hosts are
used to build the same source. In this case, some packages will be
built on one host that are not built on another. This is problematic
if build source archives are cached afterwards for offline builds.
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Bryce Ferguson <bryce.ferguson@rockwellcollins.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Peter: reword, drop exit 1, reshuffle]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The color for 'extract' is very similar to the one for 'install-images'.
Both are cyan-like.
Replace the former by a pale blue to make all colors sufficiently distinct.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Total build time also involves download. Getting a visibility on the impact
of that step can be important for users/admins, e.g. to evaluate different
methods of BR2_PRIMARY_SITE.
Colors used are some kind of purple (primary scheme) and light orange
(alternate scheme).
Signed-off-by: Mathias De Maré <mathias.de_mare@nokia.com>
[ThomasDS: rebase and update colors to avoid confusion]
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Older distributions such as CentOS6 come with python2.6, which causes build
failures in packages such as host-libglib2 because they require python2.7 and
above.
host-libglib2 will produce the error message:
/bin/sh: python2.7: command not found
Python2.7 is a hard-coded value in configure.ac. If one changes the value to
just "python," the following stack trace is produced:
Traceback (most recent call last):
File "./gdbus-2.0/codegen/gdbus-codegen.in", line 55, in <module>
self.outfile.write(LICENSE_STR.format(config.VERSION))
ValueError : sys.exit(codegen_main.codegen_main())
zero length field name in format
Instead of supporting an ancient version of Python that had its support ended
in October os 2013, it would be more pragmatic only to support Python2.7 and
above.
Luckily; CentOS6 has the centos-release-scl repository, which allows users to
install python2.7, and Debian 8 comes with Python2.7 already, making this patch
relatively low impact.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[Peter: only look at major.minor to handle x.y.z with z < 10]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some upcoming test cases can use one or more br2-external trees as
fixtures that provide packages used only in runtime tests.
Add support for br2-external into the BRTest class. Any test case can
then provide a list of paths for being used as br2-external trees
during the build of the image to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Thomas: use named argument for make_extra_opts.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow sending extra parameters to be added to the end of make command
line. Uses for these purposes:
- to configure a br2-external, using the 'BR2_EXTERNAL="dir" variable.
- to specify a make target, such as 'foo-source.'
Allow adding variables to the environment when calling make.
These added variables allow a user to override default values from BuildRoot,
such as 'BR2_DL_DIR="dl"'.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As suggested by Arnout Vandecappelle, let's document the
elf_needs_rpath() and check_elf_has_rpath() functions, before we make
them a bit more complicated with per-package directory support.
Suggested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In 36568732e4, we expanded toolchain.cmake to also define the value for
CMAKE_SYSTEM_VERSION, as the cmake documentation states that it must be
manually defined when doing cross-compilation [0]:
When the CMAKE_SYSTEM_NAME variable is set explicitly to enable
cross compiling then the value of CMAKE_SYSTEM_VERSION must also
be set explicitly to specify the target system version.
However, the fix in 36568732e4 uses the version of the kernel headers,
assuming that would be the oldest kernel we could run on. Yet, this is
not the case, because glibc (for example) has fallbacks to support
running on kernels older than the headers it was built against.
The cmake official wiki [1] additionally states:
* CMAKE_SYSTEM_VERSION : optional, version of your target system, not
used very much.
Folllowed a little bit below, by:
* CMAKE_TOOLCHAIN_FILE : absolute or relative path to a cmake script
which sets up all the toolchain related variables mentioned above
For instance for crosscompiling from Linux to Embedded Linux on PowerPC
this file could look like this:
# this one is important
SET(CMAKE_SYSTEM_NAME Linux)
#this one not so much
SET(CMAKE_SYSTEM_VERSION 1)
[...]
Furthermore, using the kernel headers version can be a bit misleading (as
it really looks like is is the correct version to use when it is not),
while it is obvious that 1 is not really the output of `uname -r` and
thus is definitely not misleading.
Finally, random searches [2] about CMAKE_SYSTEM_VERSION, mostly only
turns up issues related with Windows, Mac-OS, and to a lesser extent,
Android (where it is forcibly set to 1), with issues realted to running
under just Linux (as opposed to Adnroid) mostly non-existent.
Consequently, we revert to using the value that is suggested in the
cmake WiKi, i.e. 1, and which is basically what we also used as a
workaround in the azure-iot-sdk-c paclkage up until d300b1d3b1.
A case were we will need to have a real kernel version, is if we one day
have a cmake-based pacakge that builds and installs a kernel module [3],
because it will need the _running_ kernel version to install it in
/lib/modules/VERSION/, but in that case it will anyway most probably
not be the headers version.
[0] https://cmake.org/cmake/help/v3.8/variable/CMAKE_SYSTEM_VERSION.html
[1] https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling
[2] https://duckduckgo.com/?q=CMAKE_SYSTEM_VERSION
[3] https://stackoverflow.com/questions/38205745/cmake-system-version-not-updated-for-new-kernel
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Quoting the CMake documentation:
When the CMAKE_SYSTEM_NAME variable is set explicitly to enable cross
compiling then the value of CMAKE_SYSTEM_VERSION must also be set
explicitly to specify the target system version.
Thus, we should also set CMAKE_SYSTEM_VERSION in toolchainfile.cmake. It
is supposed to be set to the value of `uname -r` on the target. We don't
have that exact value available (unless we build the kernel), but the
value of BR2_TOOLCHAIN_HEADERS_AT_LEAST contains the (minimum) version
of the kernel it will run on, so it should be OK for all practical
purposes.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a minimal RISC-V 32-bit autobuild configuration for the
internal toolchain with glibc.
Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This enables a riscv32 system to be built with a Buildroot generated
toolchain (gcc >= 7.x, binutils >= 2.30, glibc only).
This requires a custom version of glibc 2.26 from the riscv-glibc
repository. Note that there are no tags in this repository, so the
glibc version just consists of the 40 character commit id string.
Thanks to Fabrice Bellard for pointing me towards the 32-bit glibc
repository and for providing the necessary patch to get it to build.
Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
scp download is broken, because scp is called without filename argument and
only the server is specified. The call is:
scp <server> <outputfile>
but should be:
scp <server>/<filename> <outputfile>
Instead of assuming '-u' lists a full URL including filename (which it is
not), align with the wget helper where -u is the server URL and -f gives the
filename.
With this commit, an scp download can work if FOO_SITE_METHOD is explicitly
set to 'scp' and the server does not have a scheme prefix 'scp://'.
The next commit will handle the case where a scheme prefix is present.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: s/URL/URI/, as noticed by Yann.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
GitLab has severe limitations imposed to triggers.
Using a variable in a regexp is not allowed:
| only:
| - /-$CI_JOB_NAME$/
| - /-\$CI_JOB_NAME$/
| - /-%CI_JOB_NAME%$/
Using the key 'variables' always lead to an AND with 'refs', so:
| only:
| refs:
| - branches
| - tags
| variables:
| - $CI_JOB_NAME == $CI_COMMIT_REF_NAME
would make the push of a tag not to trigger all jobs anymore.
Inheritance is used only for the second level of keys, so:
|.runtime_test: &runtime_test
| only:
| - tags
|tests.package.test_python_txaio.TestPythonPy2Txaio:
| <<: *runtime_test
| only:
| - /-TestPythonPy2Txaio$/
would override the entire key 'only', making the push of a tag not to
trigger all jobs anymore.
So, in order to have a trigger per job and still allow the push of a tag
to trigger all jobs (all this in a follow up patch), the regexp for each
job must be hardcoded in the .gitlab-ci.yml and also the inherited
values for key 'only' must be repeated for every job.
This is not a big issue, .gitlab-ci.yml is already automatically
generated from a template and there will be no need to hand-editing it
when jobs are added or removed.
Since the logic to generate the yaml file from the template will become
more complex, move the commands from the main Makefile to a script.
Using Python or other advanced scripting language for that script would
be the most versatile solution, but that would bring another dependency
on the host machine, pyyaml if Python is used. So every developer that
needs to run 'make .gitlab-ci.yml' and also the docker image used in the
GitLab pipelines would need to have pyyaml pre-installed.
Instead of adding the mentioned dependency, keep using a bash script.
While moving the commands to the script:
- mimic the behavior of the previous make target and fail on any
command that fails, by using 'set -e';
- break the original lines in one command per line, making the diff for
any patch to be applied to this file to look nicer;
- keep the script as simple as possible, without functions, just a
script that executes from the top to bottom;
- do not perform validations on the input parameters, any command that
fails already makes the script to fail;
- do not add an usage message, the script is not intended to be called
directly.
This patch does not change functionality.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Thomas: make the script output on stdout rather than take the output
file name as second argument.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since commit 38de434123 ("download: fix file:// BR2_PRIMARY_SITE
(download cache)"), the urlencode option is no longer passed to the
download backend, because we use ${backend} instead of
${backend_urlencode}.
We must get the urlencode information from backend_urlencode.
Signed-off-by: Damien Thébault <damien.thebault@vitec.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: rework commit log]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Having a docstring in the test_run() method clutters the "run-tests
-l" output:
test_run (tests.package.test_python_crossbar.TestPythonPy3Crossbar)
Test a python package. ... ok
[...]
test_run (tests.package.test_python_pexpect.TestPythonPy2Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_pexpect.TestPythonPy3Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy2Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy3Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy2Pynacl)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy3Pynacl)
Test a python package. ... ok
So let's simply drop this docstring that is not particularly useful.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This array will be re-used in another function in a follow-up commit,
so it makes sense to factor it out.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The remove_extra_deps() function removes dependencies that we are not
interested in seeing in the dependency graph. It does this for all
packages, except the 'all' package, which on full dependency graphs is
the root of the tree.
However, this doesn't take into account package-specific dependency
graphs (i.e make <pkg>-graph-depends) where the root is not 'all', but
'<pkg>'. Due to this, dependencies on "mandatory deps" were not
visible at all, i.e the toolchain package (and its dependencies) and
the skeleton package (and its dependencies) were not displayed in
package-specific dependency graphs.
To fix this, we use the existing rootpkg variable instead of
hardcoding 'all'.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, we avoid drawing the dependencies that we call 'target
exceptions', becasue they initially were returned by 'show-targets',
when they in fact were not really packages and thus should not be on
the graph.
However, those two exceptions have no longer been reported in the output
of show-targets since we merged very old initial top-level parallel
build way back in 2014, with commit a24877586a (Makefile: add support
for top-level parallel make), where they had been converted into purely
internal rules.
4 years have passed, we can now drop those exceptions from the
graph-depends script.
This concludes the cleanup initiated three years ago with commit
0b32791f00 (graph-depends: remove absent targets from
TARGET_EXCEPTIONS).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add an option to install grub2 support tools to the target.
In the context of Buildroot, some useful target tools provided are
grub2-editenv, grub2-reboot, which provide means to manage the grub2,
environment, boot order, and others.
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Inside the check_elf_has_rpath(), we check if the host binary has a
correct RPATH, which should be either an absolute path to
$(HOST_DIR)/lib, or a relative path using $ORIGIN. Those two
conditions are checked in a single statements, but as we are going to
add a third condition, let's split this up a bit:
- If we have a RPATH to $(HOST_DIR)/lib -> we're good, return 0
- If we have a RPATH to $ORIGIN/../lib -> we're good, return 0
- Otherwise, we will exit the loop, and return 1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Recently, some hash mismatch have been reported, both by users as well
as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow
aliased to 'pigz' (which stand for: parallel implementation of gzip,
which takes advantage of multi-processor system to parallelise the
compression).
Unfortunately, the output of pigz-compressed archives differ from that
of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is
the case, define a conditional dependency to host-gzip, that is used as
a download dependency for packages that will generate compressed files,
i.e. cvs, git, and svn.
Fixes:
http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Marcin Niestrój <m.niestroj@grinn-global.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, when we detect that tar is BSD-tar, we fake an unsupported
version (major, minor) and rely on the version check to reject BSD-tar.
There is no reason to use such shenanigans, when we can simply reject it
from the onset.
Simplify the logic:
- use positive logic in the condition
- directly exit in error
Also, comment that case like the other cases are commented.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Graphviz' dot utility does not like nodes which names does not start
with an ^[[:alpha:]], i.e. 18xx-ti-utils would cause grievance:
Warning: syntax ambiguity - badly delimited number '18x' in line 4 [...]/graph-depends.dot splits into two tokens
Warning: syntax ambiguity - badly delimited number '18x' in line 5 [...]/graph-depends.dot splits into two tokens
Warning: syntax ambiguity - badly delimited number '18x' in line 6 [...]/graph-depends.dot splits into two tokens
Warning: syntax ambiguity - badly delimited number '18x' in line 7 [...]/graph-depends.dot splits into two tokens
Prefix nodes with an underscore to fix that.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
We are using empty CONFIG_PREFIX_. This results in false positive match
for comment lines when merging config fragments.
To avoid false positive reports, we use separate sed expressions and
address comment lines explicitly.
This is actually is in the Linux kernel mainline (v4.20-rc2):
6bbe4385d035c6fac56f840a59861a0310ce137b
("kconfig: merge_config: avoid false positive matches from comment lines")
Signed-off-by: Nasser Afshin <Afshin.Nasser@gmail.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch allows us to define config prefix with CONFIG_ environment
variable.
By setting the proper config prefix, we will have proper 'redundant
configuration warnings' when we use '-r -m' options.
This is actually already in mainline for v4.20-rc1:
2cd3faf87d2d8f6123adf34741b9a7b98828a76f
("merge_config.sh: Allow to define config prefix")
Signed-off-by: Nasser Afshin <afshin.nasser@gmail.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. The target has no https
server, so a connection from in the target to localhost must not
succeed.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by calling 'ls' and
checking the output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that minimally uses the module.
Add haveged to the target to generate enough entropy so pynacl ->
libsodium don't hang waiting for /dev/random.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage. Call 'login' and try
wrong user/password, expecting the 'Login incorrect' message.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that creates a hash for a password and verifies
it against an incorrect and a correct password.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by creating a class with
two constants.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by checking the
corresponding representation of a 12-bit decimal number in hex, binary
and integer.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a minimal script to check the basic usage by creating and using a
small state machine.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a minimal script to check the basic usage creating a class with 2
attributes.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since commit "2927f412be support/testing: standardize defconfig
fragments style" all other test cases use the same style for defconfig
fragments:
- start after a backslash;
- be declared as a multi-line string literal;
- be indented one level more than the variable that contains it.
Do the same here for consistency.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
http://autobuild.buildroot.net/results/e29/e293aadc692d2ed337881ef2172ddf66a60bc05c/
And many more.
Install as 'host-make' rather than just 'make', as that otherwise confuses a
number of packages when they invoke recursive / sub-make. The internal job
control logic of GNU make is version dependant, so mixing versions may lead
to issues like:
make[1]: Entering directory `/home/peko/autobuild/instance-0/output/build/boa-0.94.14rc21'
(cd src && make -w --jobserver-fds=5,6 -j)
make: unrecognized option '--jobserver-fds=5,6'
With this rename, only packages explicitly opting in for our host-make
(using the BR2_MAKE / BR2_MAKE_HOST_DEPENDENCY logic) will use it.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test invokes "crossbar version" command, that checks all
dependencies found in setup.py files and prints some system related
information.
Add haveged to the target to generate enough entropy so crossbar ->
pynacl -> libsodium don't hang waiting for /dev/random.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Ricardo: move test script to a separate file, remove Python 2 variant,
add haveged to target to add entropy and avoid hanging]
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test scripts to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Get the base defconfig fragment from the immediate parent class and not
directly from TestPythonBase because it is the correct way of doing
this. This way the base class TestPythonTwisted could even be placed in
a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Test cases for python packages are very similar among each other: run a
simple script in the target that minimally tests the package.
So create a new helper class named TestPythonPackageBase that holds all
the logic to run a script on the target.
TestPythonPackageBase adds in build time one or more sample scripts to
be run on the target. The test case for the python package must
explicitly list them in the "sample_scripts" property. The test case
then automatically logins to the target, checks the scripts are really
in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt
to make the logfile more friendly, since someone analysing a failure can
easily check the expected script was executed) and then calls the python
interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script
needs more time to run than the default timeout from the test infra
(currently 5 seconds).
A simple test case for a package that only supports Python 2 will look
like this:
|from tests.package.test_python import TestPythonPackageBase
|
|
|class TestPythonPy2<Package>(TestPythonPackageBase):
| __test__ = True
| config = TestPythonPackageBase.config + \
| """
| BR2_PACKAGE_PYTHON=y
| BR2_PACKAGE_PYTHON_<PACKAGE>=y
| """
| sample_scripts = ["tests/package/sample_python_<package>.py"]
| timeout = 15
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit fixes the following flake8 warnings:
support/testing/tests/fs/test_f2fs.py:6:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:12:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:38:23: E225 missing whitespace around operator
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Suppose we use Makefile wrapper and build some project out of
buildroot tree (O=...). A command like "make
busybox-all-external-deps" will output the string "uname 022 && make
..." to stdout before the usefull information. It pollutes stdout. At
the same time if we use the same command in the buildroot source-tree
then we don't get the additional output. This patch makes wrapper
silent by default. People who prefer to see more verbose output can
use V=1.
Signed-off-by: Serj Kalichev <serj.kalichev@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 6eacea5ae0 accidentally removed these changes in merge_config.sh:
0f56304521 ("merge_config.sh: create temporary files in /tmp")
28fac3973b ("merge_config.sh: add br2-external support")
Changes were lost because commits just changed files, but didn't add patches.
Therefore not only restore our changes, but also add (updated) patches.
Missing 0f56304521 caused breaking merge_config.sh when used in out of
tree build:
$ make -C buildroot O=$PWD/output defconfig
...
$ cd output
$ echo 'BR2_TARGET_GENERIC_HOSTNAME="test"' > test.frag
$ ../buildroot/support/kconfig/merge_config.sh .config test.frag
Using .config as base
Merging test.frag
umask 0022 && make -C /home/test/buildroot O=/home/test/output/. alldefconfig
GEN /home/test/output/Makefile
*** Can't read seed configuration "./.tmp.config.qIcpASpUyh"!
make[1]: *** [Makefile:925: alldefconfig] Error 1
make: *** [Makefile:16: _all] Error 2
Fixes: 6eacea5ae0 support/kconfig: bump to kconfig from Linux 4.17-rc2
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Reported-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
which was specified, but not added during last update.
Fixes: 6eacea5ae0 ("support/kconfig: bump to kconfig from Linux 4.17-rc2")
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Resolves:
support/testing/tests/core/test_hardening.py:25:42: E231 missing whitespace after ','
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a minimal RISC-V 64-bit autobuild configuration for the
internal toolchain with glibc.
Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Catch the commonly used options of SSP, Relro, and fortify.
Using the package targets of busybox and lighttpd. This
can easily be expanded to a larger list.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. We update U-Boot and ATF to
what is used in the current version of
solidrun_macchiatobin_mainline_defconfig, for which the problem no
longer exists.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/107860312
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. Let's update to U-Boot 2017.11,
which is used by our current bananapi_m64_defconfig that was the
inspiration for this test case.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/107860310
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Use a minimal script to listen to a port and check using netstat.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: increase the delay after starting the Twisted server, as 5
seconds was not enough for Python 3.x configurations.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module and asserts a version
string for a fake package is generated.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module to use with twisted in
Python 2 and with asyncio in Python 3.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, the timestamps that we keep in build-time.log use a
second-level precision. However, as we are going to introduce a new
type of graph to draw the time line of a build, this precision is
going to be insufficient, as a number of steps are so short that they
are not even one second long, and generally the rounding to the second
gives a not so great looking graph.
Therefore, we add to the timestamps the nanoseconds using the %N date
specifier. A milli-second precision would have been sufficient, but %N
is all what date(1) provides at the sub-second level.
Since this is changing the format of the build-time.log file, this
commit adjusts the support/scripts/graph-build-time script
accordingly, to account for the floating point numbers that we have as
timestamps.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Adds a pool of worker threads to accelerate connection testing.
~7.5MB and 2% CPU per thread on a Intel i5-3230M CPU @ 2.60GHz.
Runtime is ~3min in parallel vs ~15min.
CC: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Adds support to check if a package has a URL and if that URL
is valid by doing a header request.
- Reports this information as part of the generated html output
The URL data is currently gathered from the URL string provided
in the Kconfig help sections for each package.
This check helps ensure the URLs are valid and can be used
for other scripting purposes as the product's home site/URL.
CPE XML generation is an example of a case that could use this
product URL as part of an automated update generation script.
CC: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When the function add_one_group is called on an existing group,
make sure the members of this group are not removed in the process of
deleting then re-adding the group.
Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: add curly braces when referencing ${members}, as suggested by
Yann.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This file uses leading spaces, not TABs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The host make program is already checked by dependencies.sh but we
want to check the version number even if Buildroot is able to use
GNU make >= 3.81 but some packages may require a more recent version.
For example, since version 2.28 [1], glibc requires GNU Make >= 4.0.
For packages requiring make >= 4.0, the package makefile must use:
<PKG>_DEPENDENCIES = $(BR2_MAKE_HOST_DEPENDENCY) ...
<PKG>_MAKE = $(BR2_MAKE)
BR2_MAKE1 is also available if needed.
[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[Thomas: remove extraction of "bugfix" part of the version, since it's
not used anywhere.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
wget is the only downloader currently usable with BR2_PRIMARY_SITE, and that
doesn't work at all for file:// URLs. The symptoms are these:
support/download/dl-wrapper -c '2.4.47' -d '/PATH/build/sw/source/attr' -D '/PATH/build/sw/source' -f 'attr-2.4.47.src.tar.gz' -H 'package/attr//attr.hash' -n 'attr-2.4.47' -N 'attr' -o '/PATH/build/sw/source/attr/attr-2.4.47.src.tar.gz' -u file\|urlencode+file:///NFS/buildroot_dl_cache/attr -u file\|urlencode+file:///NFS/buildroot_dl_cache -u http+http://download.savannah.gnu.org/releases/attr -u http\|urlencode+http://sources.buildroot.net/attr -u http\|urlencode+http://sources.buildroot.net --
file:///NFS/buildroot_dl_cache/attr/attr-2.4.47.src.tar.gz: Unsupported scheme `file'.
ERROR: attr-2.4.47.src.tar.gz has wrong sha256 hash:
ERROR: expected: 25772f653ac5b2e3ceeb89df50e4688891e21f723c460636548971652af0a859
ERROR: got : e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
ERROR: Incomplete download, or man-in-the-middle (MITM) attack
In the case of custom Linux kernel versions, this is fatal, because there isn't
necessarily a hash file to indicate that wget's empty tarball is wrong.
This seems to have been broken by commit c8ef0c03b0, because:
1. BR2_PRIMARY_SITE always appends "urlencode" (package/pkg-download.mk)
2. Anything with the "|urlencode" suffix in $uri will end up using wget due to
the backend case wildcarding.
3. The wget backend rejects file:/// URLs ("unsupported scheme"), and we end up
with an empty .tar.gz file in the downloads directory.
Fix that by shell-extracting the backend name from the left of "|". I'm not
positive if all URLs will have a "|", so this code only looks for a "|" left of
the "+".
Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch updates the vagrant box to ubuntu bionic 64 and switches back
to the official ubuntu image cause the issues with the official image
are now solved.
Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
While integrating proxy support in builder.py, a log flush
was left in the code. This commit cleans/removes that code.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Allow builder.py to inherit the system proxy settings from
the env if they are present.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The download wrapper is a purely internal helper, and is not supposed to
be callable manually. No need to offer some help.
Besides, the help text was way out-dated.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
With cmake packages, we are only using TARGET_LDFLAGS for executables
and not for shared libraries.
This patch adds CMAKE_SHARED_LINKER_FLAGS and
CMAKE_MODULE_LINKER_FLAGS to the cmake toolchain file so that
buildroot TARGET_LDFLAGS are used for shared and module libraries.
Signed-off-by: Damien Thébault <damien.thebault@vitec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add the i686 package list to install when using pre-built 32 bits
binaries with a redhat/fedora host distribution (glibc.i686 and
zlib.i686).
Signed-off-by: David De Grave (Essensium/Mind) <david.degrave@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
All pre-built Buildroot toolchains have been rebuilt with Buildroot
2018.05, so this commit updates the corresponding configuration
fragments to make sure the autobuilders use the new toolchains.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, we install flake8 and its dependencies via pip. We
tried to be reproducible by pinning the version of those python
packages, but we did forget quite a few of them, and thus some
dependencies for flake8 are installed as uncontrolled versions.
Furthermore, before we install flake8 and its dependencies, we
forcibly update pip, setuptools, and wheels packages to their
latest versions. This explicitly breaks reproducibility.
While we could enforce a specific version of all those packages
and still grab them from PyPI, we can simply grab them from the
distribution-provided packages instead.
Since we're using a pinned version of stretch, this already
guarantees we'll reproducibly get the same versions over and
over again. Besides, we just need to list flake8 as a package to
install to automatically get all its dependencies (again, in a
reproducible way).
This has the slight unfortunate drawback of downgrading flake8
to version 3.2.1, from version 3.5.0, as well as downgrading a
few of flake8's dependencies, as noticed by Ricardo:
http://lists.busybox.net/pipermail/buildroot/2018-May/222376.html
However, as Ricardo said, there isn't "any serious limitation of
this old version, the release notes for a version in the between
mentions 'Dramatically improve the performance' but we have a
limited number of scripts and running on Gitlab for all of them
still takes less than 5 minutes".
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As suggested in the docker best practices [0], order the package list
alphabetically, and list only one package per line.
This will be much usefull later, we need to update the list of installed
packages, like adding new ones for example.
[0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fix three issues with code style in our test infra:
- 'print' is now a function,
- exceptions need to be caught-assigned with the 'as' keyword,
- old-style "%s"%() formatting is deprecated.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: drop indices in format strings.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
'+' is a valid character in a url. The current dl-wrapper gets the
URI scheme by dropping everything after the last '+' character, with
the intension of finding 'git' from e.g. 'git+https://uri'.
If a uri has a '+' anywhere in it, it ends up using too much of the
string as a scheme, and fails to match the handler properly.
An example of where this form of URI is used is when using deploy tokens
in gitlab. It uses a form like https://<username>:<password>@gitlab.com/<group>/<repo.git>
where username for deploy token is of the form 'gitlab+deploy-token-<number>'.
Use the %% operator to search backwards until the last '+' character when
dropping the rest of the string as we know that the first '+'
in the string should be the scheme.
Signed-off-by: Robert Beckett <bbeckett@netvu.org.uk>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In commit 7517aef4d (support/docker: limit the number of layers),
we reduced the number of layers by coalescing multiple RUN commands
into less commands.
In doing so, we especially coalesced "apt-get update" with "apt-get
install".
However, the distribution we used is a pinned version of stretch, so
we know that running apt-get update will always yield the same apt
database.
If we split the two apt-get commands, then we can re-use any local
intermediate image when we need to update the list of packages to
install; this helps quite a bit when testing the docker files over
and over again, with just slight variants in the packages list.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use Python 3 style print calls, in order to make pkg-stats Python 3
compliant.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since the rework of the download infrastructure, the "file" download
helper gets passed an URL that starts with file://, but forgets to
strip it before passing it to "cp", causing a failure as the "cp"
program isn't prepared for file paths starting with file://. This is
fixed by stripping the file:// at the beginning of the URL.
In addition, the path passed to cp lacked a slash between the
directory path and the filename part of the url. This is fixed by
adding a slash at the appropriate places.
Fixes the following build failure when the "file" download method is
used:
cp: cannot stat 'file:///home/angelo/DEV/TOOLCHAINSarmv7-eabihf--glibc--bleeding-edge-2017.11-1.tar.bz2': No such file or directory
Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
A person on IRC reported a build failure with the util-linux package,
looking like this:
for I in uname26 linux32 linux64 ; do \
cd /home/aep/consulting/chargery/tracker/output/target/usr/bin && ln -sf setarch $I ; \
done
[...]
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
The issue was an empty path in the PATH variable, which means "current
working directory", causing a "ln" binary built by util-linux for the
target to be used instead of the system-provided "ln".
We already check a number of things in the PATH and LD_LIBRARY_PATH
variables in support/dependencies/dependencies.sh, but we were not
checking that PATH did not contain an empty path.
This commit fixes that and takes this opportunity to simplify the test
code for PATH and LD_LIBRARY_PATH.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Thomas: improve commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
We currently attempt a shallow clone, as tentative to save bandwidth and
download time.
However, now that we keep the git tree as a cache, it may happen that we
need to checkout an earlier commit, and that would not be present with a
shallow clone.
Furthermore, the shallow fetch is already really broken, and just
happens to work by chance. Consider the following actions, which are
basically what happens today:
mkdir git
git init git
cd git
git remote add origin https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
git fetch origin --depth 1 v4.17-rc1
if ! git fetch origin v4.17-rc1:v4.17-rc1 ; then
echo "warning"
fi
git checkout v4.17-rc1
The checkout succeeds just because of the git-fetch in the if-condition,
which is initially there to fetch the special refs from github PRs, or
gerrit reviews. That fails, but we just print a warning. If we were to
ever remove support for special refs, then the checkout would fail.
The whole purpose of the git cache is to actually save bandwidth and
download time, but in the long run. For one-offs, people would
preferably use a wget download (e.g. with the github macro) instead of
a git clone.
We switch to always doing a full clone. It is more correct, and pays off
in the long run...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When a git tree has had sub-dir <-> sub-module conversions, or has had
submodules added or removed over the course of time, checking out a
changeset across those conversions/additions/removals may leave
untracked files, or may fail because of a conflict of type.
So, before we checkout the new changeset, we forcibly remove the
submodules. The new set of submodules, if any, will be restored later.
Ideally, we would use a native git command: git submodule deinit --all.
However, that was only introduced in git 1.8.3 which, while not being
recent by modern standards, is still too old for some enterprise-grade
distributions (RHEL6 only has git-1.7.1).
So, instead, we just use git submodule foreach, to rm -rf the submodules
directory.
Again, we would ideally use 'cd $toplevel && rm -rf $path', but
$toplevel was only introduced in git 1.7.2. $path has always been there.
So, instead, we just cd back one level, and remove the basename of the
directory.
Eventually, we need to get rid of now-empty and untracked directories,
that were parents of a removed submodule. For example. ./foo/bar/ was a
submodule, so ./foo/bar/ was removed, which left ./foo/ around.
Yet again, recent-ish git versions would have removed it during the
forced checkout, but old-ish versions (e.g. 1.7.1) do not remove it with
the forced checkout.
Instead we rely on the already used forced-forced clean of directories,
untracked, and ignored content, to really get rid of extra stuff we are
not interested in.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Force the checkout to ignore and throw away any local changes. This
allows recovering from a previous partial checkout (e.g. killed by
the user, or by a CI job...)
git checkout -f has been supported since the inception of git, so we
can use it without any second thought.
Also do a forced-forced clean, to really get rid of all untracked stuff.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In some cases, the repository may be in a state we can't automatically
recover from, especially since we must still support oldish git versions
that do not provide the necessary commands or options thereof.
As a last-ditch recovery, delete the repository and recreate the cache
from scratch.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Check that the given cset is indeed something we can checkout. If not,
then exit early.
This will be useful when a later commit will trap any failing git
command to try to recover the repository by doing a clone from scratch:
when the cset is not a commit, it does not mean the repository is broken
or what, and re-cloning from scratch would not help, so no need to trash
a good cache.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
That way, we can pushd earlier, which will help with last-ditch recovery
in a followup commit.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
We really want the user not to use our git cache manually, or their
changes (committed or not) may eventually get lost.
So, add a warning file, not unlike the one we put in the target/
directory, to warn the user not to use the git tree.
Ideally, we would have carried this file in support/misc/, but the git
backend does not have access to it: the working directory is somewhere
unknown, and TOPDIR is not exported in the environment.
So, we have to carry it in-line in the backend instead.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When the run-time tests to build rust and rust-bin packages are run via Docker,
the $USER environment variable is not set, which makes cargo fail when
initializing the test project.
So add it to make cargo happy.
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This script causes a large number of flake8 warnings, is rarely used
(but even never used), and is going to be replaced at some point by
the improved pkg-stats that will give details about the upstream
version available for all packages, not just X.org packages.
Therefore, let's drop the xorg-release script in order to silence all
those flake8 warnings:
support/scripts/xorg-release:36:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:58:27: E201 whitespace after '{'
support/scripts/xorg-release:58:44: E203 whitespace before ':'
support/scripts/xorg-release:58:54: E202 whitespace before '}'
support/scripts/xorg-release:63:1: E305 expected 2 blank lines after class or function definition, found 1
support/scripts/xorg-release:64:15: E261 at least two spaces before inline comment
support/scripts/xorg-release:67:32: E261 at least two spaces before inline comment
support/scripts/xorg-release:86:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:95:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:107:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:115:20: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:123:34: E201 whitespace after '{'
support/scripts/xorg-release:124:46: E203 whitespace before ':'
support/scripts/xorg-release:124:50: E202 whitespace before '}'
support/scripts/xorg-release:127:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:141:15: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:146:21: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:176:1: E305 expected 2 blank lines after class or function definition, found 1
support/scripts/xorg-release:180:1: W391 blank line at end of file
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The version of the ARM Trusted Firmware from Marvell was a Git branch,
not a Git commit, leading to unreproducible results. So let's use a
Git commit instead, which is the latest available from the branch that
was previously used.
More specifically, this branch has recently seen a fix that is needed
for ATF to build properly with recent gcc versions:
c96ec59f8b
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test case currently fails to build with:
./build/juno/release/bl1/context_mgmt.o: In function `cm_prepare_el3_exit':
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): undefined reference to `cm_set_next_context'
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): relocation truncated to fit: R_AARCH64_JUMP26 against undefined symbol `cm_set_next_context'
This issue has been fixed upstream in commit
10c252c14b7f446c0b49ef1aafbd5d37804577dd, available since v1.3. So
while we bump, let's bump to the latest version of ATF, v1.5.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/64360659
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since commit 5563a1c6a4
("support/check-uniq-files: support weird locales and filenames"), the
'csv' Python module is no longer used by the check-uniq-files.
Due to this, flake8 complains with:
support/scripts/check-uniq-files:4:1: F401 'csv' imported but unused
Fix this by dropping the useless csv import.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit updates all the pre-built Buildroot toolchains, which have
all been rebuilt with Buildroot as of commit 046c5e2. The initial
motivation for this update is that an upcoming bump of procps-ng uses
fopencookie(), which has only been introduced in musl 1.1.19, which
itself started being used in Buildroot after the 2018.02 release.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Older versions of tar (e.g. 1.27.1) incorrectly interpret the escaping
of the regexp separator, and generate broken tarballs.
For example, given the following transform expression:
--transform="s/^\.\//squashfs-e38956b92f738518c29734399629e7cdb33072d3\//"
the resulting paths in the generated tarball would be:
squashfs-e38956b92f738518c29734399629e7cdb33072d3\/
i.e. a directory which last character is indeed a '\'.
We fix that by using a separator which is very unlikely to occur in a
filename.
Fixes:
http://autobuild.buildroot.org/results/742/7427f34e5c9f6d043b0fe6ad2c66cc0f31d2b24f/
and probably a slew of others as well...
Take this opportunity to fix indentation on the following line
(leading spaces, not TABs).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Check:
- the daemon is started;
- a message is collected;
- the daemon does not issue a warning message on startup.
When the .conf file version does not match the package version a warning
message shows up on serial on every boot. This message is generated by
syslog-ng before it is running, so it is not logged to
/var/log/messages. So in order to test the message is generated, restart
the server. It makes the message appears on /var/log/messages (since the
server is already running) where its existence can be easily tested
using grep.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The different versions of git will behave in different ways when
fetching remote references, as summarised by the table below:
| ancient git | new git
--------------------------------------------------------------------
git fetch | fetch all refs but tags | fetches all refs but tags
git fetch -t | fetches only tags | fetch all refs and tags
(git-fetch may still fetch tags, but only if reachable from a branch)
So, to cover all the bases, we do a simple fetch, to be sure we have
branches, followed by the existing fetch -t, to get extra tags.
Fixes:
http://autobuild.buildroot.net/results/0a2/0a238a7f55ea56c33b639ad03ed5796143426889/build-end.log
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
There are cases where a repository might be broken, e.g. when a previous
operation was killed or otherwise failed unexpectedly.
We fix that by always initialising the repository, as suggested by
Ricardo. git-init is safe on an otherwise-healthy repository:
Running git init in an existing repository is safe. It will not
overwrite things that are already there. [...]
Using git-init will just ensure that we have the strictly required files
to form a sane tree. Any blob that is still missing would get fetched
later on.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
git always look directories up until it finds a repository. In case
the git cache is broken, it may no longer be identified as a repository,
and git will look higher in the directories until it finds one.
In the default conditions, this would be Buildroot's own git tree
(because DL_DIR is a subdir of Buildroot), but in some situations may
very well be any repository the user has Buildroot in, like a
br2-external tree...
So, we force git to use our git cache and never look elsewhere, as
Suggested by Ricardo.
Use GIT_DIR, as it has been there for ages now, while --git-dir was
only introduced later (even if most distros ship an later version),
as suggested by Arnout.
Also fix the one call to git that was not using the wrapper.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In preparation for the removal of the Blackfin architecture, drop the
autobuilder toolchain configuration that was testing Blackfin.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
${raw_name} is never defined in dl-wrapper, and therefore the value
passed to the -N option is always empty. This causes a problem for the
'cvs' backend, which uses the value of this option as the CVS module
to be downloaded.
If the name of the CVS module is omitted, all the CVS modules from
that CVS repository are downloaded, which creates a tarball with a lot
more contents, and the actual useful contents in a sub-directory,
obviously breaking patches that should be applied, and the entire
build process that follows.
Fixes:
http://autobuild.buildroot.net/results/fcee0e3d7eeeb373313b1794092c729b1b052348/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When calling the backend-specific helper scripts, the remaining
options are in ${@}. However, in order to let the helper script know
that those remaining options should not be parsed, but instead passed
as-is to the download tool, they must be separated from the main
options by "--".
Without this, packages that use <pkg>_DL_OPTS, such as the
amd-catalyst package, cannot download their tarball anymore.
Fixes:
http://autobuild.buildroot.net/results/de818f6e4c8e63d5e8a49c445d10c34eccc40410/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The /lib/firmware directory contains random firmware for various
devices. It happens that some of them might be or appear to be ELF
files, but they shouldn't be checked by fix-rpath. For example, one of
the Qualcomm VPU firmware file appears to be an ELF file, but patchelf
isn't happy about it:
$ ./output/host/bin/patchelf --print-rpath output/target/lib/firmware/qcom/venus-4.2/venus.b00
patchelf: patchelf.cc:387: void ElfFile<Elf_Ehdr, Elf_Phdr, Elf_Shdr, Elf_Addr, Elf_Off, Elf_Dyn, Elf_Sym>::parse() [with Elf_Ehdr = Elf32_Ehdr; Elf_Phdr = Elf32_Phdr; Elf_Shdr = Elf32_Shdr; Elf_Addr = unsigned int; Elf_Off = unsigned int; Elf_Dyn = Elf32_Dyn; Elf_Sym = Elf32_Sym]: Assertion `shstrtabIndex < shdrs.size()' failed.
Aborted (core dumped)
Even though patchelf definitely shouldn't crash, it anyway doesn't
make sense to check ELF files in /lib/firmware, so let's exclude this
directory from our check.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In case the git backend gets killed right in-between it finished
initialising the repository, but before it could add the remote,
we'd end up with a repository without the 'origin' remote, so we
would not be able to change its URL.
Another case that may happen (like in the build failure, below),
is that the repository was initialised with a previous version
of Buildroot, before the commit e17719264b (download/git: don't
require too-recent git) was applied, and that trepository was
still lying around...
Fixes:
http://autobuild.buildroot.org/results/25a/25aae054634368fadb265b97ebe4dda809deff6f/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
git has supported -C only since 1.8.5, and some distros have not yet
caught up after more than 4 years...
Fall back to entering the directory.
Fixes:
http://autobuild.buildroot.net/results/35f9f7a4adc6c2cad741079e4afdf1408c94703b
Reported-by: André Hentschel <nerv@dawncrow.de>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: André Hentschel <nerv@dawncrow.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a package contains a relative symlink which first component is '..'
(thus pointing one directory higher), for example package 'meh' contains
this symlink:
foo/bar -> ../buz
then it would be stored as 'meh-version./buz' because of the
transform-name pattern replacement.
Fix it to only match the leading './'.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since c8ef0c03b0 (download: put most of the infra in dl-wrapper), the
backend for local files is now named after the scheme, which is 'file'
for a local file.
>From the same commit on, the directory part and the basename are now
passed separately, to let the backend reconstruct the full path when it
needs to do so, which is the case for the 'file' backend too.
Finaly, ff559846fd (support/download: Add support to pass options
directly to downloaders) introduced a nasty error, as it made use of
"${@}" when calling its internal function. Revert that mess now...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit "6d938bcb52 download: git: introduce cache feature" introduced a
typo that makes the tarball to contain files without the package
basename:
$ tar -tvf good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
-rw-r--r-- 0/0 8 2017-10-14 02:10 ./file
Historically, all tarballs are generated with the basename:
$ tar -tvf good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
-rw-r--r-- 0/0 8 2017-10-14 02:10 good-a238b1dfcd825d47d834af3c5223417c8411d90d/file
The hashes in the tree were calculated with the basename.
In the most common scenario, after the download ends the tarball is
generated, the hash mismatches and the download mechanism falls back to
use the tarball from http://sources.buildroot.net .
The problem can be reproduced by forcing the download of any git package
PKG that has a hash file to check against:
$ make defconfig
$ ./utils/config --set-str BR2_BACKUP_SITE ""
$ BR2_DL_DIR=$(mktemp -d) make PKG-dirclean PKG-source
Fix the typo so the basename is really added to the files, that was
clearly the intention of the code.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit adds a new column in the HTML output containing the
current version of a package in Buildroot. As such, it isn't terribly
useful, but combined with the latest upstream version added in a
follow-up commit, it will become very useful.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit adds the following options to the pkg-stats-new script:
-n, to specify a number of packages to parse instead of all packages
-p, to specify a list of packages (comma-separated) to parse instead
of all packages
These options are basically only useful when debugging/developing
this script, but they are very useful, because the script is rather
slow to run completely with all 2000+ packages, especially once
upstream versions will be fetched from release-monitoring.org.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit adds a new version of the pkg-stats script, rewritten in
Python. It is for now implemented in a separate file called,
pkg-stats-new, in order to make the diff easily readable. A future
commit will rename it to pkg-stats.
Compared to the existing shell-based pkg-stats script, the
functionality and output is basically the same. The main difference is
that the output no longer goes to stdout, but to the file passed as
argument using the -o option. This allows stdout to be used for more
debugging related information.
The way the script works is that a first function get_pkglist()
returns a list of Package objects. Then, the function
package_init_make_info() uses 'make printvars' to gather information
about all packages, stored as class variables in the Package
class. Then, we iterate over all packages, and use various methods of
the Package class to retrieve all details about the package:
infrastructure, presence of hash file, presence of license
information, etc.
calculate_stats() then calculates global statistics (how packages have
license information, how packages have a hash file, etc.). Finally,
dump_html() produces the HTML output, using a number of sub-functions.
One improvement over the shell-based version is that we can use
regexps to exclude some .mk files. Thanks to this, we can exclude all
linux-ext-*.mk files, avoiding incorrect matches.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
When the BR2_DL_DIR is a mountpoint (presumably shared between various
machine, or mounted from the local host when running in a VM), it is
possible that it does not support hardlinks (e.g. samba, or the VMWare
VMFS, etc...).
If the hardlink fails, fallback to copying the file. As a last resort,
if that also fails, eventually fallback to doing the download.
Note: this means that the dl-wrapper is no longer atomic-safe: the code
suffers of a TOCTTOU condition: the file may be created in-between the
check and the moment we try to ln/cp it. Fortunately, the dl-wrapper is
now run under an flock, so we're still safe. If we eventually go for a
more fine-grained implementation, we'll have to be careful then.
Reported-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now we keep the git clone that we download and generates our tarball
from there.
The main goal here is that if you change the version of a package (say
Linux), instead of cloning all over again, you will simply 'git fetch'
from the repo the missing objects, then generates the tarball again.
This should speed the 'source' part of the build significantly.
The drawback is that the DL_DIR will grow much larger; but time is more
important than disk space nowadays.
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For existing setups, the global donload directory may have a lot of the
required archives, so look into there before attempting a download.
We simply hard-link them if found there and not in the new per-package
loaction. Then we resume the existing procedure (which means the new
hardlink will get removed if it happened to not match the hash).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The infrastructure needs to give the 'dl_dir' to the dl-wrapper which in its
turn needs to give it to the helper. It will only be used by the 'git'
helper as of now.
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e
tar 1.27 subtly changed the tar format when a GNU long link entry is added
(which is done for path elements > 100 characters). The code used to set
the permission mode of the link entry to 0:
header = start_private_header ("././@LongLink", size, time (NULL));
FILL (header->header.mtime, '0');
FILL (header->header.mode, '0');
FILL (header->header.uid, '0');
FILL (header->header.gid, '0');
FILL (header->header.devmajor, 0);
FILL (header->header.devminor, 0);
This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with
negative and out-of-range integers), so the settings from
start_private_header() are used directly - Which are:
TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),
header->header.mtime);
MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode);
UID_TO_CHARS (0, header->header.uid);
GID_TO_CHARS (0, header->header.gid);
The end result is that tar >= 1.27 sets mode to 644.
The consequence of this is that we create different tar files when long path
names are encountered (which often happens when a package downloads a
specific sha1 from a git repo) depending on the host tar version used,
causing hash mismatches.
As a workaround, bump our minimum tar version to 1.27. It would be nicer to
only do this if we have packages from bzr/git/hg enabled, but that is an
exercise for later.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The goal here is to simplify the infrastructure by putting most of the
code in the dl-wrapper as it is easier to implement and to read.
Most of the functions were common already, this patch finalizes it by
making the pkg-download.mk pass all the parameters needed to the
dl-wrapper which in turn will pass everything to every backend.
The backend will then cherry-pick what it needs from these arguments
and act accordingly.
It eases the transition to the addition of a sub directory per package
in the DL_DIR, and later on, a git cache.
[Peter: drop ';' in BR_NO_CHECK_HASH_FOR in DOWNLOAD macro and swap cd/rm
-rf as mentioned by Yann, fix typos]
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently all download helpers accepts the local output file, the remote
locations, the changesets and so on... as positional arguments.
This was well and nice when that's was all we needed.
But then we added an option to quiesce their verbosity, and that was
shoehorned with a trivial getopts, still keeping all the existing
positional arguments as... positional arguments.
Adding yet more options while keeping positional arguments will not be
very easy, even if we do not envision any new option in the foreseeable
future (but 640K ought to be enough for everyone, remember? ;-) ).
Change all helpers to accept a set of generic options (-q for quiet and
-o for the output file) as well as helper-specific options (like -r for
the repository, -c for a changeset...).
Maxime:
Changed -R to -r for recurse (only for the git backend)
Changed -r to -u for URI (for all backend)
Change -R to -c for cset (for CVS and SVN backend)
Add the export of the BR_BACKEND_DL_GETOPTS so all the backend wrapper
can use the same option easily
Now all the backends use the same common options.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
graph-depends currently spits out a graph in .dot format. However, as
part of the upcoming introduction of <pkg>-show-recursive-depends and
<pkg>-show-recursive-rdepends, we need graph-depends to be able to
display a flat list.
Signed-off-by: George Redivo <george.redivo@datacom.ind.br>
[Thomas:
- Rebase on top of graph-depends changes
- Do not display the package name itself in the list, only its
dependencies (or reverse dependencies)
- Display the result on a single line, instead of one package per
line, in order to match what <pkg>-show-depends and
<pkg>-show-rdepends are doing today.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This will be useful for the upcoming recursive show-depends and
show-rdepends features.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Instead of hardcoded sys.stderr.write() calls. No functional change, but
allows us to easily implement a quiet option.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The graph-depends was not very consistent in colors vs. colours: some
parts were using colours, some parts were using colors.
Let's settle on the US spelling, colors.
This change the user-visble option --colours to --colors, but it is
unlikely that a lot of users customize the colors through
BR2_GRAPH_DEPS_OPTS, so this user interface change is considered
reasonable.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The graph-depends script had no main() function, and the main code was
actually spread between the function definitions, which was a real
mess.
This commit moves the global code into a main() function, which allows
to more easily follow the flow of the script. The argument parsing
code is moved into a parse_args() function.
Most of the global variables are removed, and are instead passed as
argument when appropriate. This has the side-effect that the
print_pkg_deps() function takes a lot of argument, but this is
considered better than tons of global variables.
The global variables that are removed are: max_depth, transitive,
mode, root_colour, target_colour, host_colour, outfile, dict_deps,
dict_version, stop_list, exclude_list, arrow_dir.
The root_colour/target_colour/host_colour variables are entirely
removed, and instead a single colours array is passed, and it's the
function using the colors that actually uses the different entries in
the array.
The way the print_attrs() function determines if we're display the
root node is not is changed. Instead of relying on the package name
and the mode (which requires passing the root package name, and the
mode), it relies on the depth: when the depth is 0, we're at the root
node.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some test cases don't use a full build as setup, so split the build()
method into configure() and build().
It allows a test case to perform configuration at the setup stage and
the build inside the test itself.
Call this new method just before build in the BRTest base class, to keep
the current behavior for existing test cases.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use an empty environment when calling make, but import PATH so the
buildroot tree under test can find binaries from the host machine.
Since environment variables are now ignored, move the handling of
BR2_DL_DIR to the defconfig to keep the current precedence of -d:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(DIR)
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(DIR)
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit reorganizes the toolchain-configs.csv so that the first
toolchains are a subset of "useful" toolchains to be tested by
contributors to validate a package. This subset is the one that will
be used by default by test-pkg.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In the situation where the hash is missing from the hash file, the
dl-wrapper downloads the file again and again until the developer
specifies the hash to complete the download step.
To avoid this situation, the freshly-downloaded file is not removed
anymore after a successful download.
After this change, the behaviour is as follows:
- Hash file doesn't exist, or file is in BR_NO_CHECK_HASH_FOR
=> always succeeds.
- Hash file exists, but file is not present
=> file is NOT removed, build is terminated immediately (i.e.
secondary site is not tried).
- Hash file exists, file is present, but hash mismatch
=> file is removed, secondary site is tried.
=> If all primary/secondary site downloads or hash checks fail, the
build is terminated.
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
[Arnout: extend commit log]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.
However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.
We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.
This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, when a filename contains characters not representable in the
user's locale, we fail hard, especially when the host python is python3.
This is because python2 and python3 handle encoding/decoding strings
differently, with python3 presumable doing the right thing, but it
breaks on some systems, while python2 presumable does the wrong thing,
but it works everywhere. (Just joking, obviously...)
Part of the issue being that the csv reader in python2 is broken with
UTF8.
We fix the issue by ditching the csv reader, and simply read the file in
binary mode, manually partitioning the lines on the first comma.
Then, we use the binary-encoded (really, un-encoded) package names and
filenames as values and keys, respectively.
Finally, for each filename or package we need to print, we try to decode
them with the defaults for the user settings, but catch any decoding
exception and fall back to dumping the raw, binary values. Which codec
is used by default differs between Python version, but in all cases
something sane is printed at least.
Thanks a lot to Arnout for the live help doing this patch. :-)
Reported-by: Jaap Crezee <jaap@jcz.nl>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Jaap Crezee <jaap@jcz.nl>
[Arnout: commit log improvement]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Commit c868095681 ("toolchain: fix
detection of SSP support") fixed the SSP check so that it does the
correct thing for nios2 toolchains. While this commit fixed the
description of the Sourcery NIOSII toolchain, it didn't fix the
description for the autobuilders of the br-nios2-glibc toolchain,
causing some build failures. This commit adjusts br-nios2-glibc.config
to indicate that the toolchain doesn't have SSP support.
Fixes:
http://autobuild.buildroot.net/results/6c44e328b7bffd8474d29d5bdf1ea109ec15f4ad/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:
- The extractor packages host-tar, host-xz and host-lzip
- host-ccache itself
- host-skeleton, because all packages depend on it
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:
- host-lzip, because we would otherwise depend on ourself.
- host-tar, because lzip itself is delivered as a tarball, so we need
to have host-lzip depend on host-tar, and not host-tar depend on
host-lzip
- host-skeleton, because we need to have host-lzip depend on
host-skeleton, and not the opposite.
We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.
In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.
[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-xz if building a host-xz is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:
- host-xz, because we would otherwise depend on ourself.
- host-tar, because xz itself is delivered as a tarball, so we need
to have host-xz depend on host-tar, and not host-tar depend on
host-xz
- host-skeleton, because we need to have host-xz depend on
host-skeleton, and not the opposite.
In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.
[Peter: fix s/host-tar/host-xz/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.
In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.
Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Use the latest version of the tool because it is actively maintained.
But use a fixed version of the tool and its dependencies to get stable
results. It can be manually bumped from time to time.
Before installing any Python packages, ensure pip, setuptools, and wheel
are up to date as recommended in the docs [1].
[1] https://packaging.python.org/tutorials/installing-packages/
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When using a merged /usr, the kernel module path is really
/usr/lib/modules, as /lib is a symlink to usr/lib .
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some packages (mostly, out-of-tree) may want to install binary blobs for
another architecture, outside the locations we currently exclude, like
in /opt or whatever...
Add support in check-bin-arch to accept any arbitrary location, that
individual package can each request to excude from the check, when they
are installed.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Recent systemd bump has broken DBus dameon and DBus applications can no
longer find the daemon. So we want to catch those kind of failures
early.
We also want to check that the system as a whole is stable: no unit
should be failed.
Finally, ensure that we can read the jounrnal, even when we are doing our
tricks on read-only systems.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Its use has been globbed into the more generic
BR2_NEEDS_HOST_UTF8_LOCALE option now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Julius Kriukas <julius@kriukas.lt>
Cc: Christian Stewart <kidovate@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Although the UTF-8 locales in mainstream distributions all are suffixed
with just 'utf8', the nomenclature is a bit ambiguous with the way they
are to be specified with the various LC_* variables, suffixed there with
'UTF-8'.
Also, POSIX, ISO, and IEC do not enforce any specific suffix in LC_*
variables:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02
"""
If the locale value has the form:
language[_territory][.codeset]
it refers to an implementation-provided locale, where settings of
language, territory, and codeset are implementation-defined.
"""
To avoid any confusion, use a regexp that is a bit more lax when
matching locales.
Also, quote the regexp, so that the '?' and '$' are not interpreted by
the shell.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The script support/scripts/check-uniq-files uses the argparse Python
module. In most recent Python versions (starting with 2.7), the
argparse module is part of the standard library, and we already check
for the availability of Python in
support/dependencies/dependencies.sh.
However, when running on an ancient distribution with Python 2.6, the
argparse module is not part of the Python standard library, but
available as an external module. Without this module, the build fails,
because check-uniq-files, which is used in target-finalize, fails to
run.
To avoid this failure, this commit adds a check in
support/dependencies/dependencies.sh to verify that the argparse
module is available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Using {} in format strings is only supported in sufficiently recent
Python versions. Python 2.6 doesn't support this, and only format
strings with numbered arguments: {0}, {1}, etc.
Python 2.7:
$ python -c 'print("foo {}".format(12))'
foo 12
$ python -c 'print("foo {0}".format(12))'
foo 12
Python 2.6:
$ python -c 'print("foo {}".format(12))'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: zero length field name in format
$ python -c 'print("foo {0}".format(12))'
foo 12
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When BR2_REPRODUCIBLE is enabled or host uses non UTF-8 capable locale
building systemd fails with an error:
[1/1080] Generating systemd.bg.catalog with a meson_exe.py custom command.
FAILED: catalog/systemd.bg.catalog
/buildroot/output/host/bin/python3 /buildroot/output/host/bin/meson --internal exe /buildroot/output/build/systemd-236/build/meson-private/meson_exe_sed_232a0623cc7ce2cd67ec72ed784b76307102ed76.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1079: ordinal not in range(128)
package/pkg-generic.mk:247: recipe for target '/buildroot/output/build/systemd-236/.stamp_built' failed
make: *** [/buildroot/output/build/systemd-236/.stamp_built] Error 1
This patch changes default host system locale from C to C.UTF-8 when
building systemd package to fix this issue. It also introduces
BR2_NEEDS_HOST_C_UTF8_LOCALE flag that checks if this locale is available on
the host system. If locale is not available error message is show and build
process is stopped.
Signed-off-by: Julius Kriukas <julius@kriukas.lt>
[Thomas: use C.UTF-8 instead of en_US.UTF-8.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tar 1.30 changed the --numeric-owner output for filenames > 100 characters,
leading to hash mismatches for the tar archives we create ourselves from
git. This is really a fix for a bug in earlier tar versions regarding
deterministic output, so it is unlikely to be reverted in later versions.
For more details, see:
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html
To work around this issue, blacklist tar 1.30+ similar to how we do it for
pre-1.17 versions so Buildroot falls back to building host-tar.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so
we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ
before these in case they are both needed, otherwise the tools will fail to
extract.
With the upcoming change to blacklist modern tar versions this situation is
likely to trigger more often.
The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework,
but that series is a bit too intrusive to add this close to 2018.02, so
therefore this hack.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The script check-bin-arch fails as follows on a config for PowerPC e6500
(64-bit CPU) with BR2_ARCH="powerpc" (32-bit userland desired):
ERROR: architecture for "/lib/modules/..../lib/libcrc32c.ko"
is "PowerPC64", should be "PowerPC"
This situation is perfectly acceptable: the kernel is 64-bit and so are its
modules, even though userland is 32-bit.
To keep check-bin-arch and its caller simple, just skip /lib/modules/
entirely, like is done for /lib/firmware and some others.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
in turn breaks the job in GitLab.
TestRustBase is not a test case by itself, so it can't have a method
with the name starting with "test_" otherwise nose2 assumes it is a test
case.
Move the test_run method from the base class to the derived classes.
While at it, update .gitlab-ci.yml with the new test cases.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035
Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some packages build C++ 32bits host-tools and need the g++-multilib to
be installed on the build machine. As example, qt5webengine builds a C++
host-tool when target is 32bits.
Add the check for g++-multilib to the dependencies script; and update
the Dockerfile to install g++-multilib package.
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
While we have several defconfigs building internal toolchains with
uClibc, we don't have any building internal toolchain with glibc and
musl. However, having such defconfigs is nice when we bump the C
library version, in order to immediately get feedback on build
failures.
Note that while the ARC internal defconfig uses glibc, it uses the
special ARC glibc version, so it doesn't test version bumps of the
upstream glibc C library.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To test the support for the Rust language, the following tests are added:
- building Rust compiler and Cargo from source.
- installing a pre-built Rust compiler and building Cargo from source.
For each test, a Rust test program is built and installed in the root file
system of a ARM vexpress QEMU system. The test is declared OK if the program can
be run properly from the test system.
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we refer to the latest version of the image, which means we
can't guarantee any reproducibility. Also, it measn we can't have a
separate images for the maintenance branches (especially the LTS) and
master.
Update the comment in the Dockerfile to create and push tagged images.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since we're now using a specific base image tag, we need to also use a
specific, stable repository to get additional packages from for this
image.
As such, use the Debian snapshot that matches the base image.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we are using debian:stable, which is subject to change with
time, as new stable versions of Debian are released/updated.
Use the latest tagged stable release, stretch-20171210 as of today, as
the base distribution to use.
This will ease reproducible builds in the future.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This image is not built very often, and when it is, it is important to
see what's going on, so don't be silent when installing packages from
the distro, and since that can take a bit of time it thus serves as
progress report...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The official documentation [0] suggests limiting the number of layers
generated from a dockerfile. A layer is created for each RUN (and COPY
and ADD) command. But we are only ever interested in the final image,
so the intermediate layers are useless to us.
Limit the number of RUN commands to limit the number of generated
layers.
[0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fix these warnings:
E122 continuation line missing indentation or outdented
E302 expected 2 blank lines, found 1
F401 'os' imported but unused
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E713 test for membership should be 'not in'
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix these warnings:
E201 whitespace after '['
E202 whitespace before ']'
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
Ignore these warnings:
E402 module level import not at top of file
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix these warnings:
E128 continuation line under-indented for visual indent
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix these warnings:
E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
E202 whitespace before ']'
E221 multiple spaces before operator
E225 missing whitespace around operator
E231 missing whitespace after ','
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E502 the backslash is redundant between brackets
E713 test for membership should be 'not in'
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Pass BR2_EXTERNAL value via -e option.
This will prevent merge_config.sh from silently eating any symbols defined in
external trees on a clean buildroot tree invocation.
Signed-off-by: Ilya Kuzmich <ilya.kuzmich@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, only post-build and post-image scripts were tested, each with
their own test-script.
The two test-scripts only differ in the name of the log file they
create, and it is based on the name of the script, so it is easy to
share the script.
This allows us to easily re-use it for testing post-fakeroot scripts.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Buildroot's "make nconfig" command stopped working a while ago on
Gentoo systems. Running the command would result in a crash.
The issue is caused by lxdialog's cflags which are also used to build
nconfig; It would detect *ncursesw* and turn on WIDECHAR support --
but the Makefile would still link to plain *ncurses* while building
nconfig (which was built without WIDECHAR support).
This would cause a crash after using *wattrset* on a WINDOW instance.
WIDECHAR *wattrset* would try to set the _color member in the WINDOW
struct which does not exist in the NON-WIDECHAR ncurses instance. It
would end up clobbering data outside the struct (usually _line entries).
An upstream patch fixes the issue, so we're applying it to Buildroot's
kconfig.
Signed-off-by: Guillermo A. Amaral <g@maral.me>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, our jobs on the gitlab-ci infra are running as root, which is
problematic for two reasons:
- this is not the usual way Buildroot is built;
- it may miss issues where running as non-root is problematic.
So, complement our Dockerfile with directives to add a new user and run
everything as that user, as demonstrated by this build job:
https://gitlab.com/ymorin/buildroot-ci/-/jobs/46929562
Additional, enforce an UTF-8 locale while running.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
BR2_TOOLCHAIN_BUILDROOT_INET_IPV6 has been removed with the commit [1].
Since this option is still in br-powerpc-internal-full config-fragment,
the powerpc configuration is droped by autobuild-run script:
WARN: toolchain can't be used
Missing: BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
[Sat, 06 Jan 2018 03:03:43] WARN: failed to generate configuration
[1] 4bcacfd2c0
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 4932c8a7cc introduced the
core-dependencies target to make sure that the dependencies.sh script
runs before we attempt to compile any host tool, so that the absence of
a compiler is properly detected. However, this relied on the
left-to-right evaluation of dependencies. This will no longer be true
when we enable top-level parallel build.
Fix this by letting DEPENDENCIES_HOST_PREREQ depend on
core-dependencies.
Note that it is not possible to remove the
dependencies <- core-dependencies. Indeed, it is possible that
DEPENDENCIES_HOST_PREREQ is completely empty, and in that case we still
need to check core-dependencies.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
dependencies.sh uses HOSTCC_NOCCACHE directly, and this variable is
exported from the top-level Makefile, so there is no need to pass
HOSTCC to it. HOSTCC is not used at all in dependencies.sh.
Thus, we also no longer need to apply the HOSTCC override for
core-dependencies. The core-depencies rule doesn't use HOSTCC or
HOSTCXX.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Make sure that the pie charts produced by 'graph-build' and 'graph-size'
targets are sorted on the size of each piece of the pie. Otherwise, making
visual analysis is difficult, as one needs to look at the legends of each
piece and do the sorting manually in their head.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It needs mkzftree from zisofs-tools, so we add a dependency to it, and
we call that one explicitly (to avoid using the one from the host in
PATH).
It also needs the the uncompressed kernel image, but because it is
already in target/ so it gets compressed by mkzftree. We have two
options:
- compress everything but the kernel image,
- compress everything, kernel included, and recopy it later.
We choose the latter, because it is the simplest solution. So, we always
define the kernel-copy hook, but only register it when needed.
Finally, it needs a kernel with support for transparent
(de)compression, so we update the existing test config.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: really set the transparent compression option, take into
account the renaming of the option.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Marvell U-Boot needs DTC to build, so the test case fails to build
on systems where dtc isn't available. We add
BR2_TARGET_UBOOT_NEEDS_DTC=y to make sure that we build our own DTC.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/44126707
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
These new tests only do build tests, but allow to quickly verify that
the ATF/U-Boot combinations for vexpress, Allwinner and Marvell
platforms all continue to build properly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This merges the next branch accumulated during the 2017.11 release
cycle back into the master branch.
A few conflicts had to be resolved:
- In the DEVELOPERS file, because Fabrice Fontaine was added as a
developer for libupnp in master, and for libupnp18 in
next. Resolution is simple: add him for both.
- linux/Config.in, because we updated the 4.13.x release used by
default in master, while we moved to 4.14 in next. Resolution: use
4.14.
- package/libupnp/libupnp.hash: a hash for the license file was added
in master, while the package was bumped into next. Resolution: keep
the hash for the license file, and keep the hash for the newest
version of libupnp.
- package/linux-headers/Config.in.host: default version of the kernel
headers for 4.13 was bumped to the latest 4.13.x in master, but was
changed to 4.14 in next. Resolution: use 4.14.
- package/samba4/: samba was bumped to 4.6.11 in master for security
reasons, but was bumped to 4.7.3 in next. Resolution: keep 4.7.3.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some packages (ex: skeleton-init-systemd) have a zero size so we cannot
divide by the package size. In that case make their percent zero
explicitly and avoid a ZeroDivisionError exception.
Signed-off-by: Andrey Yurovsky <yurovsky@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we do nothing about packages that touch the same file: given
a specific configuration, the result is reproducible (even though it
might not be what the user expected) because the build order is
guaranteed.
However, when we later introduce top-level parallel build, we will no
longer be able to guarantee a build order, by the mere way of it being
parallel. Reconciliating all those modified files will be impossible to
do automatically. The only way will be to refuse such situations.
As a preliminary step, introduce a helper script that detects files that
are being moified by two or more packages, and reports them and the
impacted packages, at the end of the build.
The list being reported at the end of the build will make it prominently
visible in autobuilder results, so we can assess the problem, if any.
Later on, calling that helper script can be done right after the package
installation step, to bail out early.
Thanks Arnout for the pythonist way to write default dictionaries! ;-)
Note: doing it in python rather than a shell script is impressively
faster: where the shell script takes ~1.2s on a minimalist build, the
python script only takes ~0.015s, that is about 80 times faster.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Peter Seiderer <ps.report@gmx.net>
[Thomas: rename script without .py extension.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of overriding the _svn command and injecting --non-interactive,
change the default value of BR2_SVN to include this flag so the end user
can choose not to use the flag.
This change helps users behind corporate system rules which may not
allow them to locally cache credentials and require interactive mode.
Signed-off-by: Sam Voss <sam.voss@rockwellcollins.com>
[Originally implemented by]
CC: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is needed to get the autobuilders to use a toolchain that
includes the fix merged in 9d544feb8a
("fwup: fix for ARC toolchain").
In addition, this new toolchain version also fixes for real the RPATH
issue that should have been fixed by
f90f28a6df, but wasn't done properly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
All Buildroot pre-built toolchains have been rebuilt with Buildroot
2017.11-rc1, so that they have the latest version of
glibc/musl/uClibc, and also the latest gcc/binutils updates.
Specifically, this will fix the build failures on Blackfin that were
due to the missing accept4() support:
http://autobuild.buildroot.net/results/8b5a72dd7cde685f6f68f46aeee8b1b60c96d559/
(openobex)
http://autobuild.buildroot.net/results/b19dd9ed29944d7f79c6f824669e3baaa0bb045a/
(libiio)
In terms of changes to the toolchains:
- AArch64 glibc toolchain changed to use 4.4 kernel headers instead
of 4.1, in order to increase the variety of kernel header versions
being tested.
- Most configurations now use 4.13 kernel headers instead of 4.12
(except the configurations that were explicitly using an older
kernel headers version)
- The mips64 n64 configuration is changed from using gcc 4.9 to gcc
5, since another ARM configuration already tests gcc 4.9.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixes:
http://autobuild.buildroot.org/results/2af/2af7412846c576089f8596857ab8c81ac31c1bed/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: André Hentschel <nerv@dawncrow.de>
Reviewed-by: André Hentschel <nerv@dawncrow.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, HOSTCC and HOSTCXX are set to their _NOCACHE variants in the
'dependencies' target. This is needed because at that time, ccache is
not built yet - host-ccache is one of the dependencies. However, because
this override is only specified for the 'dependencies' target (and
thereby gets inherited by its dependencies), the override is only
applied when the package is reached through the 'dependencies' target.
This is not the case when one of DEPENDENCIES_HOST_PREREQ is built
directly from the command line, e.g. when doing 'make host-ccache'. So
in that case, ccache will be built with ccache... which fails of
course.
To fix this, directly apply the override to the DEPENCIES_HOST_PREREQ
targets.
Note that this only fixes the issue for 'make host-ccache', NOT for
e.g. 'make host-ccache-configure'.
Signed-off-by: Alfredo Alvarez Fernandez <alfredo.alvarez_fernandez@nokia.com>
[Arnout: improve commit message]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
glibc is now supported for ARC so let's kick-start autobuilders
with glibc toolchain for ARC HS.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Linux 4.0 kernel doesn't build with gcc 6.x, which is used since
the toolchain update in commit
193dfffa83 ("support/testing: use more
recent toolchains"). So let's update to Linux 4.11 instead (like the
existing Qemu x86 defconfig does), and update the kernel configuration
file accordingly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
With the hard disk crash of autobuild.b.o, we lost old toolchains, so
use the latest toolchains, which have been restored.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the download directory, when specified with the -d option, is
only used to store the files downloaded by the testing infra, not those
downloaded by Buildroot.
So, we end up with this situation:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(TOP_DIR)/dl
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(BR2_DL_DIR)
This is not very consistent.
We change the behaviour so that the value of -d always takes precedence,
and is used by Buildroot as well, giving this new behaviour:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(DIR)
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(DIR)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix typo 'selectes' -> 'selects'.
Additionally, change 'will exclude' to 'excludes' to align with 'selects'.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix the remaining code style warnings from flake8:
- properly indent continuation lines;
- use proper code to test a parameter is not None.
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fix the trivial warnings from flake8:
- remove modules imported but unused;
- use 2 lines before class or module level method;
- remove blank line at end of file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Change all defconfig fragments to take advantage of
"cf3cd4388a support/tests: allow properly indented config fragment".
Make each defconfig fragment:
- start after a backslash;
- be declared as a multi-line string literal;
- be indented one level more than the variable that contains it.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Move the strip out of leading spaces in defconfig fragments from the
BRTest class to the Builder class. It actually postpones the strip out,
consequentially allowing test cases to post-process the defconfig in
their own __init__ before calling the __init__ method from BRTest.
Moving this code to the Builder class also allows any new test class
that inherits from BRTest to reuse the same code even if the new class
overrides the setUp method.
At same time, prepend a newline to the jlevel handling otherwise it
would stop working for defconfig fragments that don't end in a newline.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We sanity-check the host executables that they have a correct RPATH
pointing to the host libraries.
This is currently done by looking for all files in $(HOST_DIR) that
match the 'ELF executable' pattern (a bit more complex, but that's
idea).
However, when an executable is built with -fPIE of -fpie, it no longer
appears to be an 'ELF executable', but it rather looks like an 'ELF
sheard object' (like if it were an library.
So, we miss those files.
It turns out that the problem is a real one, because quite a few
mainline distros, expecially those based on Debian for example, have
already switched to generating PIE code by default, and thus we miss on
a whole class of systems..
We fix that by simply looking if we can find an ELF interpreter in each
file. If we there is one, this is an ELF executable; if not, it may be
anything else: we don't care (not even about ELF libraries).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
grub is no longer maintained: it is stuck at version 0.97 with huge
patches that have no opportunity to be applied upstream, as upstream
has even renamed it grub-legacy.
Besides, it no longer builds correctly with recent binutils versions,
and even the huge patches we could grab from Debian do not help the
slightest.
Since upstream really considers it dead, and there are at least two
alternatives (grub2 and syslinux), just remove grub.
Add a legacy entry.
Remove the test cases as well.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Patches with renames apply properly with patch >= 2.7, but not with
older patch versions. Since "git format-patch" by default generates
patches with renames, Buildroot developers often don't realize that
their patches will not apply properly on build machines that have
patch < 2.7. In order to prevent such a situation from happening
again, this commit adds some logic in apply-patches.sh to refuse
applying patches that contain renames.
Note that just searching for '^rename' is not sufficient, since the
patch commit message may contain the words "rename from" or "rename to"
as well. Therefore, the grep expression is made as accurate as possible,
checking both.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Arnout: spaces instead of tabs (suggested by Yann);
extend commit message.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Force gzip compression level 6 when calculating hash of a downloaded GIT repo.
To make sure the tar->gzip->checksum chain always provides consistent result.`
The script was relying on the default compression level, which must not be
necessarily consistent among different gzip versions. The level 6 is gzip's
current default compression level.
Signed-off-by: Petr Kulhavy <brain@jikos.cz>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When we list the available tests, we use test_dir, which is set from
the path of the script. However, when we run the tests, we use the
hard-coded path.
Ditto to find the config file.
For consistency, always use test_dir.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit c96b8675ea
("support/scripts/check-bin-arch: ignore symbolic links") was bogus,
because it tested ${f}, which is the relative path of the file inside
${TARGET_DIR}, so we end up testing if ${f} on the system is a
symbolic link.
This commit fixes that by testing ${TARGET_DIR}/${f}.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit da32b49f00
("instrumentation: extend packages-file-list.txt with symlinks and
directories"), the packages-file-list.txt also contains symbolic
links. Therefore, check-bin-arch is now also checking symbolic links.
However, symbolic links in $(TARGET_DIR) can have absolute path as
targets, such as:
$ ls -l output/target/sbin/ifdown
lrwxrwxrwx 1 thomas thomas 10 Sep 3 15:55 output/target/sbin/ifdown -> /sbin/ifup
Therefore, we are now potentially checking a host binary, which
obviously makes check-bin-arch fail.
This commit changes check-bin-arch to ignore symbolic links. Indeed,
we have two cases:
- The symbolic link really points to something that will in the
rootfs (such as /sbin/ifup above). In this case, /sbin/ifup will be
checked separately by check-bin-arch.
- The symbolic link doesn't point to something that will be in the
rootfs, and that is not a problem from the perspective of
check-bin-arch, which checks the architecture of target binaries.
Fixes:
http://autobuild.buildroot.net/results/16d384a0183d477646ac7692feb65f00dde7d068/
(vim)
http://autobuild.buildroot.net/results/50429c0f63a8befff9e20899327b9a8d754d99be/
(ifupdown)
http://autobuild.buildroot.net/results/1db65973e782bfa61abcbccd3501bfd235f77288/
(gawk)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a parameter to run-tests to act as a multiplier for all timeouts of
emulator.
It can be used to avoid sporadic failures on slow host machines as well
in elastic runners on the cloud.
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: rename multiplier to timeout_multiplier everywhere]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We add the 3 following combinations:
- basic systemd, read-only, network w/ ifupdown
- basic systemd, read-only, network w/ networkd
- full systemd, read-only, network w/ networkd
The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Arnout: regenerate .gitlab-ci.yml]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The "builtin" kernel does not boot a systemd-based system, so
we resort to building the same one as currently used by our
qemu_arm_vexpress_defconfig.
We test the 8 following combinations:
- busybox, read-only, without network
- busybox, read-only, with network
- busybox, read-write, without network
- busybox, read-write, with network
- basic systemd, read-write, network w/ ifupdown
- basic systemd, read-write, network w/ networkd
- full systemd, read-write, network w/ networkd
- no init system, read-only, without network
The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: update .gitlab-ci.yml]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The default audio backend for qemu is configured at compile time. It
generates annoying warning messages to qemu's stderr when running our
tests, like these:
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Explicitly set the audio backend to "none" at runtime to remove those
messages from our logs. There is no command line argument for this, so
use an environment variable when starting qemu.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain configuration used by the autobuilders to test the
internal toolchain backend on ARM contains an option that no longer
exists: BR2_UCLIBC_VERSION_NG. This option has been removed since
uClibc-ng has been made the one and only uClibc version supported.
Due to this option no longer existing, this toolchain configuration
was in fact never used, because the randgenconfig script validates
that a toolchain configuration is valid by checking that all lines are
still present in the final .config.
Therefore, this commit removes the bogus option, which will make sure
this toolchain config gets used again by our autobuilders.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This column is not used by either genrandconfig or test-pkg, so remove
it.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently have a list of toolchain configurations that are used by
the autobuilders at [1]. However, this makes it a little more difficult
for people to use these configurations, and also to have a different
list of configurations for different branches. For example if a new
architecture is introduced, the 2017.02.x branch doesn't have support
for this architecture yet so it shouldn't try to run those configs.
Therefore, include the autobuild config fragments directly in Buildroot,
so they can be branched together with the rest. We create a new
directory under support/ to store them.
Generated with
wget -nd --no-parent --recursive http://autobuild.buildroot.net/toolchains/configs/
The index.html file is removed.
The toolchain-configs.csv file is adapted so the URLs become relative
paths pointing to the config fragments.
[1] http://autobuild.buildroot.net/toolchains/configs/toolchain-configs.csv
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As suggested by Arnout in [1].
While at it, simplify the logic by always appending the BR2_JLEVEL and
defaulting to 0 (the value copied from Config.in is used for 5 years now
and is very unlikely to change).
[1] http://patchwork.ozlabs.org/patch/790525/
Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit cf3cd4388a the -j option is
silently ignored.
The configuration lines are processed using '\n'.join().
This function adds intervening occurrences of the separator, but the
resulting string does not end at a separator.
>>> "n".join(["a","b"])
'anb'
It results in a defconfig that does not end in a newline.
When BR2_JLEVEL is added by -j logic to the defconfig it ends up
concatenated to the last line of the defconfig.
BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7
The resulting .config has the default BR2_JLEVEL=0.
Instead of just workaround this problem by adding a newline before
BR2_JLEVEL when -j is used, make the defconfig to end in a newline since
it is a more future-proof solution.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The defconfig is composed on-the-fly by test infra + tests.
Dump it to the logfile before running 'make olddefconfig' so it can
easily analysed when debugging.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Depending on Python implementation used for testing, time it takes to
perform a given test can vary pretty significantly. To accout for that
allow individual test functions to specify different timeout value.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In order to be able to leverage the same test code for testing
different python interpreters (or wrappers around CPython) allow child
classes of TestPythonBase to override the name of the executable used
to run tests.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add Python3 version of TestPython2 to make sure both versions of
Python get unit-tested.
Modify the code of libc_time_test() to support that change (convert
the code to use Python3-style "print").
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
[Thomas: update .gitlab-ci.yml.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Convert TestPythonBase to a true base class that only provides code
implementing various tests without defining tests themselves in a
"discoverable" form.
To retain correct testing functionality, add TestPython2 derived class
that uses code from TestPythonBase to define actual runnable test.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: fix typo in commit log, update .gitlab-ci.yml, both pointed
by Ricardo.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some commands take more than 5 seconds to complete under QEMU, so add
provisions to allow individual unit-test to specify different duration
to avoid false negative test failures.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The user shouldn't need to pass this manually when creating a test-pkg
config file. It's an absolutely harmless option to enable always.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We really want test-pkg to do the test with a paranoid unsafe path.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This minimal configuration is also very useful outside test-pkg. In
addition, it will simplify the config merge in a later patch.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit introduces the script "fix-rpath" able to scan a tree,
detect ELF files, check their RPATH and fix it in a proper way.
The RPATH fixup is done by the patchelf utility using the option
"--make-rpath-relative <root-directory>".
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The normal shell does not support the bashism "< <(...)". Therefore
we use a normal pipe to find files containing a specific string.
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is because $(HOST_DIR)/usr is gone.
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, defining a config fragment in the runtime test infra requires
that the fragment not to be indented. This is beark, and causes grievance
when looking at the code (e.g. to fix it).
Just strip out all leading spaces/tabs when writing the configuration
lines into the config file, allowing in-line indented config fragments,
like so:
class TestFoo(bla):
config = bla.config + \
"""
FOO=y
# BAR is not set
"""
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The current test fails because of a legacy option, renamed during the
recent ext overhaul.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In commit b78b50465c, the initialisation
of BRTest.builddir was moved to the __init__ function. However, it is
set based on BRTest.outputdir and that is only set when the -o argument
is given to run-tests. When called as "run-tests -l", there is no -o
argument so BRTest.outputdir remains unset.
To fix, keep BRTest.builddir at None when BRTest.outputdir is None.
While we're at it, drop the direct access to the class member. If a
subclass wishes to set outputdir to something else before calling
BRTest.__init__, they are free to do so.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reported-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
BRTest's setUp() method contains a few assignments that initialize its
member variables. Since we will want to use these in test case
overrides, move them to the __init__ function.
Also allow the config member to be overridden, rather than always
taking the class member.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This file is not a package per-se, it includes other .mk files that
are packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The waf package infrastructure was not known by the pkg-stats script,
so let's add it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
With 2000+ packages it's not trivial to identify i.e.:
- all packages that don't have a hash file;
- all packages that have patches;
- all packages that have code style warnings;
User experience can be improved by dynamically sorting the resulting
table.
There is an open-source solution that does that in the client-side and
requires minimal changes to our script: sorttable.js. The script is
MIT licensed as stated in its website.
Also add a hint to the user that the table can be sorted.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now all packages have been updated to install things in $(HOST_DIR)/lib
instead of $(HOST_DIR)/usr/lib, there should no longer be any reason
to have $(HOST_DIR)/usr/lib in the RPATH, so we don't allow it any more.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Wolfgang Grandegger <wg@grandegger.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The tools are now installed in host/bin instead of host/usr/bin.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This silences the annoying warning that there is no hash file for our
own COPYING file.
Also change the message so that it is more obvious what we're doing.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
These tests simply build a system with musl and uclibc toolchains, and
boot them under qemu. It allows to minimally validate that our support
for musl/uclibc external toolchains is working. We already had some
tests covering glibc toolchains, so we can now easily test that all
three C libraries are supported.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
This commit is part of the series, as I've written/used those tests to
validate that things are still working correctly with all of glibc,
uclibc and musl toolchains.
Remove the redundant usr/ component of the HOST_DIR paths. Since a
previous commit added a symlink from $(HOST_DIR)/usr to $(HOST_DIR),
everything keeps on working.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move toolchainfile.cmake and Buildroot.cmake from
$(HOST_DIR)/usr/share/buildroot to $(HOST_DIR)/share/buildroot.
Build-tested with a bunch of cmake packages.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since $(HOST_DIR)/usr/{bin,sbin} are now symlinks to
$(HOST_DIR)/{bin,sbin}, it makes no sense to check them - they are
already covered.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is a step towards eliminating $(HOST_DIR)/usr. It allows us to
convert all packages installing things into $(HOST_DIR)/usr/lib without
affecting the rest.
To allow compatibility with packages that still use $(HOST_DIR)/usr as
the prefix, create a symlink from usr/lib to ../lib.
Note that the symlink creation will break when $(HOST_DIR)/usr/lib
already exists as a directory, i.e. when rebuilding in an existing
output directory. This is necessary: if we don't break it now, the
following commits (which remove the usr part from various variables)
_will_ break it.
At the same time as creating this symlink, we also have to update the
check-host-rpath script to accept both $(HOST_DIR)/usr/lib and
$(HOST_DIR)/lib, because depending on how the package derives the
path, it may be different.
Since there are some dependency chains that involve $(STAGING_DIR),
$(STAGING_DIR) may in fact be created before $(HOST_DIR). Since
$(STAGING_DIR) is a subdirectory of $(HOST_DIR), it is possible that the
newly added rule for $(HOST_DIR) never triggers. To make sure that the
rule does trigger, add an order-only dependency from $(STAGING_DIR) to
$(HOST_DIR).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It will install the script "relocate-sdk.sh" in the HOST_DIR
allowing to adjust the path to the SDK directory in all text
files after it has been moved to a new location.
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
[Thomas:
- Fix shebang to be /bin/sh instead of /bin/bash, suggested by Arnout
- Use | instead of \ as a separator for sed expressions, suggested by
Arnout, discussed with Wolfgang and others
- Remove ./ at the beginning of LOCFILE, suggested by Arnout
- Fix comment about the path check being made before doing the
replacement, suggested by Arnout
- Fix indentation, suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
By default, cut prints the entire line if the specified delimiter is not
present at all:
$ printf "foo bar" | cut -d' ' -f2
bar
$ printf "foobar" | cut -d' ' -f2
foobar
In setlocalversion, cut is presented with the output of 'hg id' which has
the format:
"<revision> <tags-if-any>"
If the current revision is not tagged, the output of 'hg id' does not
contain the delimiter (space), cut prints the entire string, and
setlocalversion thinks the version is the tag.
As setlocalversion does not print anything for tagged versions, there is no
output overall, and no correct indication of the mercurial revision.
Fix by passing the extra cut option '--only-delimited', which suppresses
output if no delimiter is found.
This problem likely went unnoticed for so long, because the tag 'tip' (i.e.
most recent revision of the branch) is treated specially: in this case the
mercurial revision _is_ printed, i.e. the situation is treated as
'untagged'.
The problem is only seen when you are _not_ at the most recent revision in
your branch.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For Gitlab-CI, we want to avoid re-generating the minimal install to
be able to run tests all the time. So let's create a docker image that
we can post on Docker Hub and then pull.
For the time being, this is just what we need for running our CI. Later
we can produce something that is also useful for users.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
"$ORIGIN/../../usr/lib" is also a valid RPATH for binaries in
"$hostdir/usr/bin". After RPATH sanitation, all RPATH
directories start with "$ORIGIN".
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
After some discussion, we found out that "tools" has the four first
letters identical to the "toolchain" subfolder, which makes it a bit
unpractical with tab-completion. So, this commit renames "tools" to
"utils", which is more tab-completion-friendly.
This has been discussed with Arnout and Yann.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: "François Perrad" <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move it to the top-level tools/ directory, so that it is easier to
find for users.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When running multiple instances of emulator in parallel, the login
prompt can take some time to appear.
Use a large timeout when waiting for the prompt to avoid random
failures.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Let the user to pass -t to set the number of testcases to run
simultaneously.
When -j is not specified, calculate it to split the available cores
between the simultaneous testcases.
Example of auto calculated -j for cpu_count 8:
-t -j total
1 9 9
2 4 8
3 3 9
4 2 8
>=5 1 t
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Let the user to override the default BR2_JLEVEL used for each testcase.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Remove unused import.
Use 2 empty lines before a class.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When using pexpect there is no need for a helper function. Just use
expect() directly everywhere.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When using pexpect there is no need for a helper function. Just use
sendline() directly everywhere.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When the parameter logfile is passed to spawn(), pexpect sends both
stdin and stdout to the logfile and it creates a double echo effect.
One way to avoid the double echo in the logfile would be to disable the
echo on the terminal just after login ("stty -echo"), but double echo of
user and password would remain.
Instead of that, send only the stdout to the logfile using the
logfile_read property.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of redirecting qemu serial to telnet, redirect it to stdio.
It allows to run testcases in parallel without random failing caused by
two emulators trying to use the same telnet port (1234).
'qemu -serial stdio' returns some extra <CR> characters, so remove them
from the log.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Replace subprocess + telnetlib with pexpect.
Use the telnet installed on the host machine instead of telnetlib, while
the serial from qemu is not yet redirected to stdio.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is useful on CentOS 7, whose "cmake" utility corresponds to version
2.8.12, which is too old for Buildroot.
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add the BR2_CMAKE_CANDIDATES variable, containing a list of candidates
to check and use as BR2_CMAKE, if possible.
This allows using "cmake3" on CentOS 7, whose default cmake corresponds
to version 2.8.12. Example:
$ make BR2_CMAKE_CANDIDATES="cmake cmake3"
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is useful on CentOS 7 whose "cmake" package provides cmake 2.8.12,
which is too old, but the "cmake3" package (from EPEL) provides version
3.6.3, which is satisfactory. Examples:
$ sh support/dependencies/check-host-cmake.sh 2.8 cmake cmake3
/usr/bin/cmake
$ sh support/dependencies/check-host-cmake.sh 3.1 cmake cmake3
/usr/bin/cmake3
$ sh support/dependencies/check-host-cmake.sh 3.8 cmake cmake3
(nothing)
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Pass the minimal version before the program name. In a later change the
script will become able to test a list of candidates.
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit fixes a problem where it was not possible to replace
/etc/shadow with a symlink to a e.g. a user partition where the
shadow file is placed. This is required, e.g. for systems where the
rootfs is mounted read-only but users should still be able to be
added. Thus, if within an filesystem overlay setup a user tries
to replace /etc/shadow with a symlink to the real file on a user
partition a buildroot build stops with an error message because
sed is called on the symlink instead of following the symlink.
This commit fixes this shortcoming.
Signed-off-by: Jens Maus <mail@jens-maus.de>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The API v0 is shutdown.
see https://bugs.busybox.net/show_bug.cgi?id=9951
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... otherwise it fails spectacularly as soon as PATH is referenced in a
package rule (i.e. very soon, fortunately):
>>> host-lzip 1.18 Downloading
/bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
/bin/bash: -c: line 1: syntax error: unexpected end of file
Fixes # 9886.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ciro Santilli <ciro.santilli@gmail.com>
[Thomas: fix typo in message, use tabs for indentation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The 'lines' variable is overwritten with its own fields. Thus it
contains a line first, and then a list of fields -- it never contains
'lines'.
Use two different variables named 'line' and 'fields' to make the code
more readable.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We only have a positive test for it, in ext4. Let's have a negative
one as well.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently call infra.smart_open() to open log files each time we
need to write to them.
Opening the file once in the constructor of Builder and Emulator and
writing to it whenever needed is simpler and slightly more efficient.
Remove smart_open and instead create a new open_log_file() function
which just opens the logfile. Also let it compute the filename, in
order to simplify even further the Builder and Emulator code.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is what the manpages usually do, and what Python does with the
automatically-added -h/--help parameter:
Before the change:
$ ./support/testing/run-tests
[...]
optional arguments:
-h, --help show this help message and exit
--list, -l list of available test cases
--all, -a execute all test cases
After the change:
$ ./support/testing/run-tests
[...]
optional arguments:
-h, --help show this help message and exit
-l, --list list of available test cases
-a, --all execute all test cases
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
has_broken_links makes it self-explanatory that this is a predicate
function, and that the return value tells whether there _are_ broken
links, not the opposite.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently messages from run-tests are quite unpleasant:
[br-tests/TestPostScripts/2017-05-09 15:51:57] Building
[br-tests/TestPostScripts/2017-05-09 15:52:23] Building done
[br-tests/TestPostScripts/2017-05-09 15:52:23] Cleaning up
.[br-tests/TestNoTimezone/2017-05-09 15:52:23] Starting
[br-tests/TestNoTimezone/2017-05-09 15:52:23] Building
[br-tests/TestNoTimezone/2017-05-09 15:53:17] Building done
[br-tests/TestNoTimezone/2017-05-09 15:53:22] Cleaning up
.[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Starting
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Building
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:33] Building done
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:37] Cleaning up
[...]
Change them in a more readable way by removing the date and using a
columnar style:
15:12:22 TestPostScripts Starting
15:12:25 TestPostScripts Building
15:12:48 TestPostScripts Building done
15:12:48 TestPostScripts Cleaning up
.15:12:48 TestNoTimezone Starting
15:12:54 TestNoTimezone Building
15:13:44 TestNoTimezone Building done
15:13:49 TestNoTimezone Cleaning up
.15:13:49 TestGlibcNonDefaultLimitedTimezone Starting
15:14:00 TestGlibcNonDefaultLimitedTimezone Building
15:14:56 TestGlibcNonDefaultLimitedTimezone Building done
15:15:01 TestGlibcNonDefaultLimitedTimezone Cleaning up
[...]
Note the '.' and other characters presented by nose2 are still
printed. They are not affected by this change.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds an initial toolchain test case, testing the ARM
CodeSourcery toolchain, just checking that the proper sysroot is used,
and that a minimal Linux system boots fine under Qemu.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds some basic tests for two Buildroot packages: python and
dropbear. These tests are by no mean meant to be exhaustive, but mainly
to serve as initial examples for other tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds a number of test cases for various filesystem formats:
ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them
except yaffs2 are runtime tested. The iso9660 set of test cases is
particularly rich, testing the proper operation of the iso9660 support
with all of grub, grub2 and isolinux.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds a few Buildroot "core" tests, testing functionalities
such as:
- post-build and post-image scripts
- root filesystem overlays
- timezone support
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When called from BR2_ROOTFS_POST_IMAGE_SCRIPT, this script
ends up with following error:
Error: Missing argument
This is because, an extra positional argument is also passed
along with BR2_ROOTFS_POST_SCRIPT_ARGS. genimage.sh didn't
have support to parse positional and optional arguments
together.
Signed-off-by: Abhimanyu Vishwakarma <Abhimanyu.V@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently the check-package script uses many files in the same
directory. This commit keeps the main script in support/scripts/ and
moves the rest into a subdirectory.
The modules were previously prefixed to make it easy to identify which
script they belong to. This is no longer needed when using a
subdirectory, so the prefix is removed.
Note: if this commit is checked out and the script is run, and later on
a previous version is checked out, the file
support/scripts/checkpackagelib/__init__.pyc needs to be manually
removed to prevent Python interpreter to look for checkpackagelib
package when only the checkpackagelib module is available.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The removal of the .git dir before creating the tarball is not anymore
just an optimization. It is necessary to make the tarball reproducible.
Also, without the removal, large tarballs (gigabytes) would be created
for some linux trees.
Update the comment accordingly.
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain name was calculated in main() for reporting to the user,
and again in build_one() for creating the build directory. Calculate
it only once, in main(), and pass the build directory as an argument
to build_one().
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This allows the page at http://autobuild.buildroot.net/stats/ to show
how many warnings returned by check-package affect each package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Warn when help text is larger than 72 columns, see [1].
Warn for wrongly indented attributes, see [1].
Warn when the convention of attributes order is not followed, see [2].
[1] http://nightly.buildroot.org/#writing-rules-config-in
[2] http://nightly.buildroot.org/#_config_files
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Check each hash entry (see [1]) and warn when:
- it does not have three fields;
- its type is unknown;
- its length does not match its type;
- the name of the file contains a directory component.
[1] http://nightly.buildroot.org/#adding-packages-hash
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Create 3 new check functions to warn when:
- there are consecutive empty lines in the file, see [1];
- the last line of the file is empty, see [2];
- there are lines with trailing whitespace, see [3].
Apply these functions to Config.*, *.mk and *.hash, but not for *.patch
files since they can contain any of these and still be valid.
[1] http://patchwork.ozlabs.org/patch/682660/
[2] http://patchwork.ozlabs.org/patch/643288/
[3] http://patchwork.ozlabs.org/patch/398984/
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Create the infra to check the style of new packages before submitting.
The overall function of the script is described inside a txt file.
It is designed to process the actual files and NOT the patch files
generated by git format-patch.
Also add the first check function, to warn if a file (Config.*, *.mk,
*.hash, *.patch) has no newline at the last line of the file, see [1].
Basic usage for simple packages:
support/scripts/check-package -vvv package/newpackage/*
Basic usage for packages with subdirs:
support/scripts/check-package -vvv $(find package/newpackage/ -type f)
See "checkpackage" in [2].
[1] http://patchwork.ozlabs.org/patch/631129/
[2] http://elinux.org/Buildroot#Todo_list
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script is a wrapper for the genimage tool used by most boards.
The board postimage script can now call this script instead of invoking
genimage command themselves.
Signed-off-by: Etienne Phelip <etienne.phelip@savoirfairelinux.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Unextected error in the br2-external script are properly caught, but
they are not reported properly, and we end up in either of two
situations:
- the .br2-external.mk file is not generated, in which case make will
try to find a rule to generate it (because the 'include' directive
tries to generate missing files);
- the .br-external.mk file is generated but does not contain the error
variable, and thus the build might not get interrupted.
We fix that by using a trap on the pseudo ERR signal, to emit the error
variable on unexpected errors.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In commit 2f6c5e513c
("support/check-bin-arch: fix for filenames with spaces"), Yann
adjuste the check-bin-arch script to properly handle filenames with
spaces.
However, he also did a subtle change of the regexp that extracts the
path of the files. It was:
"/^${package},(.+)$/!d; s//\1/;"
and Yann changed it to:
"/^${package},\.(.+)$/!d; s//\1/;"
So the file paths used to start with a dot (like "./usr/share/foo"),
and now they no longer start with a dot (like "/usr/share/foo"). While
this modification is good and makes sense, the match for
/lib/firmware/ was not adjusted accordingly, and the follow-up patch
also ignoring /usr/share was not adjusted as well.
This commit fixes those /lib/firmware/ and /usr/share/ special cases,
which will fix:
http://autobuild.buildroot.net/results/76a1475f4cdedb80426fb022ef2e644aa5625660/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
On most distros, the tar format defaults to GNU. However, at build time
the default format may be changed to posix. Also, future versions of
tar will default to posix.
Since we want the tarballs created by the git download method to be
reproducible (so their hash can be checked), we should explicitly
specify the format. Since existing tarballs on sources.buildroot.org
use the GNU format, and also the existing hashes in the *.hash files
are based on GNU format tarballs, we use the GNU format.
In addition, the Posix format encodes atime and ctime as well as mtime,
but tar offers no option like --mtime to override them. In the GNU
format, atime and ctime are only encoded if the --incremental option is
given.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
pkgutil.py is also part of Python itself. Placing pkgutil.py as is
in a folder with other scripts that require original pkgutil will
break them. This is the case with scanpypi. So rename pkgutil.py
to brpkgutil.py to avoid naming collision.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=9766
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
/usr/share normally should not contain binaries executable for the
target platform. However, it might contain ELF binaries for other
platforms, such as firmware files installed by Qemu or
pru-software-support.
Instead of special-casing each package, let's simply ignore /usr/share.
Fixes:
http://autobuild.buildroot.net/results/6f3fea9f6adaef1573fbb0dd6903b5d99e470610/
(pru-software-support)
http://autobuild.buildroot.net/results/fe8892bc22a03299fc41e30bfea5e42166838f88/
(qemu)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Filenames with spaces will break the current for loop.
Fix that by using a while-read loop, fed with the list of files on
stdin, using process substitution.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
By default, compile_dir() relies on the modification time to know if a
python file has to be built again. However in some circumstances (when
doing reproducible builds), modification times are not reliable. Thus,
this patch adds a way to force the rebuild of all python sources.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.
This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.
Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.
Example output with the firejail package enabled, when building for an
ARM target:
ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM
Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As of the version 3.6.0 compile_dir() call will treat its 'quiet'
argument as a full blown integer rather than a boolean value and perform
integer comparison operations such as '<' or '>='.
To account for that convert ReportProblem type to be a true derivative
of built-in int() and override all of int's rich comparison operators in
order to be able to "sniff" for PyCompileError in all possible use-cases
The integer value ReportProblem pretends to be is teremined by class
variable VALUE which is set to 1.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we tell cmake where to look for our own custom platform
description by passing the path to the moduls directory on the command
line.
However, this causes two different problems.
First, some packages simply set CMAKE_MODULE_PATH in their
CMakeList.txt, thus overriding our own path, and then our platform
description is not found.
Second, cmake may internally call sub-cmake (e.g. in the try_compile
macro), but the CMAKE_MODULE_PATH is not automatically passed down in
this case.
For the first problem, we could hunt down and fix all offenders, but
this is an endless endeavour, especially since packagers are told to do
so on the cmake wiki [0]:
CMAKE_MODULE_PATH
tell CMake to search first in directories listed in
CMAKE_MODULE_PATH when you use FIND_PACKAGE() or INCLUDE()
SET(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/MyCMakeScripts)
FIND_PACKAGE(HelloWorld)
The second problem could be solved by passing yet another variable on
the command line, that tells cmake to explicitly pass arbitrary
variables down to sub-cmake calls:
-DCMAKE_TRY_COMPILE_PLATFORM_VARIABLES=CMAKE_MODULE_PATH
However, this only covers the case of try_compile. Even though no other
case is known yet, we'd still risk missing locations where we would need
to propagate CMAKE_MODULE_PATH, even some where we'd have no solution
like for try_compile.
Instead, ngladitz on IRC suggested that CMAKE_MODULE_PATH be set
directly from the toolchain file.
And indeed this fixes both problems explained above.
So be it.
[0] https://cmake.org/Wiki/CMake_Useful_Variables
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The LINES variable is automatically set by bash to represent the number
of lines in the terminal. That variable can be set when the shell
receives SIGWINCH.
If the shell does receive SIGWINCH after our LINES array is filled, the
content of the array is mangled.
Rename the variable to avoid that.
Fixes#9456
Reported-by: George Y. <georgebrmz@oss3d.com>
Reported-by: Paul Stewart <paulstewartis@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some packages test the system name to decide whether to enable/disable
features or link with specific libs.
So we forcefully set the system name form our custom system file, so
that packagses still believe they are running on Linux rather than
Buildroot.
Fixes:
fastd : http://autobuild.buildroot.net/results/f1d/f1dfe90068ad62e733f17a22202235415bda3974/
paho-mqtt-c: http://autobuild.buildroot.net/results/457/457d76279e16247bf58c838a2c5dd0a4f3962c21/
libiio : http://autobuild.buildroot.net/results/281/2812b008a0ab6bab5fe4d45eb9ffe4e9496a8cb4/
and so on...
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This reverts commit 4422eca2d4.
We now have a workaround for the RPATH issue introduced in 3.7, so we
can use it again.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The handling of RPATH in cmake-3.7 has changed drastically, causing a
slew of build failures dues to libraries from the host being pulled in:
- domoticz : http://autobuild.buildroot.org/results/fd0/fd0ba54c7abf973691b39a0ca1bb4e07d749593a/
- freerdp : http://autobuild.buildroot.org/results/5d4/5d429d0e288754a541ee5d8be515454c5fccd28b/
- libcec : http://autobuild.buildroot.org/results/3f3/3f3593bab7734dd274faf5b5690895e9424cbb89/
- and so on...
The bug was reported upstream [0], which dismissed it altogether [1] as
being expected behaviour, quoting:
I don't think there is anything wrong with that change on its own.
It merely exposed some existing behavior in a new case.
Instead, upstream suggested in that same message that a platform
definition be used instead, quoting:
If a toolchain file specifies CMAKE_SYSTEM_NAME such that a custom
`Platform/MySystem.cmake` file is loaded then the latter can set
them as needed for the target platform.
So here we are doing so:
- we add a new platfom definitions that inherits from the Linux one,
then overrides the problematic settings;
- we change our toolchain file to use that platform instead;
- we tell cmake where to find additional modules, so that it can find
our custom platform file.
This has been tested to work in the following conditions:
- pre-installed host cmake, versions 3.5.1 (Ubuntu 16.04) and 3.7.2
(manually built)
- internal cmake, versions 3.6.3 (the current version as of this
patch) and 3.7.2 (with the followup patches).
Thanks to Jörg, Ben and Baruch for the help investigating the issue.
Special thanks to Jörg for handling the discussion with upstream and
pointing to the relevant messages! :-)
[0] http://public.kitware.com/pipermail/cmake/2017-February/064970.html
[1] http://public.kitware.com/pipermail/cmake/2017-February/065063.html
To be noted: Thomas suggested we set these directly in the toolchain
file. Unfortunately, wherever we put those settings in the toolchain
file, this does not work.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
cmake-3.7 has a bug in how it handles rpath, linking with libraries from
the host.
Until we completely understand the issue, just blacklist cmake-3.7.
The issue has been reported upstream:
http://public.kitware.com/pipermail/cmake/2017-February/064970.html
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Remove distribution upgrade cause it slows down the first boot and
presents a bug when executed non interactively.
* Reorganize provision scripts to be in privileged and non privileged
sections
* Add Ubuntu mirror automatic handling for apt packages sources
Fixes bug #9581
Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit teaches the generic package handling code how to extract .tar.lz
archives. When lzip is not installed on the host, host-lzip gets built
automatically.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Don't special case $(XZCAT) when constructing DL_TOOLS_DEPENDENCIES. The next
commit will introduce another extractor that automatically builds when not
installed. Introduce EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS that lists
archive extensions for which the extractor is already checked in
support/dependencies/check-host-foo.mk. Use this in the newly introduced
extractor-dependency to populate DL_TOOLS_DEPENDENCIES.
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Thomas: add missing space after "firstword", as noticed by Thomas DS.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Requested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use comm(1) to check that all our config options are properly set in the
resulting configuration, rather than our canned and fragile code.
Reported-by: Cam Hutchison <camh@xdna.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Sometimes, it interesting to have a global overview of whether the
package builds at all or not, rather than test on all toolchains.
Add an option that allows testing on a limited set of randomly choosen
toolchains.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a build is skipped, store the lines from the config snippet, that
are missing in the resulting configuration, in a file in the build
directory, for the user to inspect.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script helps in testing that a package builds fine on a wide range
of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr:
- completely rewrite the script from Thomas, with help from Luca
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change adds inode tracking to the size-stats script so that hard
links don't cause files to be double counted. This has a significant
effect on the size computation for some packages. For example, git has
around a dozen hard links to a large file. Before this change, git would
weigh in at about 170 MB with the total filesystem size reported as
175 MB. The actual rootfs.ext2 size was around 16 MB. With the change,
the git package registers at 10.5 MB with a total filesystem size of
15.8 MB.
Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.
Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.
As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.
Instead, pass the global as an argument.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Usually, Buildroot does never initialize variables with empty content.
DEPENDENCIES_HOST_PREREQ was an unjustified exception.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixes#9576
When the path to a br2-external tree is relative, make enters an endless
recursive loop (paths elided for brevity):
$ make BR2_EXTERNAL=.. foo_defconfig
make[1]: stat: ../configs/../configs/../configs[...]/toto_defconfig: Filename too long
make[1]: *** No rule to make target '../configs/../configs/../configs[...]/toto_defconfig',
needed by '../configs/../configs/../configs[...]/toto_defconfig'. Stop.
Makefile:79: recipe for target '_all' failed
make: *** [_all] Error 2
It is a bit complex to understand the actual technical reason for this
never-ending expansion; it seems it happens in the code generated by the
percent_defconfig macro. Not sure why, though...
But the root cause is the relative path.
Just use absolute, canonical paths to br2-external trees. Always.
[Peter: add bugzilla reference]
Reported-by: outtierbert@gmail.com
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Recently, the autoconf macros for libtool started using '/usr/bin/file'
to determine the type of library that is generated by the toolchain.
Packages that use this recent version of the libtool autoconf macros
will fail in a rather dramatic way when /usr/bin/file is not present
on the host: the package will still build but no shared library is
generated, which in turn may cause build failures in other packages
that link with it.
For example, libpng's configure determines that it is not possible to
build a shared library on MIPS64 because the expected output from 'file'
is not present. Therefore, only a static libpng.a is built. Later,
bandwithd links with -lpng but it doesn't use the pkg-config's
Private-Libs (because it's not linking statically) and it doesn't have
access to the NEEDED reference from the shared library. Therefore, it
doesn't link with zlib and fails with
pngrutil.c:(.text+0x55c): undefined reference to `inflate'
We cant use host-file because it is itself an autotools package and is
itself using libtool, so this would be a chicken-n-egg problem. Besides,
the libtool script really wants to call /usr/bin/file, so it would not
even find our host-file anyway.
So, just require that '/usr/bin/file' is present on the host.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Many (100+) packages supported by buildroot contain old configure
scripts (or build them from old versions of autotools) that are unable
to determine how to link shared libraries on powerpc64 and
powerpc64le. This causes that test to erroneously fail on toolchains
that are not "bi-endian" (which is the case for toolchains built by
buildroot), which causes configure to build static libraries instead
of dynamic ones. Although these builds succeed, they tend to cause
linker failures in binaries later linked against them.
Because affected configure files can be discovered automatically, this
patch introduces a hook (enabled only when building for powerpc64 and
powerpc64le) that uses a script to scan and fix each package.
Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we require a host with an UTF-8 locale as soone as we build a
Buildroot toolchain with support for locales. This means that we do
erquire such a locale when building a toolchain with either uClibc,
glibc or musl.
However, glibc and musl do not require such a locale to be present.
Use the new option to check if an UTF8 locale is needed on the host,
rather than deriving that from toolchain settings.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the br2-external script uses bash-4's associative arrays.
However, some oldish enterprise-class distros like RHEL5 still use
bash-3.1 which lacks associative arrays.
We restore compatibility with those oldish distros using 'eval' to
emulate associative arrays, as suggested by Arnout.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python3 complains about missing parentheses.
$ ./support/scripts/get-developers
File "./support/scripts/get-developers", line 45
print f
^
SyntaxError: Missing parentheses in call to 'print'
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This adds an ev3dev Linux drivers extension that provides Linux kernel
drivers for LEGO MINDSTORMS EV3 from the ev3dev project.
Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, there are two failure paths in the wrapper:
- if the tar fails, then the error is ignored because it is on the
left-hand-side of a pipe;
- if the find fails, then the error is ignored because it is a
process substitution (and there is a pipe, too).
While the former could be fixed with "set -o pipefail", the latter can
not be fixed thusly and we must use an intemediate file for it.
So, fix both issues by using intermediate files, both to generate the
list of files to include in the archive, and generate the archive in a
temporary tarball.
Fixes the following build issue, where the find is failing for whatever
unknown reason:
http://autobuild.buildroot.net/results/20f/20fd76d2256eee81837f7e9bbaefbe79d7645ae9/
And this one, where the process substitution failed, also for an unknown
reason:
http://autobuild.buildroot.org/results/018/018971ea9227b386fe25d3c264c7e80b843a9f68/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.
It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Current type for 'patches' argument is str. It supposed to only
contain names of files.
If we specify FileType as type, then we don't need to open file ourself
and it allows script to read patch from standard input as well.
e.g.
$ git show -1 | ./support/scripts/get-developers -
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
From the build configuration, Buildroot defines and set some compiler
and linker flags that should be passed to any packages build-system.
For package using the cmake-package infrastructure, this is achieved
via the toolchainfile.cmake.
This change simplifies the way the toolchainfile.cmake file handles
these flags: it now just sets them, without any attempt to extend them
with those Buildroot defined.
This change still allows overriding these flags from the configure
command line.
So, now, when a CMake-based package needs to extend them, they should
be fully set from the package *.mk file. This behavior is consistent
with what is done for others package infrastructures.
This change should not pull any regression WRT the bug #7280 [1].
However, now, when someone uses the toolchainfile.cmake file outside of
Buildroot, he/she must overload all compiler/linker flags (including the
ones Buildroot sets since they no longer get automatically added).
[1] https://bugs.busybox.net/show_bug.cgi?id=7280
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a build type is set, CMake does append some flags that can override
those set by Buildroot due to the gcc option parser (in which the last
argument controling an option wins).
Hereafter is a summary of the optimization and debug flags set by
Buildroot and appended by CMake.
* Flags set by Buildroot depending on the configuration:
BR2_ENABLE_DEBUG | Optim. level | Buildroot {C,CXX}FLAGS
=================+=====================+=======================
y | BR2_OPTIMIZE_S | -Os -gx
y | BR2_OPTIMIZE_G | -Og -gx
y | BR2_OPTIMIZE_{0..3} | -On -gx
n | BR2_OPTIMIZE_S | -Os
n | BR2_OPTIMIZE_G | -Og
n | BR2_OPTIMIZE_{0..3} | -On
* Default flags appended by CMake depending on the build type:
Build type | Flags | Effects on {C,CXX}FLAGS
===============+=================+===========================================
Debug | -g | Force -g, compatible with BR2_ENABLE_DEBUG
MinSizeRel | -Os -DNDEBUG | Set -Os, compatible with BR2_OPTIMIZE_S
Release | -O3 -DNDEBUG | Set -O3, closest to the others cases,
| | though the optimization level is forced.
RelWithDebInfo | -O2 -g -DNDEBUG | Force -g and set -O2, not friendly with BR
To avoid the CMake flags take precedence over the Buildroot ones, this
change sets in toolchainfile.cmake the per-config compiler flags CMake
can append depending on the build type Buildroot defined.
So, CMake does not mess up with the compilation flags Buildroot sets.
It is still possible to override these per-config flags on the cmake
command line.
Note:
If a CMake-based project forces the compiler and/or linker flag
definitions (the default ones or the per-config ones - e.g.
CMAKE_C_FLAGS/CMAKE_C_FLAGS_{DEBUG,RELEASE}), there is not much
Buildroot can do about it.
So, the flags will be overwritten anyway in these cases.
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- adjust comment in toolchainfile.cmake.in, as suggested by Arnout.
- also handle CMAKE_Fortran_FLAGS_*, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The chosen CMAKE_BUILD_TYPE encodes an option of the Buildroot
configuration, so it makes more sense to save it in the
toolchainfile.cmake than to pass it during configure.
It is still possible to override the build type on the cmake
command line.
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: reword description in the CHANGES file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Output of get-developers script in our manual uses --cc for developers,
but actual output of get-developers script uses --to. This patch makes
code consistent with documentation, by using --cc for developers.
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
To be noted: that link will only be valid once we have a released
manual. In the meantime, it's accessible on the nightly manual:
http://nightly.buildroot.org/#br2-external-converting
Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we support multiple br2-external trees, BR2_EXTERNAL is no
longer exported in the environment.
This means that post-build scripts in a br2-external tree can no longer
find their own files (well, they could re-invent the path by stripping
their known-relative path, but that'd be ugly, especially since we can
very well provide it).
Export the path for each br2-external trees as environment variables.
Do so for the description as well, as a courtesy.
Also, re-order variable definitions to be more logical: first, purely
internal variables, then exported variables.
Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, it is not possible for a br2-external tree to override a
defconfig bundled in Buildroot, nor is it possible to override one from
a previous br2-external tree in the stack.
However, it is interesting that a latter br2-external tree be able to
override a defconfig:
- the ones bundled in Buildroot are minimalist, and almost always
build a toolchain, so a br2-external tree may want to provide a
"better" defconfig (better, in the sense "suited for the project");
- similarly for a defconfig from a previous br2-external tree.
But we can't do that, as the rules for the defconfigs are generated in
the order the br2-external trees are specified, all after the bundled
defconfigs. Those rule are patten-matching rules, which means that the
first one to match is used, and the following ones are ignored.
Add a new utility macro, 'reverse', inspired from GMSL, that does what
it says: reverse a list of words.
Use that macro to reverse the list of br2-external trees, so that the
latters win over the formers, and even over bundled ones.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.
A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.
Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.
Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.
What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:
- the list of all br2-external names, BR2_EXTERNAL_NAMES,
- the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
point each to the actual location of the corresponding tree,
- the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,
- the space-separated list of absolute paths to the external trees,
BR2_EXTERNAL_DIRS.
Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.
This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".
Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.
If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.
This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.
Also, display the NAME and its path as a comment in the menuconfig.
This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.
The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.
Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).
Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A br2-external tree must provide external.mk and Config.in.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have one
(BR2_EXTERNAL is not empty).
There is now no reason to treat those two cases differently:
- the kconfig snippet is always generated appropriately (i.e. it would
include the br2-external tree if set, or include nothing otherwise);
- we no longer have a dummy br-external tree either.
Also, the Makefile code to handle BR2_EXTERNAL is currently quite
readable if at least a little bit tricky.
However, when we're going to add support for using multiple br2-external
trees simultaneously, this code would need to get much, much more complex.
To keep the Makefile (rather) simple, offload all of the handling of
BR2_EXTERNAL to the recently added br2-external helper script.
However, because of Makefiles idiosyncracies, we can't use a rule to
generate that Makefile fragment.
Instead, we use $(shell ...) to call the helper script, and include the
fragment twice: once before the $(shell ...) so we can grab a previously
defined BR2_EXTERNAL value, a second time to use the one passed on the
command line, if any.
Furthermore, we can't error out (e.g. on non-existent br2-external tree)
directly from the fragment or we'd get that error on subsequent calls,
with no chance to override it even from command line.
Instead, we use a variable in which we store the error, set it to empty
before the second inclusion, so that only the one newly generated, if
any, is taken into account.
Since we know the script will always be called from Makefile context
first, we know validation will occur in Makefile context first. So we
can assume that, if there is an error, it will be detected in Makefile
context. Consequently, if the script is called to generate the kconfig
fragment, validation has already occured, and there should be no error.
So we change the error function to generate Makefile code, so that
errors are caught as explained above.
Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget'
about the previous value of the BR2_EXTERNAL_MK variable, especially in
the case where BR2_EXTERNAL is now set to empty, so that we do not try
to include it later. That's why we first generate empty version of
BR2_EXTERNAL_MK, and then assign it the new value, if any.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no needed.
As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer needed.
Ditch both of those files, and:
- only generate actual content in the Kconfig snippet when we actually
do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
empty);
- add a variable that contains the path to the external.mk provided by
the user, or empty if none, and include the path set in that variable
(make can 'include' nothing without any problem! ;-) )
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.
This will ultimately allow us to use more than one br2-external tree.
Offload the "User-provided options" menu to the generated Kconfig
snippet. We can also move the definition of the Kconfig-version of
BR2_EXTERNAL into this snippet.
We introduce an extra check that was not present in the previous code,
to check that we do have permission on that directory. Prevciously, it
was handled as a side effect of not being able to cd into there, but it
is cleaner to check it expressly.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
ExtUtils::MakeMaker is one of the Perl Core modules usually packaged in
Perl package for a Debian/Ubuntu based system.
For a Fedora based system, each Perl Core modules have their own RPM
package. So install only Perl package is not enough.
Fixes:
>>> host-libxml-parser-perl 2.41 Configuring
[...]
perl `which perl` Makefile.PL
Can't locate ExtUtils/MakeMaker.pm in @INC (you may need to install the ExtUtils::MakeMaker module)
Add a new Perl module check in dependency.sh.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: François Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script, and its companion library, is more-or-less Buildroot's
equivalent to the kernel get_maintainer.pl script: it allows to get the
list of developers to whom a set of patches should be sent to.
To do so, it first relies on a text file, named DEVELOPERS, at the root
of the Buildroot source tree (added in a followup commit) to list the
developers and the files they are interested in. The DEVELOPERS file's
format is simple:
N: Firstname Lastname <email>
F: path/to/file
F: path/to/another/file
This allows to associate developers with the files they are looking
after, be they related to a package, a defconfig, a filesystem image, a
package infrastructure, the documentation, or anything else.
When a directory is given, the tool assumes that the developer handles
all files and subdirectories in this directory. For example
"package/qt5/" can be used for the developers looking after all the Qt5
packages.
Conventional shell patterns can be used, so "package/python-*" can be
used for the developers who want to look after all packages matching
"python-*".
A few files are recognized specially:
- .mk files are parsed, and if they contain $(eval
$(<something>-package)), the developer is assumed to be looking after
the corresponding package. This way, autobuilder failures for this
package can be reported directly to this developer.
- arch/Config.in.<arch> files are recognized as "the developer is
looking after the <arch> architecture". In this case, get-developer
parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH
values. This way, autobuilder failures for this package can be
reported directly to this developer.
- pkg/pkg-<infra>.mk are recognized as "the developer is looking after
the <infra> package infrastructure. In this case, any patch that adds
or touches a .mk file that uses this infrastructure will be sent to
this developer.
Examples of usage:
$ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch
git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>"
$ ./support/scripts/get-developers -p imx-lib
Arnout Vandecappelle <arnout@mind.be>
Gary Bisson <gary.bisson@boundarydevices.com>
$ ./support/scripts/get-developers -a bfin
Waldemar Brodkorb <wbx@openadk.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We currently have four lists of packages in the manual:
- the non-virtual target packages,
- the virtual target packages,
- the host packages,
- the deprecated features.
Those list take more than half of the manual. They do not serve much
purpose except to show off.
After the recent discussion on the list [0], remove them all.
We can now get rid of our biggish and complex generating script (and its
companion library kconfiglib).
[0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently all cmake packages depend on host-cmake. Unfortunately
host-cmake takes a long time to configure and build: almost 7 minutes
on a dual-core i5 with SSD. The time does not change even with ccache
enabled.
Indeed, building host-cmake is avoidable if it is already installed on
the build host: CMake is supposed to be quite portable, and the only
patch in Buildroot for the CMake package seems to only affect
target-cmake.
Thus we automatically skip building host-cmake and use the one on the
system if:
- cmake is available on the system and
- it is recent enough.
First, we leverage the existing infrastructure in
support/dependencies/dependencies.mk to find out whether there's a
suitable cmake executable on the system. Its path can be passed in the
BR2_CMAKE environment variable, otherwise it defaults to "cmake". If
it is enabled, found and suitable then we set BR2_CMAKE_HOST_DEPENDENCY
to empty; otherwise we set BR2_CMAKE_HOST_DEPENDENCY to 'host-cmake' and
override BR2_CMAKE with "$(HOST_DIR)/usr/bin/cmake" to revert to using
our own cmake (the old behaviour).
Then in pkg-cmake.mk we replace the hard-coded dependency on host-cmake
to using the BR2_CMAKE_HOST_DEPENDENCY variable, and we use $(BR2_CMAKE)
instead of $(HOST_DIR)/usr/bin/cmake.
Unlike what we do for host-tar and host-xzcat, for host-cmake we do
not add host-cmake to DEPENDENCIES_HOST_PREREQ. If we did, host-cmake
would be a dependency for _any_ package when it's not installed on the
host, even when no cmake package is selected.
Cmake versions older than 3.0 are affected by the bug described and
fixed in Buildroot in ef2c1970e4 ("cmake: add patch to fix Qt mkspecs
detection"). The bug was fixed in upstream CMake in version 3.0 [0].
Amongst all the cmake packages currently in Buildroot, the currently
highest version mentioned in cmake_minimum_required() is 3.1 (grantlee
and opencv3).
Thus we use 3.1 as the lowest required cmake for now, until a package is
bumped, or a new package added, with a higher required version.
[0] https://cmake.org/gitweb?p=cmake.git;h=e8b8b37ef6fef094940d3384df5a1d421b9fa568
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Davide Viti <zinosat@tiscali.it>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
[yann.morin.1998@free.fr:
- simplify logic in check-host-cmake.mk;
- set and use BR2_CMAKE_HOST_DEPENDENCY, drop USE_SYSTEM_CMAKE;
- bump to cmake 3.1 for grantlee and opencv;
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
pkg-utils.mk contains various definitions that are used in the package
infrastructures and packages themselves.
However, those definitions can be useful in other parts of Buildroot,
and are already used in a few places that are not related to the package
infrastructure. Also, $(sep) will be needed early in the Makefile when
we eventually support multiple br2-external trees.
Since this file only contains definitions, we can include it anytime.
So, consider that file to no longer be specific to the package infras:
- move it to support and rename it,
- move a few similar definitions from the main Makefile to that file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This adds support to pass options to the underlying command that is used
by downloader. Useful for retrieving data with server-side checking for
user login or passwords, use a proxy or use specific options for cloning
a repository via git and hg.
Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change enforces the CMAKE_SYSROOT value set in the toolchainfile.cmake.
This fix overrides the CMake heuristics used to guess it, and turns off some
non-desirable behavior adding "-isystem ..." flags to the compiler command
line, misleading the compiler and making the build failed due to some
unfound standard headers.
Fixes:
http://autobuild.buildroot.net/results/f7e/f7e92678e91a6cb15ccf32d4a7d75b39f49d6000/defconfig
(and others)
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The size-stats script fails when the usb_modeswitch_data is enabled,
because this package installs files that contain commas in their
name. However, the size-stats script also uses comma as a separator for
its CSV files, causing a "ValueError: too many values to unpack" in:
pkg, fpath = l.split(",")
Fix this by splitting only the two fields that need to be split.
The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested
a fix.
Fixes bug #9136.
Reported-by: Matthias <porto.rio@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The "--no-patch" option used by the git downloader appeared on git
1.8.4. Systems with older git versions show an error and fall back to
the wget downloader, which isn't suitable for all the cases.
Signed-off-by: Enrique Ocaña González <eocanha@igalia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since efl update to 1.15 version, the efl package is a "real"
Buildroot package. It doesn't contain any subdirectories anymore.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.
Use "set -e" to exit immediately if a command return an error.
Be sure to ignore any expected error: when we check if a patch to be
applied has the same basename as an already applied patch, the grep
would fail when no such patch was already applied. We should not fail
in this case.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.
Use "set -e" to exit immediately if a command return an error.
[1] http://patchwork.ozlabs.org/patch/626196
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[Thomas:
- add comment in scancpan about the version dependency, suggested by
Yann E. Morin.
- add comment in perl.mk about the need to sync any version change with
scancpan, also suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
sha256 checksum will be computed locally either by scanpypi at package
creation or by hand by package updates. Define this checksum as
'computed locally' so that one doesn't need to change this comment by
package updates. Also put comments for both md5 and sha256 in one line.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Make sure a help text is terminated with a full stop.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since the fortran support is conditional, only enable it when needed.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Benjamin Kamath <bkamath@spaceflight.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a new package variable that packages can set to specify that they
need git submodules.
Only accept this option if the download method is git, as we can not get
submodules via an http download (via wget).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Aleksandar Simeonov <aleksandar@barix.com>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some git repositories may be split into a master repository and
submodules. Up until now, we did not have support for submodules,
because we were using bare clones, in which it is not possible to
update the list of submodules.
Now that we are using plain clones with a working copy, we can retrieve
the submdoules.
Add an option to the git download helper to kick the update of
submodules, so that they are only fetched for those packages that
require them. Also document the existing -q option at the same time.
Submodules have a .git file at their root, which contains the path to
the real .git directory of the master repository. Since we remove it,
there is no point in keeping those .git files either.
Note: this is currently unused, but will be enabled with the follow-up
patch that adds the necessary parts in the pkg-generic and pkg-download
infrastructures.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We currently use git-archive to generate the tarball. This is all handy
and dandy, but git-archive does not support submodules. In the follow-up
patch, we're going to handle submodules, so we would not be able to use
git-archive.
Instead, we manually generate the archive:
- extract the tree to the requested cset,
- get the date of the commit to store in the archive,
- store only numeric owners,
- store owner and group as 0 (zero, although any arbitrary value would
have been fine, as long as it's a constant),
- sort the files to store in the archive.
We also get rid of the .git directory, because there is no reason to
keep it in the context of Buildroot. Some people would love to keep it
so as to speed up later downloads when updating a package, but that is
not really doable. For example:
- use current Buildroot
- it would need foo-12345, so do a clone and keep the .git in the
generated tarball
- update Buildroot
- it would need foo-98765
For that second clone, how could we know we would have to first extract
foo-12345 ? So, the .git in the archive is pretty much useless for
Buildroot.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we are using bare clones, so as to minimise the disk usage,
most notably for largeish repositories such as the one for the Linux
kernel, which can go beyond the 1GiB barrier.
However, this precludes updating (and thus using) the submodules, if
any, of the repositories, as a working copy is required to use
submodules (becaue we need to know the list of submodules, where to find
them, where to clone them, what cset to checkout, and all those is
dependent upon the checked out cset of the father repository).
Switch to using /plain/ clones with a working copy.
This means that the extra refs used by some forges (like pull-requests
for Github, or changes for gerrit...) are no longer fetched as part of
the clone, because git does not offer to do a mirror clone when there is
a working copy.
Instead, we have to fetch those special refs by hand. Since there is no
easy solution to know whether the cset the user asked for is such a
special ref or not, we just try to always fetch the cset requested by
the user; if this fails, we assume that this is not a special ref (most
probably, it is a sha1) and we defer the check to the archive creation,
which would fail if the requested cset is missing anyway.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In most cases Python's package dependencies found in setup.py are
runtime dependencies and hence don't need to be mentioned in *.mk
file.
Also add '# runtime' tag to select statements in Config.in.
__create_mk_requirements() itself is left for future uses (cffi backend
handling etc.).
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.
We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.
However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.
Update the legal-info header accordingly.
Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Patches we save can come from various locations:
- bundled with Buildroot
- downloaded
- from one or more global-patch-dir
It is possible that two patches lying into different locations have the
same basename, like so (first is bundled, second is from an hypothetical
global-patch-dir):
package/foo/0001-fix-Makefile.patch
/path/to/my/patches/foo/0001-fix-Makefile.patch
In that case, when running legal-info, we'd save only the second patch,
overwriting the first. That would be problematic, because:
- either the second patch depends on the first, and thus would no longer
apply (this is easy to detect, though),
- or the second patch does not depend on the first, and the compliance
delivery will not be complete (this is much harder to detect).
We fix that by checking that no two patches have the same same basename.
If we find that the basename of the patch to be applied collides with
that of a previously applied patch, we error out and report the duplicate.
The unfortunate side-effect is that existing setups will now break in
that situation, but that's a minor, corner-case issue that is easily
fixed.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: adjust coding style, fix minor typos in the commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we only store the filename of the applied patches.
However, we are soon to want to install those patches in the legal-info
directory, so we'll have to know where those patches come from.
Instead of duplicating the logic to find the patches (bundled,
downloaded, from a global patch dir...), just store the full path to
each of those patches so we can retrieve them more easily later on.
Also always create the list-file, even if empty, so that we need not
test for its existence before reading it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
[Tested only with patches in the Buildroot sources]
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: used $PWD instead of $(pwd), as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A utility for creating python package from the python package index.
It fetches packages info from http://pypi.python.org and generates
corresponding packages files.
Signed-off-by: Denis THULIN <denis.thulin@openwide.fr>
Tested-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: minor tweaks.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.
To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.
The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.
The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.
A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
- rework for python 3.5
- integrate Samuel proposal that allows to detect compilation
failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The graph-build-time help text currently looks like this:
usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER]
[--alternate-colors] [--input OUTPUT] --output OUTPUT
Obviously, naming the parameter for --input as OUTPUT is not a very
good idea, so this commit fixes that to name it "INPUT", as expected.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When preparing the legal-info, the source archives are copied in the
legal-info/ output directory. When the archives are big, it can take
quite a bit of time and unnecessarily uses disk space. When the
legal-info output directory is on the same filesystem as the BR2_DL_DIR,
we can easily reduce copy time and disk usage by just using hardlins
instead of copying. However, the BR2_DL_DIR may be on a different
filesystem, so we must fallback to copying in this case
Introduce a helper script that copies a source file into a destination
directory, by first attempting to hard-link, and falling back to a
plain copy in case the hardlink fails.
In case the destination already exists, it is forcibly removed first, to
avoid clobering any existing target file (and especially any hardlink to
it), since cp -f does not remove the destination file, but clobbers it.
In some situations, it will be necessary that the destination file is
named differently than the source, so if a third argument is specified,
it is treated as the basename of the destination file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
At least syslinux is installing stuff in HOST_DIR/sbin.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Make graph-depends script opening the output file in text mode since
only ascii characters will be written.
This change fixes the following error occuring when the default host
python interpreter is python3:
make: Entering directory '/opt/buildroot'
Getting targets
Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...]
Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...]
Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...]
Getting version for ['toolchain-external', 'toolchain', 'busybox', ...]
Traceback (most recent call last):
File "/opt/buildroot/support/scripts/graph-depends", line 425, in <module>
outfile.write("digraph G {\n")
TypeError: a bytes-like object is required, not 'str'
Makefile:807: recipe for target 'graph-depends' failed
make[1]: *** [graph-depends] Error 1
Makefile:84: recipe for target '_all' failed
make: *** [_all] Error 2
make: Leaving directory '/opt/buildroot'
While with python2, adding 'b' to the openning mode has no effect on
Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]).
Therefore, just open the outfile in default (i.e. text) mode.
[1] https://docs.python.org/3/library/functions.html#open
[2] https://docs.python.org/2/library/functions.html#open
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, these flags are recursively propagated. This behavior is
not expected by users, because it can cause dependencies explosively.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, without the flag -recommend, scancpan takes as dependency
only one which has the relationship "requires"; this mode works fine.
And, with the flag -recommend, scancpan takes all ones (ie. with
relationship "requires" or "recommends") in the same way; this mode
never works fine, because it is too simplistic.
With this commit, the "not required" dependencies are handled as
optional BR package or skipped if a cyclic dependency is detected.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The way we use it, gzip will store the current time in the header, which
leads to unreproducible archives.
Fix that by telling gzip to not store the name and date of the file it
compresses, with the -n option. Since it compresses its stdin, there was
already no filename stored; now there's even no date stored.
Note: gzip has had -n since at least 1.2.4, released in 1993, so
virtually every gzip out there nowadays has it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Just like the --stop-on and --exclude options allow to stop on or
exclude virtual packages from the list by passing the "virtual" magic
value, this commit extends the graph-depends logic to support a "host"
magic value for --stop-on and --exclude. This will allow to draw the
graph by stopping on host packages, or by excluding host packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: minor code beautification suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The condition to determine if a virtual package should be excluded
from the list due to "virtual" being passed in --exclude is under a
loop iterating over each entry of the exclude_list, but it doesn't use
the iterator of this list.
Indeed, the condition contains:
"virtual" in exclude_list
which checks automatically if "virtual" was passed in the list. Due to
this, there is no need for this check to be within the "for p in
exclude_list" iteration. This commit fixes that by moving the check
outside of the loop.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add an option to graph-depends to only do the dependency checks and not
generate the dot program.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, if there is a circular dependency in the packages, the
graph-depends script just errors out with a Python RuntimeError which is
not caught, resulting in a very-long backtrace which does not provide
any hint as what the real issue is (even if "RuntimeError: maximum
recursion depth exceeded" is a pretty good hint at it).
We fix that by recursing the dependency chain of each package, until we
either end up with a package with no dependency, or with a package
already seen along the current dependency chain.
We need to introduce a new function, check_circular_deps(), because we
can't re-use the existing ones:
- remove_mandatory_deps() does not iterate,
- remove_transitive_deps() does iterate, but we do not call it for the
top-level package if it is not 'all'
- it does not make sense to use those functions anyway, as they were
not designed to _check_ but to _act_ on the dependency chain.
Since we've had time-related issues in the past, we do not want to
introduce yet another time-hog, so here are timings with the circular
dependency check:
$ time python -m cProfile -s cumtime support/scripts/graph-depends
[...]
28352654 function calls (20323050 primitive calls) in 87.292 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.012 0.012 87.292 87.292 graph-depends:24(<module>)
21 0.000 0.000 73.685 3.509 subprocess.py:473(_eintr_retry_call)
7 0.000 0.000 73.655 10.522 subprocess.py:768(communicate)
7 73.653 10.522 73.653 10.522 {method 'read' of 'file' objects}
5/1 0.027 0.005 43.488 43.488 graph-depends:164(get_all_depends)
5 0.003 0.001 43.458 8.692 graph-depends:135(get_depends)
1 0.001 0.001 25.712 25.712 graph-depends:98(get_version)
1 0.001 0.001 13.457 13.457 graph-depends:337(remove_extra_deps)
1717 1.672 0.001 13.050 0.008 graph-depends:290(remove_transitive_deps)
9784086/2672326 5.079 0.000 11.363 0.000 graph-depends:274(is_dep)
2883343/1980154 2.650 0.000 6.942 0.000 graph-depends:262(is_dep_uncached)
1 0.000 0.000 4.529 4.529 graph-depends:121(get_targets)
2883343 1.123 0.000 1.851 0.000 graph-depends:246(is_dep_cache_insert)
9784086 1.783 0.000 1.783 0.000 graph-depends:255(is_dep_cache_lookup)
2881580 0.728 0.000 0.728 0.000 {method 'update' of 'dict' objects}
1 0.001 0.001 0.405 0.405 graph-depends:311(check_circular_deps)
12264/1717 0.290 0.000 0.404 0.000 graph-depends:312(recurse)
[...]
real 1m27.371s
user 1m15.075s
sys 0m12.673s
The cumulative time spent in check_circular_deps is just below 0.5s,
which is largely less than 1% of the total run time.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, graph-depends outputs the dotfile program to stdout, and uses
stderr to trace the dependencies it is currently looking for.
Redirection was done because the output was directly piped into the dot
program to generate the final PDF/SVG/... dependency graph, but that
meant that an error in the graph-depends script was never caught
(because shell pipes only return the final command exit status, and an
empty dot program is perfectly valid so dot would not complain).
Add an option to tell graph-depends where to store the generated dot
program, and keep stdout as the default if not specified.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with
the rest of the new option naming.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Leverage the CSV files produces by size-stats (make graph-size) to allow
for a comparison of rootfs size between two different buildroot
compilations.
The script takes the file-size CSV files of two compilations as input, and
produces a textual report of the differences per package.
Using the -d/--detail flag, the report will show the file size changes
instead of package size changes.
The -t/--threshold option allows to ignore file size differences smaller
or equal than the given threshold (in bytes).
Example output is:
Size difference per package (bytes), threshold = 0
--------------------------------------------------------------------------------
-8192 busybox
228572 added dmalloc
301584 added jq
--------------------------------------------------------------------------------
521964 TOTAL
or with detailed view:
Size difference per file (bytes), threshold = 0
--------------------------------------------------------------------------------
-8192 bin/busybox
18152 added usr/bin/jq
39252 added usr/bin/dmalloc
46968 added usr/lib/libdmalloc.so
47288 added usr/lib/libdmallocxx.so
47316 added usr/lib/libdmallocth.so
47748 added usr/lib/libdmallocthcxx.so
283432 added usr/lib/libjq.so.1.0.4
--------------------------------------------------------------------------------
521964 TOTAL
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This patch adds a Vagrant file to buildroot. With this file
you can provision a complete buildroot developing environment
in minutes on all major platforms (Linux/Mac/Windows).
[Peter: bump to 2GB RAM, hardcode Buildroot release, add unzip,
drop website update and tweak manual text as suggested by Yann]
Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In some cases, the toolchain sources are now recovered and available in
the legal-info output. So, adapt the header to use conditional instead
of an definitive negation.
Also update the part about saving the sources: it's not the license list
that defines whether sources are installed, but rather whether the
package is redistributable or not.
Update the header accordingly.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
It hasn't been updated since it was added in 2008, and nowadays things kind
of stuff should be handled with genimage.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Creating temporary files in /tmp (or the path pointed by $TMPDIR) allows the
buildroot top directory to be read-only and shareable between multible builds.
This follows what other scripts do, e.g. check-kernel-headers.sh.
Signed-off-by: Henrique Marks <henrique.marks@datacom.ind.br>
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Allows user to specify other access methods than :pserver:anonymous@
on CVS repositories. This shall be defined in the <pkg>_SITE variable.
[Thomas:
- as suggested by Yann, quote the variable expansion
- as suggested by Yann, use a regexp match
- tweak commit log]
Signed-off-by: Joao Mano <joao@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In efe7f68 (support/download: generate reproducible Bazaar archives),
bzr was instructed to store files with the timestamp set to the date
they were last modified in the repository, instead of the current date,
using the --per-file-timestamp option.
However, this option has been added only in bzr-2.2 (August 2010) which
is not available on older distros.
We fix that by not using --per-file-timestamp when the bzr version is
older than 2.2, in which case we just generate the archive with the
current date set on files.
This means the archive is thus non-reproducible, and if a hash is
available for that archive, the hash will not match, and Buildroot will
try to download from the mirror (if any) or fail (if no mirror).
Fixes:
http://autobuild.buildroot.org/results/51f/51f4ff5462c15a85937d411f457096224d00fdcdhttp://autobuild.buildroot.org/results/b88/b8828b5fbc16128408c2f44169ac23de7e34d770http://autobuild.buildroot.org/results/fb4/fb4b0fb2131b40c18273dbe5e51b393cb6df18ec
...
[Peter: simplify sed invocation]
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The apply-patches.sh script was using a mix of tabs and spaces, and
some three-space indentation. Normalize everything to four-space
indentation.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A copy/paste error in the ArgumentParser() constructor call disclosed
the fact that the author of the script has shamefully based his work
on the existing graph-build-time script. This commit fixes this
mistake, therefore hiding in a better way how size-stats was
vampirized from graph-build-time.
Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Similarly to what has previously been done for the Hg download backend,
instruct bzr to generate the archive on stdout, so that we can generate
reproducible archives.
When instructing bzr to generate the output file by itself, it uses a
temporary file that is then fed to gzip, which in turn stores the
timestamp of that file in the generated archive, whereas when the output
is generated on stdout, there is no timestamp, so the archive is then
reproducible.
Bizarely enough, we can tell 'bazaar' not to generate a bazaar in the
archive. Cool, uh? ;-]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When hg directly creates the output file, the hash for that file changes
everytime.
However, if we just tell hg to output the archive on stdout and we do
the redirect to the file, then the archive is reproducible.
(The reason is that in the first case, a temporary file is created and
then compressed, and gzip is adding the filename and its timestamp in
the gzip header, while in the second case, there is no temporary file,
and thus no timestamp and thus it is reproducible.)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For large configurations, the execution time of
remove_transitive_deps() becomes really high, due to the number of
nested loops + the is_dep() function being recursive.
For an allyespackageconfig, the remove_extra_deps() function takes 334
seconds to execute, and the overall time to generate the .dot file is
6 minutes and 39 seconds. Here is a timing of the different
graph-depends steps and the overall execution time:
Getting dependencies: 42.5735 seconds
Turn deps into a dict: 0.0023 seconds
Remove extra deps: 334.1542 seconds
Get version: 22.4919 seconds
Generate .dot: 0.0197 seconds
real 6m39.289s
user 6m16.644s
sys 0m8.792s
By adding a very simple cache for the results of is_dep(), we bring
down the execution time of the "Remove extra deps" step from 334
seconds to just 4 seconds, reducing the overall execution time to 1
minutes and 10 seconds:
Getting dependencies: 42.9546 seconds
Turn deps into a dict: 0.0025 seconds
Remove extra deps: 4.9643 seconds
Get version: 22.1865 seconds
Generate .dot: 0.0207 seconds
real 1m10.201s
user 0m47.716s
sys 0m7.948s
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Gustavo Zacarias <gustavo.zacarias@free-electrons.com>
[yann.morin.1998@free.fr:
- rename is_dep() to is_dep_uncached(), keep existig code as-is
- add is_dep() as a cached-version of is_dep_uncached()
- use constructs more conform with 2to3
- use exceptions (EAFP) rather than check-before-use (LBYL) to be more
pythonist; that even decreases the duration yet a little bit more!
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some users may provide custom download commands with spaces in their
arguments, like so:
BR2_HG="hg --config foo.bar='some space-separated value'"
However, the way we currently call those commands does not account
for the extra quotes, and each space-separated part of the command is
interpreted as separate arguments.
Fix that by calling 'eval' on the commands.
Because of the eval, we must further quote our own arguments, to avoid
the eval further splitting them in case there are spaces (even though
we do not support paths with spaces, better be clean from the onset to
avoid breakage in the future).
We change all the wrappers to use a wrapper-function, even those with
a single call, so they all look alike.
Note that we do not single-quote some of the variables, like ${verbose}
because it can be empty and we really do not want to generate an
empty-string argument. That's not a problem, as ${verbose} would not
normally contain space-separated values (it could get set to something
like '-q -v' but in that case we'd still want two arguments, so that's
fine).
Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A series file for quilt has a valid syntax of:
fixes/autoconf.diff -p1
fixes/doc-html-local-css.diff -p1
fixes/gnu-inline.diff -p1
However, with the current way that a series file is handled, it will
error out because the -p1 is tried as a file. This is because in the
for loop that iterates the files, we only look for comment lines. Then
each line is used within a bash for loop which uses spaces a
delimiter. In order to fix this, we should only use the string that
comes before a space in the series file.
Note that the format allows for any arbitrary depth to the -pN field.
But since we'll have only one package with -pN fields, and all will be
-p1, we for now always assume -p1. This will have to be fixed whenever
we get a package with other values.
Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
[yann.morin.1998@free.fr: expand comment about the format of a series
file and how we interpret it]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
CC: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Despite the comment saying so, the trailing '/' in the host directory is
not removed. Note however that it is properly removed from extracted
RPATH tags.
This is not visible when the host directory is our default $(O)/host
location, but breaks for user-supplied external host directory, when
the user leaves a trailing slash in the path.
Fix that.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When specifying BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION, a user may want to
specify the SHA of a reference different than a branch or tag.
For instance, Gerrit stores the patchsets under refs/changes/xx/xxx, and
Github stores the pull requests under refs/pull/xxx/head.
When cloning a repository with --bare, you don't fetch these references.
This patch uses --mirror for a full clone, in order to give the user
access to all references of the Git repository.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: "Maxime Hadjinlian" <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.
We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.
With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.
However, it turns our that not all the host programs we build end up
with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS
In this case, there are two situations:
- the program is not linked to one of our host libraries: it in fact
does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it
should have had an RPATH tag pointing to the host directory.
For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.
We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.
[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.
[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a series file exists, we should use every file mentioned in it,
not just the ones ending with .patch or .diff. Also, there's no need
to uncompress anything if it's mentioned in a series file (the tools
that manipulate series files don't support compressed patches).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Doug Kehn <rdkehn@yahoo.com>
Tested-by: Doug Kehn <rdkehn@yahoo.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
'quiet' variable is set and exported, but it is not used. We can safely
remove it.
This variable is inherited from the Makefile of the Linux kernel, and
is not used in Buildroot.
In support/scripts/mkmakefile, 'quiet' value is checked, but the test
is always true ('quiet' is never set to silent_), so the test can be
removed as well.
Signed-off-by: Cédric Marie <cedric.marie@openmailbox.org>
Reviewed-by: "James Knight" <james.d.knight@live.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
add this heuristic when no specific license file is found
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The format of the users table files is non trivial, so it is sometimes
handy to add comments explaining the syntax (or simply the reason for
the user) inline in the files.
Ignore empty lines and comment lines prefixed with '#' similar to shell
or makedevs files.
Packages that defined no user (the vast majority) would cause an empty
line to be present in the internal users table, hence the reason we
skipped empty usernames. Now that we ignore empty lines, we no longer
need to check for empty usernames.
Reported-by: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This new script uses the data collected by the step_pkg_size
instrumentation hook to generate a pie chart of the size contribution
of each package to the target root filesystem, and two CSV files with
statistics about the package size and file size. To achieve this, it
looks at each file in $(TARGET_DIR), and using the
packages-file-list.txt information collected by the step_pkg_size
hook, it determines to which package the file belongs. It is therefore
able to give the size installed by each package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Thomas:
- Check for Thread::Queue, not Thread:Queue.
- Use 'printf' instead of 'echo -e', since printf is POSIX, but not
'echo -e'.]
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
All the complexity with the different ways that CMAKE_C_COMPILER and
CMAKE_C_COMPILER_ARG1 can be set are no longer needed, it's all handled
by the toolchain wrapper now.
Note that it is still necessary to handle this for the host build.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
the perl dependency of cpan module is no longer generated by scancpan,
but added at the infrastructure level
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a module is native or depends of a native module, it must be
disabled for static builds via its Config.in
We detect native modules by looking at the filenames listed in the
MANIFEST. If there is a file which looks like it contains code that
much be compiled (e.g. .c, .h and so on...), then we exclude that
module (and its dependencies) from static builds.
That's what we tried to do so far, but failed when there was a
comment on the same line as the filename in the manifest, like so:
foo-bar.c # Bla bla bla
Fix that by detecting either endof-line (as currently done) or
end-of-string.
For an example of failed build of perl-html-parser, see
http://autobuild.buildroot.net/results/128/128671dfa23d843698a63220c2fac1f44e1d5845/
[Thomas: use better commit log proposed by Yann E. Morin.]
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
with Perl 5.22, Module-Build is no longer a core module
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When the version of a package is a Mercurial tag, the download fails,
with:
abort: unknown revision 'X.Y.Z'!
This is because, in Mercurial, tags are commits like the others, and
when we clone, we actively request a tag. But then, the server
"dereferences" that tag and sends us the revision pointed to by that
tag. Of course, since the tag is a commit after the revision we got,
we do not have the revision adding the tag.
So, we just have to download the full repository to be sure we have
the tags in our local clone.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit 5f117c3 (webkit: mark as deprecated), generation of the
manual has been broken.
This is because that commit added a deprecated dependency on a
prompt-less symbol, BR2_PACKAGE_WEBKIT_ARCH_SUPPORTS. However, the
generation script does not check that a symbol has a prompt before
it attempts to add it to the deprecated list. So, we end up with
traceback:
Writing the virtual-packages list in:
/home/ymorin/dev/buildroot/O/build/docs/manual/virtual-package-list.txt
Traceback (most recent call last):
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 510, in <module>
buildroot.print_list(list_name, dry_run=args.dry_run, output=output)
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 466, in print_list
item_label=item_label)
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 126, in format_asciidoc_table
enable_choice=enable_choice))
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 350, in _format_symbol_prompt_location
return "| {0:<40} <| {1}\n".format(get_label_func(symbol),
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 458, in <lambda>
get_label = lambda x: self._get_symbol_label(x, mark_depr)
File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 313, in _get_symbol_label
label = symbol.get_prompts()[0]
IndexError: list index out of range
However, we can not use the existing _is_deprecated filter function to
filter out symbols without prompts, because this function is also used
to add a '(deprecated)' tag in the man package list (not that it would
not work, but it does not seem /right/). Furthermore, it could also be
used (but is currently not) to build the list of virtual packages, which
do not have a prompt.
So, introduce a filter function, aptly named _is_deprecated_feature(),
to be used as the filter to find deprecated feature, and keep the
existing _is_deprecated() that can be used in any context to decide
whether a symbol is deprecated or not.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In 50c8b7e (support/download: support -q in all download backends), the
backend were made to respect the quietness of the main Makefile, when -s
is poassed on the 'make' command line. In doing so, they were all made
to be verbose by default.
However, the verbosity of some of the tools, like scp, is very high, and
is in fact intended for debug purposes.
Drop being verbose by default, just use whatever each tool deems normal
output. Only respect the quietness requested by the user.
Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
underscore is not allowed in BR package name.
this problem was found with the Perl module DB_File
which must give the BR package perl-db-file.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 971faf8 (Makefile: fix out-of-tree builds with multiple targets
with 'all') renamed the default target to '_all' to avoid name-clashing.
In doing so, I forgot to also fix the instance in the .PHONY rule.
Fix that now.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
skeleton being a mandatory dependency, we don't want all our packages to
have a link back to that node, the graph would be awful.
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Small optimization so we don't have another 'make' level (caused by the
umask fix) when running the generated makefile.
Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The introduction of <pkg>_STRIP_COMPONENTS broke the build of the
target tar package, because support/dependencies/check-host-tar.mk
defines TAR_STRIP_COMPONENTS to --strip-components. Which leads to
have the package infrastructure do:
$$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS)
which for the tar package evaluates to:
$$(TAR_STRIP_COMPONENTS)=$$(TAR_STRIP_COMPONENTS)
which evalutes to:
--strip-components=--strip-components
Which obviously doesn't work really well. And in fact the
TAR_STRIP_COMPONENTS definition in
support/dependencies/check-host-tar.mk is no longer necessary: it was
needed in the days where we were trying to support old tar versions
that did not support --strip-components. But nowadays, when such an
old tar version is encountered, we build our own host-tar which
supports --strip-components.
Fixes:
http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is useful when a tag is not avaiable.
Also fix support for Fedora where the command "cvs -r :<version>" doesn't work.
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Kconfiglib now runs as either Python 2 or Python 3.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
These weren't available when gen-manual-lists.py was first written.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Buildroot doesn't use $srctree from what I could tell.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Corresponds to a95f477 in https://github.com/ulfalizer/Kconfiglib.
Fixes:
- Unset user values when loading a zero-byte .config. (5e54e2c)
- Ignore indented .config assignments. (f8a7510)
- Do not require $srctree to be set for non-kernel projects. (d56e9c1)
- Allow digits in $-references to symbols. (ecacdd5)
- Add Symbol.is_allnoconfig_y(). (deaa624)
- Fix small output issue with Comments inside Choices.
Also adds Python 3 support and has a lot of internal cleanup and
optimization.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The following allows a user definition to specify that a created user
entry should not have a password value set. Original implementation
allowed a user definition to provide a password value of "=" (no quotes)
to generate a crypt-encoded empty string value. In some cases, it may be
desired to have no value specified for a user's password. By using a
value "-" for a password, no value will be set in the shadow value.
An example when this can be used is when logging into a terminal.
Logging into a session with an encoded empty password will prompt a user
to enter a password since it does not know the password is empty. If the
password field blank, a login session will not prompt for a password.
Signed-off-by: James Knight <james.knight@rockwellcollins.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Following commit 95a572282e (pkg-infra: move the git download helper to a
script, 2014-07-02), move the comment describing the shallow clone trickery as
well. Merge this comment with the existing helper comment that was added in
7e40a1103a (support/download: convert git to use the wrapper, 2014-08-03).
Rename $($(PKG)_DL_VERSION) to ${cset} to match the helper code context.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that custom external toolchains to be downloaded properly instruct
to not fail on a missing hash, restore the mandatory hash check for
everything else.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
For clarity, this commit renames the TARGETS variable to the more
meaningful PACKAGES variable. Indeed, only packages (handled by one of
the package infrastructures) should be listed in this variable, and
not other random non-package targets.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: fix issues noticed by Arnout:
- Rewrap the linux/Config.in paragraph
- Revert the "is a toolchain dependency" -> "has a toolchain
dependency" change from pkg-generic.mk, as the original was
correct.]
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When downloading from a repository, we explicitly pass no hash file,
because we can't check hashes in that case.
However, we're still printing a message that there is a missign hash
file.
Beside being a bit annoying (since we can't do anything about it), it
may also be wrong, especially for packages for which we support multiple
versions, with some being downloaded via a git clone and others as
tarballs.
Just print no warning when the path to the hash file is empty.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When the user selects a custom toolchain to be downloaded, there's no
hash for that toolchain, so the download fails, now that hashes are
mandatory.
Fix that by simply exiting as if there was no error, until we have a
better fix...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of silently accepting a missing .hash file, print a warning.
This can be grepped from a build log, to find packages that still have
no hash, with the long-term goal of adding hashes for all packages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
At the time we introduced hashes, we did not want to be too harsh in the
beginning, and give people some time to adapt and accept the hashes. So
we so far only whined^Wwarned about a missing hash (when the .hash file
exists).
Some time has passed now, and people are still missing updating hashes
when bumping packages.
Let's make that warning a little bit more annoying...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When checking hashes reports no hash for a file, and this is treated as
an error (now: because BR2_ENFORCE_CHECK_HASH is set; later: because
that will be the new and only behaviour), exit promptly in error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Return different exit codes depending on the error that occured:
0: no error (hash file missing, or all hashes match)
1: unknown option
2: hash file exists, but at least one hash in error
3: hash file exists, but no hash for file to check
4: hash file exists, but at least one hash type unknown
This will be used in a later patch to decide whether the downloaded file
should be kept or removed.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add support to explicitly state that an archive has no hash.
This can be used for archives downloaded from a repository, like a
git-clone or a subversion checkout, or using the github helper.
This will come in handy when we'll eventually make hashes mandatory as
soon as a .hash file exists: for some packages, like gcc, some versions
are downloaded as archives from upstream, while other versions may come
from a GitHub repository (via the github herlper).
In this case, a .hash file would exist, that contains hashes for the
downloaded tarballs, but archives downloaded from the repository would
not have a hash (since it is currently not possible to have reproducible
such archives). So, we'd need a way to explicitly state there is no
hash, on purpose, for those archives.
So, add 'none' as a new type of hash.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, specifying a hash file for our download wrapper is mandatory.
However, when we download a git, svn, bzr, hg or cvs tree, there's by
design no hash to check the download against.
Since we're going to have hash checking mandatory when a hash file
exists, this would break those downloads from a repository.
So, make specifying a hash file optional when calling our download
wrapper and bail out early from the check-hash script if no hash file is
specified.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Like for --stop-on, make --exclude recognise the keyword 'virtual',
to stop on virtual packages (as explained in the help...).
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Similar to --stop-on, but also omits the package from the graph.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a new option to graph-depends, that users can set to stop the graph
on a specific (set of) package(s).
This accepts any actual package name, or the 'virtual' keyword to stop
on virtual packages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The current error messages are a bit terse, and do not provide all the
required information.
Expand them to provide more context.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Tom Elliott <tommygunsster@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change prevents CMake from searching outside the sysroot location
for CMake modules when cross-compiling.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Most of targets listed in TARGET_EXCEPTIONS these days are long
gone, so why still keep them?
Most of those targets were removed in this commit:
http://git.buildroot.net/buildroot/commit/?id=02b88600312554bf166f6cfd71f7f2ede783096a
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@sunsite.dk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
virtual packages are found by their version,
so we retrieve the version of all packages
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
