This is useful on CentOS 7 whose "cmake" package provides cmake 2.8.12,
which is too old, but the "cmake3" package (from EPEL) provides version
3.6.3, which is satisfactory. Examples:
$ sh support/dependencies/check-host-cmake.sh 2.8 cmake cmake3
/usr/bin/cmake
$ sh support/dependencies/check-host-cmake.sh 3.1 cmake cmake3
/usr/bin/cmake3
$ sh support/dependencies/check-host-cmake.sh 3.8 cmake cmake3
(nothing)
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Pass the minimal version before the program name. In a later change the
script will become able to test a list of candidates.
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit fixes a problem where it was not possible to replace
/etc/shadow with a symlink to a e.g. a user partition where the
shadow file is placed. This is required, e.g. for systems where the
rootfs is mounted read-only but users should still be able to be
added. Thus, if within an filesystem overlay setup a user tries
to replace /etc/shadow with a symlink to the real file on a user
partition a buildroot build stops with an error message because
sed is called on the symlink instead of following the symlink.
This commit fixes this shortcoming.
Signed-off-by: Jens Maus <mail@jens-maus.de>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The API v0 is shutdown.
see https://bugs.busybox.net/show_bug.cgi?id=9951
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... otherwise it fails spectacularly as soon as PATH is referenced in a
package rule (i.e. very soon, fortunately):
>>> host-lzip 1.18 Downloading
/bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
/bin/bash: -c: line 1: syntax error: unexpected end of file
Fixes # 9886.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ciro Santilli <ciro.santilli@gmail.com>
[Thomas: fix typo in message, use tabs for indentation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The 'lines' variable is overwritten with its own fields. Thus it
contains a line first, and then a list of fields -- it never contains
'lines'.
Use two different variables named 'line' and 'fields' to make the code
more readable.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We only have a positive test for it, in ext4. Let's have a negative
one as well.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently call infra.smart_open() to open log files each time we
need to write to them.
Opening the file once in the constructor of Builder and Emulator and
writing to it whenever needed is simpler and slightly more efficient.
Remove smart_open and instead create a new open_log_file() function
which just opens the logfile. Also let it compute the filename, in
order to simplify even further the Builder and Emulator code.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This is what the manpages usually do, and what Python does with the
automatically-added -h/--help parameter:
Before the change:
$ ./support/testing/run-tests
[...]
optional arguments:
-h, --help show this help message and exit
--list, -l list of available test cases
--all, -a execute all test cases
After the change:
$ ./support/testing/run-tests
[...]
optional arguments:
-h, --help show this help message and exit
-l, --list list of available test cases
-a, --all execute all test cases
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
has_broken_links makes it self-explanatory that this is a predicate
function, and that the return value tells whether there _are_ broken
links, not the opposite.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently messages from run-tests are quite unpleasant:
[br-tests/TestPostScripts/2017-05-09 15:51:57] Building
[br-tests/TestPostScripts/2017-05-09 15:52:23] Building done
[br-tests/TestPostScripts/2017-05-09 15:52:23] Cleaning up
.[br-tests/TestNoTimezone/2017-05-09 15:52:23] Starting
[br-tests/TestNoTimezone/2017-05-09 15:52:23] Building
[br-tests/TestNoTimezone/2017-05-09 15:53:17] Building done
[br-tests/TestNoTimezone/2017-05-09 15:53:22] Cleaning up
.[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Starting
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Building
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:33] Building done
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:37] Cleaning up
[...]
Change them in a more readable way by removing the date and using a
columnar style:
15:12:22 TestPostScripts Starting
15:12:25 TestPostScripts Building
15:12:48 TestPostScripts Building done
15:12:48 TestPostScripts Cleaning up
.15:12:48 TestNoTimezone Starting
15:12:54 TestNoTimezone Building
15:13:44 TestNoTimezone Building done
15:13:49 TestNoTimezone Cleaning up
.15:13:49 TestGlibcNonDefaultLimitedTimezone Starting
15:14:00 TestGlibcNonDefaultLimitedTimezone Building
15:14:56 TestGlibcNonDefaultLimitedTimezone Building done
15:15:01 TestGlibcNonDefaultLimitedTimezone Cleaning up
[...]
Note the '.' and other characters presented by nose2 are still
printed. They are not affected by this change.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds an initial toolchain test case, testing the ARM
CodeSourcery toolchain, just checking that the proper sysroot is used,
and that a minimal Linux system boots fine under Qemu.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds some basic tests for two Buildroot packages: python and
dropbear. These tests are by no mean meant to be exhaustive, but mainly
to serve as initial examples for other tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds a number of test cases for various filesystem formats:
ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them
except yaffs2 are runtime tested. The iso9660 set of test cases is
particularly rich, testing the proper operation of the iso9660 support
with all of grub, grub2 and isolinux.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds a few Buildroot "core" tests, testing functionalities
such as:
- post-build and post-image scripts
- root filesystem overlays
- timezone support
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When called from BR2_ROOTFS_POST_IMAGE_SCRIPT, this script
ends up with following error:
Error: Missing argument
This is because, an extra positional argument is also passed
along with BR2_ROOTFS_POST_SCRIPT_ARGS. genimage.sh didn't
have support to parse positional and optional arguments
together.
Signed-off-by: Abhimanyu Vishwakarma <Abhimanyu.V@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently the check-package script uses many files in the same
directory. This commit keeps the main script in support/scripts/ and
moves the rest into a subdirectory.
The modules were previously prefixed to make it easy to identify which
script they belong to. This is no longer needed when using a
subdirectory, so the prefix is removed.
Note: if this commit is checked out and the script is run, and later on
a previous version is checked out, the file
support/scripts/checkpackagelib/__init__.pyc needs to be manually
removed to prevent Python interpreter to look for checkpackagelib
package when only the checkpackagelib module is available.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The removal of the .git dir before creating the tarball is not anymore
just an optimization. It is necessary to make the tarball reproducible.
Also, without the removal, large tarballs (gigabytes) would be created
for some linux trees.
Update the comment accordingly.
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain name was calculated in main() for reporting to the user,
and again in build_one() for creating the build directory. Calculate
it only once, in main(), and pass the build directory as an argument
to build_one().
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This allows the page at http://autobuild.buildroot.net/stats/ to show
how many warnings returned by check-package affect each package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Warn when help text is larger than 72 columns, see [1].
Warn for wrongly indented attributes, see [1].
Warn when the convention of attributes order is not followed, see [2].
[1] http://nightly.buildroot.org/#writing-rules-config-in
[2] http://nightly.buildroot.org/#_config_files
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Check each hash entry (see [1]) and warn when:
- it does not have three fields;
- its type is unknown;
- its length does not match its type;
- the name of the file contains a directory component.
[1] http://nightly.buildroot.org/#adding-packages-hash
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Create 3 new check functions to warn when:
- there are consecutive empty lines in the file, see [1];
- the last line of the file is empty, see [2];
- there are lines with trailing whitespace, see [3].
Apply these functions to Config.*, *.mk and *.hash, but not for *.patch
files since they can contain any of these and still be valid.
[1] http://patchwork.ozlabs.org/patch/682660/
[2] http://patchwork.ozlabs.org/patch/643288/
[3] http://patchwork.ozlabs.org/patch/398984/
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Create the infra to check the style of new packages before submitting.
The overall function of the script is described inside a txt file.
It is designed to process the actual files and NOT the patch files
generated by git format-patch.
Also add the first check function, to warn if a file (Config.*, *.mk,
*.hash, *.patch) has no newline at the last line of the file, see [1].
Basic usage for simple packages:
support/scripts/check-package -vvv package/newpackage/*
Basic usage for packages with subdirs:
support/scripts/check-package -vvv $(find package/newpackage/ -type f)
See "checkpackage" in [2].
[1] http://patchwork.ozlabs.org/patch/631129/
[2] http://elinux.org/Buildroot#Todo_list
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script is a wrapper for the genimage tool used by most boards.
The board postimage script can now call this script instead of invoking
genimage command themselves.
Signed-off-by: Etienne Phelip <etienne.phelip@savoirfairelinux.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Unextected error in the br2-external script are properly caught, but
they are not reported properly, and we end up in either of two
situations:
- the .br2-external.mk file is not generated, in which case make will
try to find a rule to generate it (because the 'include' directive
tries to generate missing files);
- the .br-external.mk file is generated but does not contain the error
variable, and thus the build might not get interrupted.
We fix that by using a trap on the pseudo ERR signal, to emit the error
variable on unexpected errors.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In commit 2f6c5e513c
("support/check-bin-arch: fix for filenames with spaces"), Yann
adjuste the check-bin-arch script to properly handle filenames with
spaces.
However, he also did a subtle change of the regexp that extracts the
path of the files. It was:
"/^${package},(.+)$/!d; s//\1/;"
and Yann changed it to:
"/^${package},\.(.+)$/!d; s//\1/;"
So the file paths used to start with a dot (like "./usr/share/foo"),
and now they no longer start with a dot (like "/usr/share/foo"). While
this modification is good and makes sense, the match for
/lib/firmware/ was not adjusted accordingly, and the follow-up patch
also ignoring /usr/share was not adjusted as well.
This commit fixes those /lib/firmware/ and /usr/share/ special cases,
which will fix:
http://autobuild.buildroot.net/results/76a1475f4cdedb80426fb022ef2e644aa5625660/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
On most distros, the tar format defaults to GNU. However, at build time
the default format may be changed to posix. Also, future versions of
tar will default to posix.
Since we want the tarballs created by the git download method to be
reproducible (so their hash can be checked), we should explicitly
specify the format. Since existing tarballs on sources.buildroot.org
use the GNU format, and also the existing hashes in the *.hash files
are based on GNU format tarballs, we use the GNU format.
In addition, the Posix format encodes atime and ctime as well as mtime,
but tar offers no option like --mtime to override them. In the GNU
format, atime and ctime are only encoded if the --incremental option is
given.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
pkgutil.py is also part of Python itself. Placing pkgutil.py as is
in a folder with other scripts that require original pkgutil will
break them. This is the case with scanpypi. So rename pkgutil.py
to brpkgutil.py to avoid naming collision.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=9766
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
/usr/share normally should not contain binaries executable for the
target platform. However, it might contain ELF binaries for other
platforms, such as firmware files installed by Qemu or
pru-software-support.
Instead of special-casing each package, let's simply ignore /usr/share.
Fixes:
http://autobuild.buildroot.net/results/6f3fea9f6adaef1573fbb0dd6903b5d99e470610/
(pru-software-support)
http://autobuild.buildroot.net/results/fe8892bc22a03299fc41e30bfea5e42166838f88/
(qemu)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Filenames with spaces will break the current for loop.
Fix that by using a while-read loop, fed with the list of files on
stdin, using process substitution.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
By default, compile_dir() relies on the modification time to know if a
python file has to be built again. However in some circumstances (when
doing reproducible builds), modification times are not reliable. Thus,
this patch adds a way to force the rebuild of all python sources.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.
This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.
Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.
Example output with the firejail package enabled, when building for an
ARM target:
ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM
Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As of the version 3.6.0 compile_dir() call will treat its 'quiet'
argument as a full blown integer rather than a boolean value and perform
integer comparison operations such as '<' or '>='.
To account for that convert ReportProblem type to be a true derivative
of built-in int() and override all of int's rich comparison operators in
order to be able to "sniff" for PyCompileError in all possible use-cases
The integer value ReportProblem pretends to be is teremined by class
variable VALUE which is set to 1.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we tell cmake where to look for our own custom platform
description by passing the path to the moduls directory on the command
line.
However, this causes two different problems.
First, some packages simply set CMAKE_MODULE_PATH in their
CMakeList.txt, thus overriding our own path, and then our platform
description is not found.
Second, cmake may internally call sub-cmake (e.g. in the try_compile
macro), but the CMAKE_MODULE_PATH is not automatically passed down in
this case.
For the first problem, we could hunt down and fix all offenders, but
this is an endless endeavour, especially since packagers are told to do
so on the cmake wiki [0]:
CMAKE_MODULE_PATH
tell CMake to search first in directories listed in
CMAKE_MODULE_PATH when you use FIND_PACKAGE() or INCLUDE()
SET(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/MyCMakeScripts)
FIND_PACKAGE(HelloWorld)
The second problem could be solved by passing yet another variable on
the command line, that tells cmake to explicitly pass arbitrary
variables down to sub-cmake calls:
-DCMAKE_TRY_COMPILE_PLATFORM_VARIABLES=CMAKE_MODULE_PATH
However, this only covers the case of try_compile. Even though no other
case is known yet, we'd still risk missing locations where we would need
to propagate CMAKE_MODULE_PATH, even some where we'd have no solution
like for try_compile.
Instead, ngladitz on IRC suggested that CMAKE_MODULE_PATH be set
directly from the toolchain file.
And indeed this fixes both problems explained above.
So be it.
[0] https://cmake.org/Wiki/CMake_Useful_Variables
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The LINES variable is automatically set by bash to represent the number
of lines in the terminal. That variable can be set when the shell
receives SIGWINCH.
If the shell does receive SIGWINCH after our LINES array is filled, the
content of the array is mangled.
Rename the variable to avoid that.
Fixes#9456
Reported-by: George Y. <georgebrmz@oss3d.com>
Reported-by: Paul Stewart <paulstewartis@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some packages test the system name to decide whether to enable/disable
features or link with specific libs.
So we forcefully set the system name form our custom system file, so
that packagses still believe they are running on Linux rather than
Buildroot.
Fixes:
fastd : http://autobuild.buildroot.net/results/f1d/f1dfe90068ad62e733f17a22202235415bda3974/
paho-mqtt-c: http://autobuild.buildroot.net/results/457/457d76279e16247bf58c838a2c5dd0a4f3962c21/
libiio : http://autobuild.buildroot.net/results/281/2812b008a0ab6bab5fe4d45eb9ffe4e9496a8cb4/
and so on...
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This reverts commit 4422eca2d4.
We now have a workaround for the RPATH issue introduced in 3.7, so we
can use it again.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The handling of RPATH in cmake-3.7 has changed drastically, causing a
slew of build failures dues to libraries from the host being pulled in:
- domoticz : http://autobuild.buildroot.org/results/fd0/fd0ba54c7abf973691b39a0ca1bb4e07d749593a/
- freerdp : http://autobuild.buildroot.org/results/5d4/5d429d0e288754a541ee5d8be515454c5fccd28b/
- libcec : http://autobuild.buildroot.org/results/3f3/3f3593bab7734dd274faf5b5690895e9424cbb89/
- and so on...
The bug was reported upstream [0], which dismissed it altogether [1] as
being expected behaviour, quoting:
I don't think there is anything wrong with that change on its own.
It merely exposed some existing behavior in a new case.
Instead, upstream suggested in that same message that a platform
definition be used instead, quoting:
If a toolchain file specifies CMAKE_SYSTEM_NAME such that a custom
`Platform/MySystem.cmake` file is loaded then the latter can set
them as needed for the target platform.
So here we are doing so:
- we add a new platfom definitions that inherits from the Linux one,
then overrides the problematic settings;
- we change our toolchain file to use that platform instead;
- we tell cmake where to find additional modules, so that it can find
our custom platform file.
This has been tested to work in the following conditions:
- pre-installed host cmake, versions 3.5.1 (Ubuntu 16.04) and 3.7.2
(manually built)
- internal cmake, versions 3.6.3 (the current version as of this
patch) and 3.7.2 (with the followup patches).
Thanks to Jörg, Ben and Baruch for the help investigating the issue.
Special thanks to Jörg for handling the discussion with upstream and
pointing to the relevant messages! :-)
[0] http://public.kitware.com/pipermail/cmake/2017-February/064970.html
[1] http://public.kitware.com/pipermail/cmake/2017-February/065063.html
To be noted: Thomas suggested we set these directly in the toolchain
file. Unfortunately, wherever we put those settings in the toolchain
file, this does not work.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
cmake-3.7 has a bug in how it handles rpath, linking with libraries from
the host.
Until we completely understand the issue, just blacklist cmake-3.7.
The issue has been reported upstream:
http://public.kitware.com/pipermail/cmake/2017-February/064970.html
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
* Remove distribution upgrade cause it slows down the first boot and
presents a bug when executed non interactively.
* Reorganize provision scripts to be in privileged and non privileged
sections
* Add Ubuntu mirror automatic handling for apt packages sources
Fixes bug #9581
Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit teaches the generic package handling code how to extract .tar.lz
archives. When lzip is not installed on the host, host-lzip gets built
automatically.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Don't special case $(XZCAT) when constructing DL_TOOLS_DEPENDENCIES. The next
commit will introduce another extractor that automatically builds when not
installed. Introduce EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS that lists
archive extensions for which the extractor is already checked in
support/dependencies/check-host-foo.mk. Use this in the newly introduced
extractor-dependency to populate DL_TOOLS_DEPENDENCIES.
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Thomas: add missing space after "firstword", as noticed by Thomas DS.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Requested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use comm(1) to check that all our config options are properly set in the
resulting configuration, rather than our canned and fragile code.
Reported-by: Cam Hutchison <camh@xdna.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Sometimes, it interesting to have a global overview of whether the
package builds at all or not, rather than test on all toolchains.
Add an option that allows testing on a limited set of randomly choosen
toolchains.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a build is skipped, store the lines from the config snippet, that
are missing in the resulting configuration, in a file in the build
directory, for the user to inspect.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script helps in testing that a package builds fine on a wide range
of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr:
- completely rewrite the script from Thomas, with help from Luca
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change adds inode tracking to the size-stats script so that hard
links don't cause files to be double counted. This has a significant
effect on the size computation for some packages. For example, git has
around a dozen hard links to a large file. Before this change, git would
weigh in at about 170 MB with the total filesystem size reported as
175 MB. The actual rootfs.ext2 size was around 16 MB. With the change,
the git package registers at 10.5 MB with a total filesystem size of
15.8 MB.
Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.
Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.
As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.
Instead, pass the global as an argument.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Usually, Buildroot does never initialize variables with empty content.
DEPENDENCIES_HOST_PREREQ was an unjustified exception.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixes#9576
When the path to a br2-external tree is relative, make enters an endless
recursive loop (paths elided for brevity):
$ make BR2_EXTERNAL=.. foo_defconfig
make[1]: stat: ../configs/../configs/../configs[...]/toto_defconfig: Filename too long
make[1]: *** No rule to make target '../configs/../configs/../configs[...]/toto_defconfig',
needed by '../configs/../configs/../configs[...]/toto_defconfig'. Stop.
Makefile:79: recipe for target '_all' failed
make: *** [_all] Error 2
It is a bit complex to understand the actual technical reason for this
never-ending expansion; it seems it happens in the code generated by the
percent_defconfig macro. Not sure why, though...
But the root cause is the relative path.
Just use absolute, canonical paths to br2-external trees. Always.
[Peter: add bugzilla reference]
Reported-by: outtierbert@gmail.com
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Recently, the autoconf macros for libtool started using '/usr/bin/file'
to determine the type of library that is generated by the toolchain.
Packages that use this recent version of the libtool autoconf macros
will fail in a rather dramatic way when /usr/bin/file is not present
on the host: the package will still build but no shared library is
generated, which in turn may cause build failures in other packages
that link with it.
For example, libpng's configure determines that it is not possible to
build a shared library on MIPS64 because the expected output from 'file'
is not present. Therefore, only a static libpng.a is built. Later,
bandwithd links with -lpng but it doesn't use the pkg-config's
Private-Libs (because it's not linking statically) and it doesn't have
access to the NEEDED reference from the shared library. Therefore, it
doesn't link with zlib and fails with
pngrutil.c:(.text+0x55c): undefined reference to `inflate'
We cant use host-file because it is itself an autotools package and is
itself using libtool, so this would be a chicken-n-egg problem. Besides,
the libtool script really wants to call /usr/bin/file, so it would not
even find our host-file anyway.
So, just require that '/usr/bin/file' is present on the host.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Many (100+) packages supported by buildroot contain old configure
scripts (or build them from old versions of autotools) that are unable
to determine how to link shared libraries on powerpc64 and
powerpc64le. This causes that test to erroneously fail on toolchains
that are not "bi-endian" (which is the case for toolchains built by
buildroot), which causes configure to build static libraries instead
of dynamic ones. Although these builds succeed, they tend to cause
linker failures in binaries later linked against them.
Because affected configure files can be discovered automatically, this
patch introduces a hook (enabled only when building for powerpc64 and
powerpc64le) that uses a script to scan and fix each package.
Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we require a host with an UTF-8 locale as soone as we build a
Buildroot toolchain with support for locales. This means that we do
erquire such a locale when building a toolchain with either uClibc,
glibc or musl.
However, glibc and musl do not require such a locale to be present.
Use the new option to check if an UTF8 locale is needed on the host,
rather than deriving that from toolchain settings.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the br2-external script uses bash-4's associative arrays.
However, some oldish enterprise-class distros like RHEL5 still use
bash-3.1 which lacks associative arrays.
We restore compatibility with those oldish distros using 'eval' to
emulate associative arrays, as suggested by Arnout.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python3 complains about missing parentheses.
$ ./support/scripts/get-developers
File "./support/scripts/get-developers", line 45
print f
^
SyntaxError: Missing parentheses in call to 'print'
Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This adds an ev3dev Linux drivers extension that provides Linux kernel
drivers for LEGO MINDSTORMS EV3 from the ev3dev project.
Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, there are two failure paths in the wrapper:
- if the tar fails, then the error is ignored because it is on the
left-hand-side of a pipe;
- if the find fails, then the error is ignored because it is a
process substitution (and there is a pipe, too).
While the former could be fixed with "set -o pipefail", the latter can
not be fixed thusly and we must use an intemediate file for it.
So, fix both issues by using intermediate files, both to generate the
list of files to include in the archive, and generate the archive in a
temporary tarball.
Fixes the following build issue, where the find is failing for whatever
unknown reason:
http://autobuild.buildroot.net/results/20f/20fd76d2256eee81837f7e9bbaefbe79d7645ae9/
And this one, where the process substitution failed, also for an unknown
reason:
http://autobuild.buildroot.org/results/018/018971ea9227b386fe25d3c264c7e80b843a9f68/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.
It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Current type for 'patches' argument is str. It supposed to only
contain names of files.
If we specify FileType as type, then we don't need to open file ourself
and it allows script to read patch from standard input as well.
e.g.
$ git show -1 | ./support/scripts/get-developers -
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
From the build configuration, Buildroot defines and set some compiler
and linker flags that should be passed to any packages build-system.
For package using the cmake-package infrastructure, this is achieved
via the toolchainfile.cmake.
This change simplifies the way the toolchainfile.cmake file handles
these flags: it now just sets them, without any attempt to extend them
with those Buildroot defined.
This change still allows overriding these flags from the configure
command line.
So, now, when a CMake-based package needs to extend them, they should
be fully set from the package *.mk file. This behavior is consistent
with what is done for others package infrastructures.
This change should not pull any regression WRT the bug #7280 [1].
However, now, when someone uses the toolchainfile.cmake file outside of
Buildroot, he/she must overload all compiler/linker flags (including the
ones Buildroot sets since they no longer get automatically added).
[1] https://bugs.busybox.net/show_bug.cgi?id=7280
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a build type is set, CMake does append some flags that can override
those set by Buildroot due to the gcc option parser (in which the last
argument controling an option wins).
Hereafter is a summary of the optimization and debug flags set by
Buildroot and appended by CMake.
* Flags set by Buildroot depending on the configuration:
BR2_ENABLE_DEBUG | Optim. level | Buildroot {C,CXX}FLAGS
=================+=====================+=======================
y | BR2_OPTIMIZE_S | -Os -gx
y | BR2_OPTIMIZE_G | -Og -gx
y | BR2_OPTIMIZE_{0..3} | -On -gx
n | BR2_OPTIMIZE_S | -Os
n | BR2_OPTIMIZE_G | -Og
n | BR2_OPTIMIZE_{0..3} | -On
* Default flags appended by CMake depending on the build type:
Build type | Flags | Effects on {C,CXX}FLAGS
===============+=================+===========================================
Debug | -g | Force -g, compatible with BR2_ENABLE_DEBUG
MinSizeRel | -Os -DNDEBUG | Set -Os, compatible with BR2_OPTIMIZE_S
Release | -O3 -DNDEBUG | Set -O3, closest to the others cases,
| | though the optimization level is forced.
RelWithDebInfo | -O2 -g -DNDEBUG | Force -g and set -O2, not friendly with BR
To avoid the CMake flags take precedence over the Buildroot ones, this
change sets in toolchainfile.cmake the per-config compiler flags CMake
can append depending on the build type Buildroot defined.
So, CMake does not mess up with the compilation flags Buildroot sets.
It is still possible to override these per-config flags on the cmake
command line.
Note:
If a CMake-based project forces the compiler and/or linker flag
definitions (the default ones or the per-config ones - e.g.
CMAKE_C_FLAGS/CMAKE_C_FLAGS_{DEBUG,RELEASE}), there is not much
Buildroot can do about it.
So, the flags will be overwritten anyway in these cases.
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- adjust comment in toolchainfile.cmake.in, as suggested by Arnout.
- also handle CMAKE_Fortran_FLAGS_*, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The chosen CMAKE_BUILD_TYPE encodes an option of the Buildroot
configuration, so it makes more sense to save it in the
toolchainfile.cmake than to pass it during configure.
It is still possible to override the build type on the cmake
command line.
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: reword description in the CHANGES file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Output of get-developers script in our manual uses --cc for developers,
but actual output of get-developers script uses --to. This patch makes
code consistent with documentation, by using --cc for developers.
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
To be noted: that link will only be valid once we have a released
manual. In the meantime, it's accessible on the nightly manual:
http://nightly.buildroot.org/#br2-external-converting
Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we support multiple br2-external trees, BR2_EXTERNAL is no
longer exported in the environment.
This means that post-build scripts in a br2-external tree can no longer
find their own files (well, they could re-invent the path by stripping
their known-relative path, but that'd be ugly, especially since we can
very well provide it).
Export the path for each br2-external trees as environment variables.
Do so for the description as well, as a courtesy.
Also, re-order variable definitions to be more logical: first, purely
internal variables, then exported variables.
Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, it is not possible for a br2-external tree to override a
defconfig bundled in Buildroot, nor is it possible to override one from
a previous br2-external tree in the stack.
However, it is interesting that a latter br2-external tree be able to
override a defconfig:
- the ones bundled in Buildroot are minimalist, and almost always
build a toolchain, so a br2-external tree may want to provide a
"better" defconfig (better, in the sense "suited for the project");
- similarly for a defconfig from a previous br2-external tree.
But we can't do that, as the rules for the defconfigs are generated in
the order the br2-external trees are specified, all after the bundled
defconfigs. Those rule are patten-matching rules, which means that the
first one to match is used, and the following ones are ignored.
Add a new utility macro, 'reverse', inspired from GMSL, that does what
it says: reverse a list of words.
Use that macro to reverse the list of br2-external trees, so that the
latters win over the formers, and even over bundled ones.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.
A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.
Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.
Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.
What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:
- the list of all br2-external names, BR2_EXTERNAL_NAMES,
- the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
point each to the actual location of the corresponding tree,
- the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,
- the space-separated list of absolute paths to the external trees,
BR2_EXTERNAL_DIRS.
Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.
This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".
Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.
If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.
This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.
Also, display the NAME and its path as a comment in the menuconfig.
This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.
The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.
Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).
Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A br2-external tree must provide external.mk and Config.in.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have one
(BR2_EXTERNAL is not empty).
There is now no reason to treat those two cases differently:
- the kconfig snippet is always generated appropriately (i.e. it would
include the br2-external tree if set, or include nothing otherwise);
- we no longer have a dummy br-external tree either.
Also, the Makefile code to handle BR2_EXTERNAL is currently quite
readable if at least a little bit tricky.
However, when we're going to add support for using multiple br2-external
trees simultaneously, this code would need to get much, much more complex.
To keep the Makefile (rather) simple, offload all of the handling of
BR2_EXTERNAL to the recently added br2-external helper script.
However, because of Makefiles idiosyncracies, we can't use a rule to
generate that Makefile fragment.
Instead, we use $(shell ...) to call the helper script, and include the
fragment twice: once before the $(shell ...) so we can grab a previously
defined BR2_EXTERNAL value, a second time to use the one passed on the
command line, if any.
Furthermore, we can't error out (e.g. on non-existent br2-external tree)
directly from the fragment or we'd get that error on subsequent calls,
with no chance to override it even from command line.
Instead, we use a variable in which we store the error, set it to empty
before the second inclusion, so that only the one newly generated, if
any, is taken into account.
Since we know the script will always be called from Makefile context
first, we know validation will occur in Makefile context first. So we
can assume that, if there is an error, it will be detected in Makefile
context. Consequently, if the script is called to generate the kconfig
fragment, validation has already occured, and there should be no error.
So we change the error function to generate Makefile code, so that
errors are caught as explained above.
Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget'
about the previous value of the BR2_EXTERNAL_MK variable, especially in
the case where BR2_EXTERNAL is now set to empty, so that we do not try
to include it later. That's why we first generate empty version of
BR2_EXTERNAL_MK, and then assign it the new value, if any.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no needed.
As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer needed.
Ditch both of those files, and:
- only generate actual content in the Kconfig snippet when we actually
do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
empty);
- add a variable that contains the path to the external.mk provided by
the user, or empty if none, and include the path set in that variable
(make can 'include' nothing without any problem! ;-) )
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.
This will ultimately allow us to use more than one br2-external tree.
Offload the "User-provided options" menu to the generated Kconfig
snippet. We can also move the definition of the Kconfig-version of
BR2_EXTERNAL into this snippet.
We introduce an extra check that was not present in the previous code,
to check that we do have permission on that directory. Prevciously, it
was handled as a side effect of not being able to cd into there, but it
is cleaner to check it expressly.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
ExtUtils::MakeMaker is one of the Perl Core modules usually packaged in
Perl package for a Debian/Ubuntu based system.
For a Fedora based system, each Perl Core modules have their own RPM
package. So install only Perl package is not enough.
Fixes:
>>> host-libxml-parser-perl 2.41 Configuring
[...]
perl `which perl` Makefile.PL
Can't locate ExtUtils/MakeMaker.pm in @INC (you may need to install the ExtUtils::MakeMaker module)
Add a new Perl module check in dependency.sh.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: François Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This script, and its companion library, is more-or-less Buildroot's
equivalent to the kernel get_maintainer.pl script: it allows to get the
list of developers to whom a set of patches should be sent to.
To do so, it first relies on a text file, named DEVELOPERS, at the root
of the Buildroot source tree (added in a followup commit) to list the
developers and the files they are interested in. The DEVELOPERS file's
format is simple:
N: Firstname Lastname <email>
F: path/to/file
F: path/to/another/file
This allows to associate developers with the files they are looking
after, be they related to a package, a defconfig, a filesystem image, a
package infrastructure, the documentation, or anything else.
When a directory is given, the tool assumes that the developer handles
all files and subdirectories in this directory. For example
"package/qt5/" can be used for the developers looking after all the Qt5
packages.
Conventional shell patterns can be used, so "package/python-*" can be
used for the developers who want to look after all packages matching
"python-*".
A few files are recognized specially:
- .mk files are parsed, and if they contain $(eval
$(<something>-package)), the developer is assumed to be looking after
the corresponding package. This way, autobuilder failures for this
package can be reported directly to this developer.
- arch/Config.in.<arch> files are recognized as "the developer is
looking after the <arch> architecture". In this case, get-developer
parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH
values. This way, autobuilder failures for this package can be
reported directly to this developer.
- pkg/pkg-<infra>.mk are recognized as "the developer is looking after
the <infra> package infrastructure. In this case, any patch that adds
or touches a .mk file that uses this infrastructure will be sent to
this developer.
Examples of usage:
$ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch
git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>"
$ ./support/scripts/get-developers -p imx-lib
Arnout Vandecappelle <arnout@mind.be>
Gary Bisson <gary.bisson@boundarydevices.com>
$ ./support/scripts/get-developers -a bfin
Waldemar Brodkorb <wbx@openadk.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We currently have four lists of packages in the manual:
- the non-virtual target packages,
- the virtual target packages,
- the host packages,
- the deprecated features.
Those list take more than half of the manual. They do not serve much
purpose except to show off.
After the recent discussion on the list [0], remove them all.
We can now get rid of our biggish and complex generating script (and its
companion library kconfiglib).
[0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently all cmake packages depend on host-cmake. Unfortunately
host-cmake takes a long time to configure and build: almost 7 minutes
on a dual-core i5 with SSD. The time does not change even with ccache
enabled.
Indeed, building host-cmake is avoidable if it is already installed on
the build host: CMake is supposed to be quite portable, and the only
patch in Buildroot for the CMake package seems to only affect
target-cmake.
Thus we automatically skip building host-cmake and use the one on the
system if:
- cmake is available on the system and
- it is recent enough.
First, we leverage the existing infrastructure in
support/dependencies/dependencies.mk to find out whether there's a
suitable cmake executable on the system. Its path can be passed in the
BR2_CMAKE environment variable, otherwise it defaults to "cmake". If
it is enabled, found and suitable then we set BR2_CMAKE_HOST_DEPENDENCY
to empty; otherwise we set BR2_CMAKE_HOST_DEPENDENCY to 'host-cmake' and
override BR2_CMAKE with "$(HOST_DIR)/usr/bin/cmake" to revert to using
our own cmake (the old behaviour).
Then in pkg-cmake.mk we replace the hard-coded dependency on host-cmake
to using the BR2_CMAKE_HOST_DEPENDENCY variable, and we use $(BR2_CMAKE)
instead of $(HOST_DIR)/usr/bin/cmake.
Unlike what we do for host-tar and host-xzcat, for host-cmake we do
not add host-cmake to DEPENDENCIES_HOST_PREREQ. If we did, host-cmake
would be a dependency for _any_ package when it's not installed on the
host, even when no cmake package is selected.
Cmake versions older than 3.0 are affected by the bug described and
fixed in Buildroot in ef2c1970e4 ("cmake: add patch to fix Qt mkspecs
detection"). The bug was fixed in upstream CMake in version 3.0 [0].
Amongst all the cmake packages currently in Buildroot, the currently
highest version mentioned in cmake_minimum_required() is 3.1 (grantlee
and opencv3).
Thus we use 3.1 as the lowest required cmake for now, until a package is
bumped, or a new package added, with a higher required version.
[0] https://cmake.org/gitweb?p=cmake.git;h=e8b8b37ef6fef094940d3384df5a1d421b9fa568
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Davide Viti <zinosat@tiscali.it>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
[yann.morin.1998@free.fr:
- simplify logic in check-host-cmake.mk;
- set and use BR2_CMAKE_HOST_DEPENDENCY, drop USE_SYSTEM_CMAKE;
- bump to cmake 3.1 for grantlee and opencv;
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
pkg-utils.mk contains various definitions that are used in the package
infrastructures and packages themselves.
However, those definitions can be useful in other parts of Buildroot,
and are already used in a few places that are not related to the package
infrastructure. Also, $(sep) will be needed early in the Makefile when
we eventually support multiple br2-external trees.
Since this file only contains definitions, we can include it anytime.
So, consider that file to no longer be specific to the package infras:
- move it to support and rename it,
- move a few similar definitions from the main Makefile to that file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This adds support to pass options to the underlying command that is used
by downloader. Useful for retrieving data with server-side checking for
user login or passwords, use a proxy or use specific options for cloning
a repository via git and hg.
Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change enforces the CMAKE_SYSROOT value set in the toolchainfile.cmake.
This fix overrides the CMake heuristics used to guess it, and turns off some
non-desirable behavior adding "-isystem ..." flags to the compiler command
line, misleading the compiler and making the build failed due to some
unfound standard headers.
Fixes:
http://autobuild.buildroot.net/results/f7e/f7e92678e91a6cb15ccf32d4a7d75b39f49d6000/defconfig
(and others)
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The size-stats script fails when the usb_modeswitch_data is enabled,
because this package installs files that contain commas in their
name. However, the size-stats script also uses comma as a separator for
its CSV files, causing a "ValueError: too many values to unpack" in:
pkg, fpath = l.split(",")
Fix this by splitting only the two fields that need to be split.
The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested
a fix.
Fixes bug #9136.
Reported-by: Matthias <porto.rio@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The "--no-patch" option used by the git downloader appeared on git
1.8.4. Systems with older git versions show an error and fall back to
the wget downloader, which isn't suitable for all the cases.
Signed-off-by: Enrique Ocaña González <eocanha@igalia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since efl update to 1.15 version, the efl package is a "real"
Buildroot package. It doesn't contain any subdirectories anymore.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.
Use "set -e" to exit immediately if a command return an error.
Be sure to ignore any expected error: when we check if a patch to be
applied has the same basename as an already applied patch, the grep
would fail when no such patch was already applied. We should not fail
in this case.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.
Use "set -e" to exit immediately if a command return an error.
[1] http://patchwork.ozlabs.org/patch/626196
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[Thomas:
- add comment in scancpan about the version dependency, suggested by
Yann E. Morin.
- add comment in perl.mk about the need to sync any version change with
scancpan, also suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
sha256 checksum will be computed locally either by scanpypi at package
creation or by hand by package updates. Define this checksum as
'computed locally' so that one doesn't need to change this comment by
package updates. Also put comments for both md5 and sha256 in one line.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Make sure a help text is terminated with a full stop.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since the fortran support is conditional, only enable it when needed.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Benjamin Kamath <bkamath@spaceflight.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a new package variable that packages can set to specify that they
need git submodules.
Only accept this option if the download method is git, as we can not get
submodules via an http download (via wget).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Aleksandar Simeonov <aleksandar@barix.com>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some git repositories may be split into a master repository and
submodules. Up until now, we did not have support for submodules,
because we were using bare clones, in which it is not possible to
update the list of submodules.
Now that we are using plain clones with a working copy, we can retrieve
the submdoules.
Add an option to the git download helper to kick the update of
submodules, so that they are only fetched for those packages that
require them. Also document the existing -q option at the same time.
Submodules have a .git file at their root, which contains the path to
the real .git directory of the master repository. Since we remove it,
there is no point in keeping those .git files either.
Note: this is currently unused, but will be enabled with the follow-up
patch that adds the necessary parts in the pkg-generic and pkg-download
infrastructures.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We currently use git-archive to generate the tarball. This is all handy
and dandy, but git-archive does not support submodules. In the follow-up
patch, we're going to handle submodules, so we would not be able to use
git-archive.
Instead, we manually generate the archive:
- extract the tree to the requested cset,
- get the date of the commit to store in the archive,
- store only numeric owners,
- store owner and group as 0 (zero, although any arbitrary value would
have been fine, as long as it's a constant),
- sort the files to store in the archive.
We also get rid of the .git directory, because there is no reason to
keep it in the context of Buildroot. Some people would love to keep it
so as to speed up later downloads when updating a package, but that is
not really doable. For example:
- use current Buildroot
- it would need foo-12345, so do a clone and keep the .git in the
generated tarball
- update Buildroot
- it would need foo-98765
For that second clone, how could we know we would have to first extract
foo-12345 ? So, the .git in the archive is pretty much useless for
Buildroot.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we are using bare clones, so as to minimise the disk usage,
most notably for largeish repositories such as the one for the Linux
kernel, which can go beyond the 1GiB barrier.
However, this precludes updating (and thus using) the submodules, if
any, of the repositories, as a working copy is required to use
submodules (becaue we need to know the list of submodules, where to find
them, where to clone them, what cset to checkout, and all those is
dependent upon the checked out cset of the father repository).
Switch to using /plain/ clones with a working copy.
This means that the extra refs used by some forges (like pull-requests
for Github, or changes for gerrit...) are no longer fetched as part of
the clone, because git does not offer to do a mirror clone when there is
a working copy.
Instead, we have to fetch those special refs by hand. Since there is no
easy solution to know whether the cset the user asked for is such a
special ref or not, we just try to always fetch the cset requested by
the user; if this fails, we assume that this is not a special ref (most
probably, it is a sha1) and we defer the check to the archive creation,
which would fail if the requested cset is missing anyway.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In most cases Python's package dependencies found in setup.py are
runtime dependencies and hence don't need to be mentioned in *.mk
file.
Also add '# runtime' tag to select statements in Config.in.
__create_mk_requirements() itself is left for future uses (cffi backend
handling etc.).
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.
We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.
However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.
Update the legal-info header accordingly.
Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Patches we save can come from various locations:
- bundled with Buildroot
- downloaded
- from one or more global-patch-dir
It is possible that two patches lying into different locations have the
same basename, like so (first is bundled, second is from an hypothetical
global-patch-dir):
package/foo/0001-fix-Makefile.patch
/path/to/my/patches/foo/0001-fix-Makefile.patch
In that case, when running legal-info, we'd save only the second patch,
overwriting the first. That would be problematic, because:
- either the second patch depends on the first, and thus would no longer
apply (this is easy to detect, though),
- or the second patch does not depend on the first, and the compliance
delivery will not be complete (this is much harder to detect).
We fix that by checking that no two patches have the same same basename.
If we find that the basename of the patch to be applied collides with
that of a previously applied patch, we error out and report the duplicate.
The unfortunate side-effect is that existing setups will now break in
that situation, but that's a minor, corner-case issue that is easily
fixed.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: adjust coding style, fix minor typos in the commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we only store the filename of the applied patches.
However, we are soon to want to install those patches in the legal-info
directory, so we'll have to know where those patches come from.
Instead of duplicating the logic to find the patches (bundled,
downloaded, from a global patch dir...), just store the full path to
each of those patches so we can retrieve them more easily later on.
Also always create the list-file, even if empty, so that we need not
test for its existence before reading it.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
[Tested only with patches in the Buildroot sources]
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: used $PWD instead of $(pwd), as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A utility for creating python package from the python package index.
It fetches packages info from http://pypi.python.org and generates
corresponding packages files.
Signed-off-by: Denis THULIN <denis.thulin@openwide.fr>
Tested-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: minor tweaks.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.
To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.
The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.
The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.
A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
- rework for python 3.5
- integrate Samuel proposal that allows to detect compilation
failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The graph-build-time help text currently looks like this:
usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER]
[--alternate-colors] [--input OUTPUT] --output OUTPUT
Obviously, naming the parameter for --input as OUTPUT is not a very
good idea, so this commit fixes that to name it "INPUT", as expected.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When preparing the legal-info, the source archives are copied in the
legal-info/ output directory. When the archives are big, it can take
quite a bit of time and unnecessarily uses disk space. When the
legal-info output directory is on the same filesystem as the BR2_DL_DIR,
we can easily reduce copy time and disk usage by just using hardlins
instead of copying. However, the BR2_DL_DIR may be on a different
filesystem, so we must fallback to copying in this case
Introduce a helper script that copies a source file into a destination
directory, by first attempting to hard-link, and falling back to a
plain copy in case the hardlink fails.
In case the destination already exists, it is forcibly removed first, to
avoid clobering any existing target file (and especially any hardlink to
it), since cp -f does not remove the destination file, but clobbers it.
In some situations, it will be necessary that the destination file is
named differently than the source, so if a third argument is specified,
it is treated as the basename of the destination file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
