Commit Graph

786 Commits

Author SHA1 Message Date
Ricardo Martincoski
77b4b948fc support/testing: use pexpect.sendline directly
When using pexpect there is no need for a helper function. Just use
sendline() directly everywhere.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
e664c5f71c support/testing: let pexpect write stdout to log
When the parameter logfile is passed to spawn(), pexpect sends both
stdin and stdout to the logfile and it creates a double echo effect.

One way to avoid the double echo in the logfile would be to disable the
echo on the terminal just after login ("stty -echo"), but double echo of
user and password would remain.

Instead of that, send only the stdout to the logfile using the
logfile_read property.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
8ebc227806 support/testing: use qemu stdio in emulator
Instead of redirecting qemu serial to telnet, redirect it to stdio.

It allows to run testcases in parallel without random failing caused by
two emulators trying to use the same telnet port (1234).

'qemu -serial stdio' returns some extra <CR> characters, so remove them
from the log.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
eb0fab80f6 support/testing: use pexpect in emulator
Replace subprocess + telnetlib with pexpect.

Use the telnet installed on the host machine instead of telnetlib, while
the serial from qemu is not yet redirected to stdio.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Carlos Santos
5fc47a9f6b core: add "cmake3" to the list of cmake candidates
This is useful on CentOS 7, whose "cmake" utility corresponds to version
2.8.12, which is too old for Buildroot.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 21:26:20 +02:00
Carlos Santos
914f75f156 core: allow having a list of "cmake" candidates
Add the BR2_CMAKE_CANDIDATES variable, containing a list of candidates
to check and use as BR2_CMAKE, if possible.

This allows using "cmake3" on CentOS 7, whose default cmake corresponds
to version 2.8.12. Example:

    $ make BR2_CMAKE_CANDIDATES="cmake cmake3"

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 21:26:15 +02:00
Carlos Santos
cacc6d0b61 core: allow check-host-cmake.sh to try several candidates
This is useful on CentOS 7 whose "cmake" package provides cmake 2.8.12,
which is too old, but the "cmake3" package (from EPEL) provides version
3.6.3, which is satisfactory. Examples:

    $ sh support/dependencies/check-host-cmake.sh 2.8 cmake cmake3
    /usr/bin/cmake

    $ sh support/dependencies/check-host-cmake.sh 3.1 cmake cmake3
    /usr/bin/cmake3

    $ sh support/dependencies/check-host-cmake.sh 3.8 cmake cmake3
    (nothing)

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 21:25:45 +02:00
Carlos Santos
77a7a15e4c core: reverse the argument order in check-host-cmake
Pass the minimal version before the program name. In a later change the
script will become able to test a list of candidates.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 21:23:53 +02:00
Jens Maus
82f64203a8 support/scripts: allow /etc/shadow to be symlinked
This commit fixes a problem where it was not possible to replace
/etc/shadow with a symlink to a e.g. a user partition where the
shadow file is placed. This is required, e.g. for systems where the
rootfs is mounted read-only but users should still be able to be
added. Thus, if within an filesystem overlay setup a user tries
to replace /etc/shadow with a symlink to the real file on a user
partition a buildroot build stops with an error message because
sed is called on the symlink instead of following the symlink.
This commit fixes this shortcoming.

Signed-off-by: Jens Maus <mail@jens-maus.de>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 17:49:48 +02:00
Bernd Kuhls
9eebeb4482 scanpypi: fix comment typo
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-06-17 13:15:05 +02:00
Francois Perrad
c3839e048e scancpan: refactor with $mirror
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:32:01 +02:00
Francois Perrad
e27c0b5acb scancpan: refactor with MetaCPAN::API source()
instead of get_manifest()

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:31:50 +02:00
Francois Perrad
8710996bb7 scancpan: update with MetaCPAN API v1
The API v0 is shutdown.
see https://bugs.busybox.net/show_bug.cgi?id=9951

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:31:44 +02:00
Rahul Bedarkar
52c724d3d4 support/misc/target-dir-warning.txt: fix typo
s/owernship/ownership/

Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-06-14 21:14:59 +02:00
Francois Perrad
9cbe364f57 support/scripts: fix shebang with bash
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-08 21:43:05 +02:00
Yann E. MORIN
f00d6ec67b support/dependencies: check PATH does not carry a \n
... otherwise it fails spectacularly as soon as PATH is referenced in a
package rule (i.e. very soon, fortunately):

    >>> host-lzip 1.18 Downloading
    /bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
    /bin/bash: -c: line 1: syntax error: unexpected end of file

Fixes # 9886.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ciro Santilli <ciro.santilli@gmail.com>
[Thomas: fix typo in message, use tabs for indentation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-08 21:40:32 +02:00
Francois Perrad
ed0ce49537 support/scripts: fix shebang with python
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-06-07 21:55:10 +02:00
Peter Korsgaard
dd2020aadf Update for 2017.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-05-31 23:55:40 +02:00
Luca Ceresoli
ad79bb5085 support/testing: rename variables for readability
The 'lines' variable is overwritten with its own fields. Thus it
contains a line first, and then a list of fields -- it never contains
'lines'.

Use two different variables named 'line' and 'fields' to make the code
more readable.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:54:43 +02:00
Luca Ceresoli
cfecdfadc1 support/testing: fs/ext: add a negative test for the "extent" feature
We only have a positive test for it, in ext4. Let's have a negative
one as well.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:48:08 +02:00
Luca Ceresoli
d332f2c521 support/testing: simplify logging by keeping the log file open
We currently call infra.smart_open() to open log files each time we
need to write to them.

Opening the file once in the constructor of Builder and Emulator and
writing to it whenever needed is simpler and slightly more efficient.

Remove smart_open and instead create a new open_log_file() function
which just opens the logfile. Also let it compute the filename, in
order to simplify even further the Builder and Emulator code.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:59:58 +02:00
Luca Ceresoli
fa3c5cad44 support/testing/run-tests: help: put the one-letter form before the long form
This is what the manpages usually do, and what Python does with the
automatically-added -h/--help parameter:

Before the change:
  $ ./support/testing/run-tests
    [...]
  optional arguments:
    -h, --help            show this help message and exit
    --list, -l            list of available test cases
    --all, -a             execute all test cases

After the change:
  $ ./support/testing/run-tests
    [...]
  optional arguments:
    -h, --help            show this help message and exit
    -l, --list            list of available test cases
    -a, --all             execute all test cases

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:49 +02:00
Luca Ceresoli
ee72dbe576 support/testing: document get_elf_prog_interpreter
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:38 +02:00
Luca Ceresoli
52258901ec support/testing: rename check_broken_links to has_broken_links
has_broken_links makes it self-explanatory that this is a predicate
function, and that the return value tells whether there _are_ broken
links, not the opposite.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:32 +02:00
Luca Ceresoli
bab4a2fd37 support/testing: remove unused variable
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-10 23:42:27 +02:00
Luca Ceresoli
08151c6aec support/testing: show messages in a more readable format
Currently messages from run-tests are quite unpleasant:

[br-tests/TestPostScripts/2017-05-09 15:51:57] Building
[br-tests/TestPostScripts/2017-05-09 15:52:23] Building done
[br-tests/TestPostScripts/2017-05-09 15:52:23] Cleaning up
.[br-tests/TestNoTimezone/2017-05-09 15:52:23] Starting
[br-tests/TestNoTimezone/2017-05-09 15:52:23] Building
[br-tests/TestNoTimezone/2017-05-09 15:53:17] Building done
[br-tests/TestNoTimezone/2017-05-09 15:53:22] Cleaning up
.[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Starting
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Building
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:33] Building done
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:37] Cleaning up
[...]

Change them in a more readable way by removing the date and using a
columnar style:

15:12:22 TestPostScripts                          Starting
15:12:25 TestPostScripts                          Building
15:12:48 TestPostScripts                          Building done
15:12:48 TestPostScripts                          Cleaning up
.15:12:48 TestNoTimezone                           Starting
15:12:54 TestNoTimezone                           Building
15:13:44 TestNoTimezone                           Building done
15:13:49 TestNoTimezone                           Cleaning up
.15:13:49 TestGlibcNonDefaultLimitedTimezone       Starting
15:14:00 TestGlibcNonDefaultLimitedTimezone       Building
15:14:56 TestGlibcNonDefaultLimitedTimezone       Building done
15:15:01 TestGlibcNonDefaultLimitedTimezone       Cleaning up
[...]

Note the '.' and other characters presented by nose2 are still
printed. They are not affected by this change.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-10 23:42:13 +02:00
Thomas Petazzoni
fb71372d5f support/testing: add toolchain tests
This commit adds an initial toolchain test case, testing the ARM
CodeSourcery toolchain, just checking that the proper sysroot is used,
and that a minimal Linux system boots fine under Qemu.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
8db88df8c9 support/testing: add package tests
This commit adds some basic tests for two Buildroot packages: python and
dropbear. These tests are by no mean meant to be exhaustive, but mainly
to serve as initial examples for other tests.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
bf4a6490e4 support/testing: add fs tests
This commit adds a number of test cases for various filesystem formats:
ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them
except yaffs2 are runtime tested. The iso9660 set of test cases is
particularly rich, testing the proper operation of the iso9660 support
with all of grub, grub2 and isolinux.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
96e21b617d support/testing: add core tests
This commit adds a few Buildroot "core" tests, testing functionalities
such as:

 - post-build and post-image scripts
 - root filesystem overlays
 - timezone support

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
a732fb222b support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.

This core infrastructure commit includes the following aspects:

 - A base test class, called BRTest, defined in
   support/testing/infra/basetest.py. This base test class inherited
   from the Python provided unittest.TestCase, and must be subclassed by
   all Buildroot test cases.

   Its main purpose is to provide the Python unittest setUp() and
   tearDown() methods. In our case, setUp() takes care of building the
   Buildroot system described in the test case, and instantiate the
   Emulator object in case runtime testing is needed. The tearDown()
   method simply cleans things up (stop the emulator, remove the output
   directory).

 - A Builder class, defined in support/testing/infra/builder.py, simply
   responsible for building the Buildroot system in each test case.

 - An Emulator class, defined in support/testing/infra/emulator.py,
   responsible for running the generated system under Qemu, allowing
   each test case to run arbitrary commands inside the emulated system.

 - A run-tests script, which is the entry point to start the tests.

Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Abhimanyu Vishwakarma
e73cf8a228 genimage.sh: fix calling from BR2_ROOTFS_POST_IMAGE_SCRIPT
When called from BR2_ROOTFS_POST_IMAGE_SCRIPT, this script
ends up with following error:

Error: Missing argument

This is because, an extra positional argument is also passed
along with BR2_ROOTFS_POST_SCRIPT_ARGS. genimage.sh didn't
have support to parse positional and optional arguments
together.

Signed-off-by: Abhimanyu Vishwakarma <Abhimanyu.V@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-04-27 22:55:52 +02:00
Ricardo Martincoski
7b394c4926 check-package: move parts to subdirectory
Currently the check-package script uses many files in the same
directory. This commit keeps the main script in support/scripts/ and
moves the rest into a subdirectory.

The modules were previously prefixed to make it easy to identify which
script they belong to. This is no longer needed when using a
subdirectory, so the prefix is removed.

Note: if this commit is checked out and the script is run, and later on
a previous version is checked out, the file
support/scripts/checkpackagelib/__init__.pyc needs to be manually
removed to prevent Python interpreter to look for checkpackagelib
package when only the checkpackagelib module is available.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-20 22:29:41 +02:00
Ricardo Martincoski
5d6ec40b75 download/git: clarify why .git is removed
The removal of the .git dir before creating the tarball is not anymore
just an optimization. It is necessary to make the tarball reproducible.
Also, without the removal, large tarballs (gigabytes) would be created
for some linux trees.

Update the comment accordingly.

Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-20 22:27:23 +02:00
Yann E. MORIN
37308b979a support/test-pkg: run legal-info
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-17 15:54:33 +02:00
Arnout Vandecappelle
92b10f63c5 support/test-pkg: calculate toolchain name only once
The toolchain name was calculated in main() for reporting to the user,
and again in build_one() for creating the build directory. Calculate
it only once, in main(), and pass the build directory as an argument
to build_one().

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-17 15:54:22 +02:00
Thomas Petazzoni
c0274ae8b5 pkg-stats: also parse toolchain packages
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-09 09:57:44 +02:00
Thomas Petazzoni
33b78b2ecf pkg-stats: ignore additional package infra .mk files
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-09 09:45:10 +02:00
Thomas Petazzoni
b1e102d1ab pkg-stats: add new column with check-package warnings
This allows the page at http://autobuild.buildroot.net/stats/ to show
how many warnings returned by check-package affect each package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 21:35:33 +02:00
Ricardo Martincoski
40ef732490 check-package: check *.mk for typo in variable
Warn when a variable is defined in a .mk file and it don't start with
the package name.

This function generates false warnings and the maintenance of the
whitelist can be an extra burden, but it catches some typos really hard
to see:
- POPLER_CONF_OPTS [1]
- BALELD_LICENSE [2]
- DRDB_UTILS_DEPENDENCIES [3]
- PERL_LIBWWW_LICENSE_FILES [4]
- AVRDUDR_LICENSE_FILES [5]
- GST1_PLUGINS_ULGY_HAS_GPL_LICENSE [6]
- ON2_8170_LICENSE [7]
- LIBFDTI_CONF_OPTS [8][9]
- IPSEC_DEPENDENCIES [10]

[1] http://patchwork.ozlabs.org/patch/681533
[2] http://patchwork.ozlabs.org/patch/643293
[3] http://patchwork.ozlabs.org/patch/449589
[4] http://patchwork.ozlabs.org/patch/464545
[5] http://patchwork.ozlabs.org/patch/305060
[6] http://patchwork.ozlabs.org/patch/253089
[7] http://patchwork.ozlabs.org/patch/250523
[8] http://patchwork.ozlabs.org/patch/394125
[9] fe7a4b524b
[10] dff1d590b2

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:39 +02:00
Ricardo Martincoski
203b33fab2 check-package: check Config.* files
Warn when help text is larger than 72 columns, see [1].
Warn for wrongly indented attributes, see [1].
Warn when the convention of attributes order is not followed, see [2].

[1] http://nightly.buildroot.org/#writing-rules-config-in
[2] http://nightly.buildroot.org/#_config_files

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:38 +02:00
Ricardo Martincoski
8e87d76c12 check-package: check *.mk files
Warn when there are obvious indentation errors:
- the number of expect tabs is not yet checked since it is more complex
  to achieve;
- the content inside define ... endef should be indented with tab(s),
  see [1];
- line just after a backslash should be indented with tab(s), see [2];
- other lines should not be indented, see [3];
- ignore empty lines and comments.
Warn when there is no well-formatted header in the file:
- 80 hashes at lines 1 and 5;
- 1 hash at lines 2 and 4;
- empty line at line 6;
- see [4];
- ignore files that only include other mk files.
Warn when there are more than one space before backslash, see [5].
Warn when there is a trailing backslash [6].
Warn for flags set to default value YES or NO, see [7], [8], [9].

[1] http://patchwork.ozlabs.org/patch/681429/
[2] http://patchwork.ozlabs.org/patch/681430/
[3] http://patchwork.ozlabs.org/patch/559209/
[4] http://nightly.buildroot.org/#writing-rules-mk
[5] http://patchwork.ozlabs.org/patch/649084/
[6] http://patchwork.ozlabs.org/patch/535550/
[7] http://patchwork.ozlabs.org/patch/704718/
[8] http://nightly.buildroot.org/#_infrastructure_for_packages_with_specific_build_systems
[9] http://nightly.buildroot.org/#_infrastructure_for_autotools_based_packages

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:35 +02:00
Ricardo Martincoski
23b0b4c028 check-package: check *.patch files
Warn when the name of the patch file does not start with number (apply
order), see [1].
Warn when the patch was generated using git format-patch without -N, see
[2].
Warn when the patch file has no SoB, see [3].

[1] http://nightly.buildroot.org/#_providing_patches
[2] http://patchwork.ozlabs.org/patch/704753/
[3] http://nightly.buildroot.org/#_format_and_licensing_of_the_package_patches

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:34 +02:00
Ricardo Martincoski
6c931bbf5f check-package: check *.hash files
Check each hash entry (see [1]) and warn when:
- it does not have three fields;
- its type is unknown;
- its length does not match its type;
- the name of the file contains a directory component.

[1] http://nightly.buildroot.org/#adding-packages-hash

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:33 +02:00
Ricardo Martincoski
c6bf3239cb check-package: check whitespace and empty lines
Create 3 new check functions to warn when:
- there are consecutive empty lines in the file, see [1];
- the last line of the file is empty, see [2];
- there are lines with trailing whitespace, see [3].

Apply these functions to Config.*, *.mk and *.hash, but not for *.patch
files since they can contain any of these and still be valid.

[1] http://patchwork.ozlabs.org/patch/682660/
[2] http://patchwork.ozlabs.org/patch/643288/
[3] http://patchwork.ozlabs.org/patch/398984/

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:32 +02:00
Ricardo Martincoski
111132903d support/scripts/check-package: new script
Create the infra to check the style of new packages before submitting.
The overall function of the script is described inside a txt file.
It is designed to process the actual files and NOT the patch files
generated by git format-patch.

Also add the first check function, to warn if a file (Config.*, *.mk,
*.hash, *.patch) has no newline at the last line of the file, see [1].

Basic usage for simple packages:
support/scripts/check-package -vvv package/newpackage/*

Basic usage for packages with subdirs:
support/scripts/check-package -vvv $(find package/newpackage/ -type f)

See "checkpackage" in [2].

[1] http://patchwork.ozlabs.org/patch/631129/
[2] http://elinux.org/Buildroot#Todo_list

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:31 +02:00
Yann E. MORIN
762cd7cdcb support/test-pkg: print number of toolchains and progress
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-06 21:51:29 +02:00
Phelip Etienne
c9062dcdf6 support/scripts: add generic genimage script
This script is a wrapper for the genimage tool used by most boards.
The board postimage script can now call this script instead of invoking
genimage command themselves.

Signed-off-by: Etienne Phelip <etienne.phelip@savoirfairelinux.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:50:22 +02:00
Rahul Bedarkar
f8ff7ab0be scripts/scanpypi: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:43:36 +02:00
Rahul Bedarkar
e261d807f3 scripts/scancpan: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:35:14 +02:00
Yann E. MORIN
c5fa9308ea core/br2-external: properly report unexpected errors
Unextected error in the br2-external script are properly caught, but
they are not reported properly, and we end up in either of two
situations:

  - the .br2-external.mk file is not generated, in which case make will
    try to find a rule to generate it (because the 'include' directive
    tries to generate missing files);

  - the .br-external.mk file is generated but does not contain the error
    variable, and thus the build might not get interrupted.

We fix that by using a trap on the pseudo ERR signal, to emit the error
variable on unexpected errors.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-26 23:44:04 +02:00
Thomas Petazzoni
d5d642a43f support/scripts/check-bin-arch: fix after "fix for filenames with spaces"
In commit 2f6c5e513c
("support/check-bin-arch: fix for filenames with spaces"), Yann
adjuste the check-bin-arch script to properly handle filenames with
spaces.

However, he also did a subtle change of the regexp that extracts the
path of the files. It was:

  "/^${package},(.+)$/!d; s//\1/;"

and Yann changed it to:

  "/^${package},\.(.+)$/!d; s//\1/;"

So the file paths used to start with a dot (like "./usr/share/foo"),
and now they no longer start with a dot (like "/usr/share/foo"). While
this modification is good and makes sense, the match for
/lib/firmware/ was not adjusted accordingly, and the follow-up patch
also ignoring /usr/share was not adjusted as well.

This commit fixes those /lib/firmware/ and /usr/share/ special cases,
which will fix:

  http://autobuild.buildroot.net/results/76a1475f4cdedb80426fb022ef2e644aa5625660/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-22 21:43:54 +01:00
Arnout Vandecappelle
0f369a9231 download/git: create GNU format tar files
On most distros, the tar format defaults to GNU. However, at build time
the default format may be changed to posix. Also, future versions of
tar will default to posix.

Since we want the tarballs created by the git download method to be
reproducible (so their hash can be checked), we should explicitly
specify the format. Since existing tarballs on sources.buildroot.org
use the GNU format, and also the existing hashes in the *.hash files
are based on GNU format tarballs, we use the GNU format.

In addition, the Posix format encodes atime and ctime as well as mtime,
but tar offers no option like --mtime to override them. In the GNU
format, atime and ctime are only encoded if the --incremental option is
given.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-21 23:15:46 +01:00
Yegor Yefremov
3b627c89dc graph-depends: rename pkgutil.py to brpkgutil.py
pkgutil.py is also part of Python itself. Placing pkgutil.py as is
in a folder with other scripts that require original pkgutil will
break them. This is the case with scanpypi. So rename pkgutil.py
to brpkgutil.py to avoid naming collision.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=9766

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-21 23:11:04 +01:00
Thomas Petazzoni
ba72d71d79 support/script/check-bin-arch: ignore /usr/share
/usr/share normally should not contain binaries executable for the
target platform. However, it might contain ELF binaries for other
platforms, such as firmware files installed by Qemu or
pru-software-support.

Instead of special-casing each package, let's simply ignore /usr/share.

Fixes:

  http://autobuild.buildroot.net/results/6f3fea9f6adaef1573fbb0dd6903b5d99e470610/
  (pru-software-support)

  http://autobuild.buildroot.net/results/fe8892bc22a03299fc41e30bfea5e42166838f88/
  (qemu)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:50 +01:00
Yann E. MORIN
2f6c5e513c support/check-bin-arch: fix for filenames with spaces
Filenames with spaces will break the current for loop.

Fix that by using a while-read loop, fed with the list of files on
stdin, using process substitution.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:17 +01:00
Jérôme Pouiller
6e45c5500b pycompile: allow to force compilation
By default, compile_dir() relies on the modification time to know if a
python file has to be built again. However in some circumstances (when
doing reproducible builds), modification times are not reliable. Thus,
this patch adds a way to force the rebuild of all python sources.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 23:00:25 +01:00
Thomas Petazzoni
bbb7f6c16c Makefile: add check of binaries architecture
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.

This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.

Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.

Example output with the firejail package enabled, when building for an
ARM target:

ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM

Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 22:22:22 +01:00
Andrey Smirnov
61fcd08247 scripts/pycompile: Accomodate latest Python 3 codebase
As of the version 3.6.0 compile_dir() call will treat its 'quiet'
argument as a full blown integer rather than a boolean value and perform
integer comparison operations such as '<' or '>='.

To account for that convert ReportProblem type to be a true derivative
of built-in int() and override all of int's rich comparison operators in
order to be able to "sniff" for PyCompileError in all possible use-cases

The integer value ReportProblem pretends to be is teremined by class
variable VALUE which is set to 1.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-19 15:04:32 +01:00
Benoît Allard
ac53042bf2 support/scripts/test-pkg: extend logfile with olddefconfig output
Signed-off-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-09 21:34:33 +01:00
Yann E. MORIN
894ad2b094 core/pkg-cmake: better way to pass our CMAKE_MODULE_PATH
Currently, we tell cmake where to look for our own custom platform
description by passing the path to the moduls directory on the command
line.

However, this causes two different problems.

First, some packages simply set CMAKE_MODULE_PATH in their
CMakeList.txt, thus overriding our own path, and then our platform
description is not found.

Second, cmake may internally call sub-cmake (e.g. in the try_compile
macro), but the CMAKE_MODULE_PATH is not automatically passed down in
this case.

For the first problem, we could hunt down and fix all offenders, but
this is an endless endeavour, especially since packagers are told to do
so on the cmake wiki [0]:

    CMAKE_MODULE_PATH
        tell CMake to search first in directories listed in
        CMAKE_MODULE_PATH when you use FIND_PACKAGE() or INCLUDE()
        SET(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/MyCMakeScripts)
        FIND_PACKAGE(HelloWorld)

The second problem could be solved by passing yet another variable on
the command line, that tells cmake to explicitly pass arbitrary
variables down to sub-cmake calls:

    -DCMAKE_TRY_COMPILE_PLATFORM_VARIABLES=CMAKE_MODULE_PATH

However, this only covers the case of try_compile. Even though no other
case is known yet, we'd still risk missing locations where we would need
to propagate CMAKE_MODULE_PATH, even some where we'd have no solution
like for try_compile.

Instead, ngladitz on IRC suggested that CMAKE_MODULE_PATH be set
directly from the toolchain file.

And indeed this fixes both problems explained above.

So be it.

[0] https://cmake.org/Wiki/CMake_Useful_Variables

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-05 21:34:49 +01:00
Yann E. MORIN
00d34e8a6f support/mkusers: rename variable to avoid name clashing
The LINES variable is automatically set by bash to represent the number
of lines in the terminal. That variable can be set when the shell
receives SIGWINCH.

If the shell does receive SIGWINCH after our LINES array is filled, the
content of the array is mangled.

Rename the variable to avoid that.

Fixes #9456

Reported-by: George Y. <georgebrmz@oss3d.com>
Reported-by: Paul Stewart <paulstewartis@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-04 11:56:51 +01:00
Yann E. MORIN
56aaef5533 support/cmake: impersonate a Linux system even more
Some packages test the system name to decide whether to enable/disable
features or link with specific libs.

So we forcefully set the system name form our custom system file, so
that packagses still believe they are running on Linux rather than
Buildroot.

Fixes:
    fastd      : http://autobuild.buildroot.net/results/f1d/f1dfe90068ad62e733f17a22202235415bda3974/
    paho-mqtt-c: http://autobuild.buildroot.net/results/457/457d76279e16247bf58c838a2c5dd0a4f3962c21/
    libiio     : http://autobuild.buildroot.net/results/281/2812b008a0ab6bab5fe4d45eb9ffe4e9496a8cb4/
    and so on...

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-02 22:09:34 +01:00
Yann E. MORIN
6593a0b987 Revert "dependencies/cmake: blacklist cmake 3.7"
This reverts commit 4422eca2d4.

We now have a workaround for the RPATH issue introduced in 3.7, so we
can use it again.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-01 22:13:49 +01:00
Yann E. MORIN
c69b14fe2f core/pkg-cmake: provide our own platform description
The handling of RPATH in cmake-3.7 has changed drastically, causing a
slew of build failures dues to libraries from the host being pulled in:

  - domoticz : http://autobuild.buildroot.org/results/fd0/fd0ba54c7abf973691b39a0ca1bb4e07d749593a/
  - freerdp  : http://autobuild.buildroot.org/results/5d4/5d429d0e288754a541ee5d8be515454c5fccd28b/
  - libcec   : http://autobuild.buildroot.org/results/3f3/3f3593bab7734dd274faf5b5690895e9424cbb89/
  - and so on...

The bug was reported upstream [0], which dismissed it altogether [1] as
being expected behaviour, quoting:

    I don't think there is anything wrong with that change on its own.
    It merely exposed some existing behavior in a new case.

Instead, upstream suggested in that same message that a platform
definition be used instead, quoting:

    If a toolchain file specifies CMAKE_SYSTEM_NAME such that a custom
    `Platform/MySystem.cmake` file is loaded then the latter can set
    them as needed for the target platform.

So here we are doing so:

  - we add a new platfom definitions that inherits from the Linux one,
    then overrides the problematic settings;

  - we change our toolchain file to use that platform instead;

  - we tell cmake where to find additional modules, so that it can find
    our custom platform file.

This has been tested to work in the following conditions:

 - pre-installed host cmake, versions 3.5.1 (Ubuntu 16.04) and 3.7.2
   (manually built)

  - internal cmake, versions 3.6.3 (the current version as of this
    patch) and 3.7.2 (with the followup patches).

Thanks to Jörg, Ben and Baruch for the help investigating the issue.
Special thanks to Jörg for handling the discussion with upstream and
pointing to the relevant messages! :-)

[0] http://public.kitware.com/pipermail/cmake/2017-February/064970.html
[1] http://public.kitware.com/pipermail/cmake/2017-February/065063.html

To be noted: Thomas suggested we set these directly in the toolchain
file. Unfortunately, wherever we put those settings in the toolchain
file, this does not work.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-01 21:56:24 +01:00
Peter Korsgaard
083c0735e9 Update for 2017.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-02-28 22:00:23 +01:00
Yann E. MORIN
4422eca2d4 dependencies/cmake: blacklist cmake 3.7
cmake-3.7 has a bug in how it handles rpath, linking with libraries from
the host.

Until we completely understand the issue, just blacklist cmake-3.7.

The issue has been reported upstream:
    http://public.kitware.com/pipermail/cmake/2017-February/064970.html

Reported-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-02-28 09:40:27 +01:00
Angelo Compagnucci
410082e1d9 support/misc/Vagrantfile: reorganize provisioning
* Remove distribution upgrade cause it slows down the first boot and
  presents a bug when executed non interactively.

* Reorganize provision scripts to be in privileged and non privileged
  sections

* Add Ubuntu mirror automatic handling for apt packages sources

Fixes bug #9581

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-26 21:15:13 +01:00
Baruch Siach
f165032e4f package: add generic support for lz archives
This commit teaches the generic package handling code how to extract .tar.lz
archives. When lzip is not installed on the host, host-lzip gets built
automatically.

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-15 22:11:11 +01:00
Baruch Siach
dc7fcbe494 package: refactor listing of extractor dependencies
Don't special case $(XZCAT) when constructing DL_TOOLS_DEPENDENCIES. The next
commit will introduce another extractor that automatically builds when not
installed. Introduce EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS that lists
archive extensions for which the extractor is already checked in
support/dependencies/check-host-foo.mk. Use this in the newly introduced
extractor-dependency to populate DL_TOOLS_DEPENDENCIES.

Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Thomas: add missing space after "firstword", as noticed by Thomas DS.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-15 22:11:11 +01:00
Yann E. MORIN
5bf12ad141 support/test-pkg: report number and types of failures
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
c3c69c4dbb support/test-pkg: be less verbose
Requested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
aeabb58dc5 support/test-pkg: simplify the config check
Use comm(1) to check that all our config options are properly set in the
resulting configuration, rather than our canned and fragile code.

Reported-by: Cam Hutchison <camh@xdna.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
55bbbe04c3 support/test-pkg: check the config snippet exists
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:49 +01:00
Yann E. MORIN
67a221be1b support/test-pkg: fix code style
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:46 +01:00
Yann E. MORIN
9e7885d5f7 support/test-pkg: add option to limit the number of tests
Sometimes, it interesting to have a global overview of whether the
package builds at all or not, rather than test on all toolchains.

Add an option that allows testing on a limited set of randomly choosen
toolchains.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 23:12:36 +01:00
Yann E. MORIN
d5c58ceec2 support/test-pkg: store lines missing from resulting configuraiton
When a build is skipped, store the lines from the config snippet, that
are missing in the resulting configuration, in a file in the build
directory, for the user to inspect.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:57:13 +01:00
Yann E. MORIN
47d5da8bcc support/scripts: add script to test a package
This script helps in testing that a package builds fine on a wide range
of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr:
 - completely rewrite the script from Thomas, with help from Luca
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:42:38 +01:00
Frank Hunleth
2ec0a7d11c size-stats: don't count hard links
This change adds inode tracking to the size-stats script so that hard
links don't cause files to be double counted. This has a significant
effect on the size computation for some packages. For example, git has
around a dozen hard links to a large file. Before this change, git would
weigh in at about 170 MB with the total filesystem size reported as
175 MB. The actual rootfs.ext2 size was around 16 MB. With the change,
the git package registers at 10.5 MB with a total filesystem size of
15.8 MB.

Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 19:38:53 +01:00
Thomas De Schampheleire
412e6c80ad graph-depends: split off get_rdepends from get_depends
The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.

Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:28:00 +01:00
Thomas De Schampheleire
12683184b1 graph-depends: split off get_version/get_depends into pkgutil.py
Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:26:55 +01:00
Thomas De Schampheleire
bd8b531d24 graph-depends: avoid use of global var 'rule' in get_depends
Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.

As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.

Instead, pass the global as an argument.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-05 15:20:02 +01:00
Jérôme Pouiller
99c397fcd3 core: do not reset DEPENDENCIES_HOST_PREREQ in dependencies.mk
Usually, Buildroot does never initialize variables with empty content.
DEPENDENCIES_HOST_PREREQ was an unjustified exception.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-01-28 20:30:41 +13:00
Peter Korsgaard
a3e4b0fb53 Update for 2016.11.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-25 09:55:14 +01:00
Yann E. MORIN
05576fca13 core/br2-external: fix use of relative paths
Fixes #9576

When the path to a br2-external tree is relative, make enters an endless
recursive loop (paths elided for brevity):

    $ make BR2_EXTERNAL=.. foo_defconfig
    make[1]: stat: ../configs/../configs/../configs[...]/toto_defconfig: Filename too long
    make[1]: *** No rule to make target '../configs/../configs/../configs[...]/toto_defconfig',
    needed by '../configs/../configs/../configs[...]/toto_defconfig'.  Stop.
    Makefile:79: recipe for target '_all' failed
    make: *** [_all] Error 2

It is a bit complex to understand the actual technical reason for this
never-ending expansion; it seems it happens in the code generated by the
percent_defconfig macro. Not sure why, though...

But the root cause is the relative path.

Just use absolute, canonical paths to br2-external trees. Always.

[Peter: add bugzilla reference]
Reported-by: outtierbert@gmail.com
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-16 13:55:09 +01:00
Francois Perrad
ec45554cfd perl: bump to version 5.24.0
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-04 17:06:27 +01:00
Peter Korsgaard
67f23a77c2 Update for 2016.11.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-29 22:24:06 +01:00
Yann E. MORIN
e09e1873a1 support/dependencies: ensure we have 'file' on the host
Recently, the autoconf macros for libtool started using '/usr/bin/file'
to determine the type of library that is generated by the toolchain.
Packages that use this recent version of the libtool autoconf macros
will fail in a rather dramatic way when /usr/bin/file is not present
on the host: the package will still build but no shared library is
generated, which in turn may cause build failures in other packages
that link with it.

For example, libpng's configure determines that it is not possible to
build a shared library on MIPS64 because the expected output from 'file'
is not present. Therefore, only a static libpng.a is built. Later,
bandwithd links with -lpng but it doesn't use the pkg-config's
Private-Libs (because it's not linking statically) and it doesn't have
access to the NEEDED reference from the shared library. Therefore, it
doesn't link with zlib and fails with

    pngrutil.c:(.text+0x55c): undefined reference to `inflate'

We cant use host-file because it is itself an autotools package and is
itself using libtool, so this would be a chicken-n-egg problem. Besides,
the libtool script really wants to call /usr/bin/file, so it would not
even find our host-file anyway.

So, just require that '/usr/bin/file' is present on the host.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-12-27 18:02:42 +01:00
Danomi Manchego
1ac9d83f55 mkusers: fix spelling in intro comment
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-20 09:32:06 +01:00
Sam bobroff
41444d7f89 pkg-autotools: generic configure fix for powerpc64
Many (100+) packages supported by buildroot contain old configure
scripts (or build them from old versions of autotools) that are unable
to determine how to link shared libraries on powerpc64 and
powerpc64le. This causes that test to erroneously fail on toolchains
that are not "bi-endian" (which is the case for toolchains built by
buildroot), which causes configure to build static libraries instead
of dynamic ones. Although these builds succeed, they tend to cause
linker failures in binaries later linked against them.

Because affected configure files can be discovered automatically, this
patch introduces a hook (enabled only when building for powerpc64 and
powerpc64le) that uses a script to scan and fix each package.

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-12-05 22:51:17 +01:00
Yann E. MORIN
40685f6a51 support/dependencies: use the new kconfig option to check UTF8 locale
Currently, we require a host with an UTF-8 locale as soone as we build a
Buildroot toolchain with support for locales. This means that we do
erquire such a locale when building a toolchain with either uClibc,
glibc or musl.

However, glibc and musl do not require such a locale to be present.

Use the new option to check if an UTF8 locale is needed on the host,
rather than deriving that from toolchain settings.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-12-04 15:38:16 +01:00
Peter Korsgaard
a7eb052ff8 Update for 2016.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-11-30 23:16:22 +01:00
Yann E. MORIN
b14b02698e core/br2-external: restore compatibility with old distros
Currently, the br2-external script uses bash-4's associative arrays.

However, some oldish enterprise-class distros like RHEL5 still use
bash-3.1 which lacks associative arrays.

We restore compatibility with those oldish distros using 'eval' to
emulate associative arrays, as suggested by Arnout.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 23:31:20 +01:00
Stefan Nickl
0fb2263d29 pkg-cmake: Change -DNEBUG to -DNDEBUG
Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-06 22:30:40 +01:00
Gaël PORTAY
c29f023fc6 get-developers: fix parentheses for print
Python3 complains about missing parentheses.

$ ./support/scripts/get-developers
  File "./support/scripts/get-developers", line 45
    print f
          ^
SyntaxError: Missing parentheses in call to 'print'

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-05 11:49:05 +01:00
David Lechner
7d4a2f2ca4 linux: add ev3dev extension
This adds an ev3dev Linux drivers extension that provides Linux kernel
drivers for LEGO MINDSTORMS EV3 from the ev3dev project.

Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-02 17:52:24 +01:00
Yann E. MORIN
4f87aa7db5 support/download: make the git wrapper more robust
Currently, there are two failure paths in the wrapper:

  - if the tar fails, then the error is ignored because it is on the
    left-hand-side of a pipe;

  - if the find fails, then the error is ignored because it is a
    process substitution (and there is a pipe, too).

While the former could be fixed with "set -o pipefail", the latter can
not be fixed thusly and we must use an intemediate file for it.

So, fix both issues by using intermediate files, both to generate the
list of files to include in the archive, and generate the archive in a
temporary tarball.

Fixes the following build issue, where the find is failing for whatever
unknown reason:
    http://autobuild.buildroot.net/results/20f/20fd76d2256eee81837f7e9bbaefbe79d7645ae9/

And this one, where the process substitution failed, also for an unknown
reason:
    http://autobuild.buildroot.org/results/018/018971ea9227b386fe25d3c264c7e80b843a9f68/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 23:49:35 +02:00
Yann E. MORIN
2a2eb55ca7 core/graph-depends: add option to graph reverse dependencies
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.

It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 22:59:05 +02:00
Rahul Bedarkar
1b0df8f23c scripts/get-developers: correct type of patches argument
Current type for 'patches' argument is str. It supposed to only
contain names of files.

If we specify FileType as type, then we don't need to open file ourself
and it allows script to read patch from standard input as well.

e.g.
$ git show -1 | ./support/scripts/get-developers -

Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 12:43:27 +02:00
Thomas Petazzoni
7d2407b3e7 toolchainfile.cmake: fix description of CMAKE_Fortran_FLAGS_{DEBUG,RELEASE}
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-22 16:44:06 +02:00
Samuel Martin
10a6756fa1 toochainfile.cmake: rework the way Buildroot sets flags
From the build configuration, Buildroot defines and set some compiler
and linker flags that should be passed to any packages build-system.

For package using the cmake-package infrastructure, this is achieved
via the toolchainfile.cmake.

This change simplifies the way the toolchainfile.cmake file handles
these flags: it now just sets them, without any attempt to extend them
with those Buildroot defined.

This change still allows overriding these flags from the configure
command line.

So, now, when a CMake-based package needs to extend them, they should
be fully set from the package *.mk file. This behavior is consistent
with what is done for others package infrastructures.

This change should not pull any regression WRT the bug #7280 [1].

However, now, when someone uses the toolchainfile.cmake file outside of
Buildroot, he/she must overload all compiler/linker flags (including the
ones Buildroot sets since they no longer get automatically added).

[1] https://bugs.busybox.net/show_bug.cgi?id=7280

Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-22 16:23:44 +02:00
Samuel Martin
12494ef48f toolchainfile.cmake: set per-config appended {C, CXX}FLAGS
When a build type is set, CMake does append some flags that can override
those set by Buildroot due to the gcc option parser (in which the last
argument controling an option wins).

Hereafter is a summary of the optimization and debug flags set by
Buildroot and appended by CMake.

* Flags set by Buildroot depending on the configuration:

  BR2_ENABLE_DEBUG | Optim. level        | Buildroot {C,CXX}FLAGS
  =================+=====================+=======================
          y        | BR2_OPTIMIZE_S      | -Os -gx
          y        | BR2_OPTIMIZE_G      | -Og -gx
          y        | BR2_OPTIMIZE_{0..3} | -On -gx
          n        | BR2_OPTIMIZE_S      | -Os
          n        | BR2_OPTIMIZE_G      | -Og
          n        | BR2_OPTIMIZE_{0..3} | -On

* Default flags appended by CMake depending on the build type:

  Build type     | Flags           | Effects on {C,CXX}FLAGS
  ===============+=================+===========================================
  Debug          | -g              | Force -g, compatible with BR2_ENABLE_DEBUG
  MinSizeRel     | -Os -DNDEBUG    | Set -Os, compatible with BR2_OPTIMIZE_S
  Release        | -O3 -DNDEBUG    | Set -O3, closest to the others cases,
                 |                 | though the optimization level is forced.
  RelWithDebInfo | -O2 -g -DNDEBUG | Force -g and set -O2, not friendly with BR

To avoid the CMake flags take precedence over the Buildroot ones, this
change sets in toolchainfile.cmake the per-config compiler flags CMake
can append depending on the build type Buildroot defined.
So, CMake does not mess up with the compilation flags Buildroot sets.

It is still possible to override these per-config flags on the cmake
command line.

Note:
  If a CMake-based project forces the compiler and/or linker flag
  definitions (the default ones or the per-config ones - e.g.
  CMAKE_C_FLAGS/CMAKE_C_FLAGS_{DEBUG,RELEASE}), there is not much
  Buildroot can do about it.
  So, the flags will be overwritten anyway in these cases.

Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
 - adjust comment in toolchainfile.cmake.in, as suggested by Arnout.
 - also handle CMAKE_Fortran_FLAGS_*, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-22 16:20:17 +02:00
Samuel Martin
a471150a7b package/pkg-cmake.mk: move CMAKE_BUILD_TYPE definition into toolchainfile.cmake
The chosen CMAKE_BUILD_TYPE encodes an option of the Buildroot
configuration, so it makes more sense to save it in the
toolchainfile.cmake than to pass it during configure.

It is still possible to override the build type on the cmake
command line.

Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: reword description in the CHANGES file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-22 16:15:32 +02:00
Rahul Bedarkar
a1d12fe939 get-developers: use --cc instead of --to for developers
Output of get-developers script in our manual uses --cc for developers,
but actual output of get-developers script uses --to. This patch makes
code consistent with documentation, by using --cc for developers.

Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:38:16 +02:00
Yann E. MORIN
e30fa61d80 core/br2-external: fix link to manual for converting old trees
To be noted: that link will only be valid once we have a released
manual. In the meantime, it's accessible on the nightly manual:
    http://nightly.buildroot.org/#br2-external-converting

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:20:18 +02:00
Yann E. MORIN
5feff44038 core/br2-external: export paths and descriptions
Now that we support multiple br2-external trees, BR2_EXTERNAL is no
longer exported in the environment.

This means that post-build scripts in a br2-external tree can no longer
find their own files (well, they could re-invent the path by stripping
their known-relative path, but that'd be ugly, especially since we can
very well provide it).

Export the path for each br2-external trees as environment variables.
Do so for the description as well, as a courtesy.

Also, re-order variable definitions to be more logical: first, purely
internal variables, then exported variables.

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:19:14 +02:00
Angelo Compagnucci
6294204686 support/misc/Vagrantfile: bump to ubuntu 16.04
* Updating to ubuntu 16.04
* Fixing dependencies
* Fixing locale complaints
* Removing unused packages

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:00:38 +02:00
Yann E. MORIN
49117c1028 core: support description for br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
339e1c9500 core: allow a br2-external tree to override a defconfig
Currently, it is not possible for a br2-external tree to override a
defconfig bundled in Buildroot, nor is it possible to override one from
a previous br2-external tree in the stack.

However, it is interesting that a latter br2-external tree be able to
override a defconfig:

  - the ones bundled in Buildroot are minimalist, and almost always
    build a toolchain, so a br2-external tree may want to provide a
    "better" defconfig (better, in the sense "suited for the project");

  - similarly for a defconfig from a previous br2-external tree.

But we can't do that, as the rules for the defconfigs are generated in
the order the br2-external trees are specified, all after the bundled
defconfigs. Those rule are patten-matching rules, which means that the
first one to match is used, and the following ones are ignored.

Add a new utility macro, 'reverse', inspired from GMSL, that does what
it says: reverse a list of words.

Use that macro to reverse the list of br2-external trees, so that the
latters win over the formers, and even over bundled ones.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
20cd497387 core: add support for multiple br2-external trees
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.

A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.

Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.

Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.

What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:

  - the list of all br2-external names, BR2_EXTERNAL_NAMES,

  - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
    point each to the actual location of the corresponding tree,

  - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,

  - the space-separated list of absolute paths to the external trees,
    BR2_EXTERNAL_DIRS.

Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.

This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".

Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.

If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
a71e311df6 docs/manual: add appendix to convert old br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
[Peter: slightly reword]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
fc34cf772c core: introduce per br2-external NAME
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.

This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.

Also, display the NAME and its path as a comment in the menuconfig.

This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.

The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.

Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).

Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
f834b0ef21 core/br2-external: validate even more
A br2-external tree must provide external.mk and Config.in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
6bd19ccf0d core: offload handling of BR2_EXTERNAL into the script
Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have one
(BR2_EXTERNAL is not empty).

There is now no reason to treat those two cases differently:

  - the kconfig snippet is always generated appropriately (i.e. it would
    include the br2-external tree if set, or include nothing otherwise);

  - we no longer have a dummy br-external tree either.

Also, the Makefile code to handle BR2_EXTERNAL is currently quite
readable if at least a little bit tricky.

However, when we're going to add support for using multiple br2-external
trees simultaneously, this code would need to get much, much more complex.

To keep the Makefile (rather) simple, offload all of the handling of
BR2_EXTERNAL to the recently added br2-external helper script.

However, because of Makefiles idiosyncracies, we can't use a rule to
generate that Makefile fragment.

Instead, we use $(shell ...) to call the helper script, and include the
fragment twice: once before the $(shell ...) so we can grab a previously
defined BR2_EXTERNAL value, a second time to use the one passed on the
command line, if any.

Furthermore, we can't error out (e.g. on non-existent br2-external tree)
directly from the fragment or we'd get that error on subsequent calls,
with no chance to override it even from command line.

Instead, we use a variable in which we store the error, set it to empty
before the second inclusion, so that only the one newly generated, if
any, is taken into account.

Since we know the script will always be called from Makefile context
first, we know validation will occur in Makefile context first. So we
can assume that, if there is an error, it will be detected in Makefile
context. Consequently, if the script is called to generate the kconfig
fragment, validation has already occured, and there should be no error.
So we change the error function to generate Makefile code, so that
errors are caught as explained above.

Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget'
about the previous value of the BR2_EXTERNAL_MK variable, especially in
the case where BR2_EXTERNAL is now set to empty, so that we do not try
to include it later. That's why we first generate empty version of
BR2_EXTERNAL_MK, and then assign it the new value, if any.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
64e12a370c core: get rid of our dummy br2-external tree
Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no needed.

As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer needed.

Ditch both of those files, and:

  - only generate actual content in the Kconfig snippet when we actually
    do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
    empty);

  - add a variable that contains the path to the external.mk provided by
    the user, or empty if none, and include the path set in that variable
    (make can 'include' nothing without any problem! ;-) )

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
0f4435e950 core: do not hard-code inclusion of br2-external in Kconfig
Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.

This will ultimately allow us to use more than one br2-external tree.

Offload the "User-provided options" menu to the generated Kconfig
snippet. We can also move the definition of the Kconfig-version of
BR2_EXTERNAL into this snippet.

We introduce an extra check that was not present in the previous code,
to check that we do have permission on that directory. Prevciously, it
was handled as a side effect of not being able to cd into there, but it
is cleaner to check it expressly.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Romain Naour
a2c70577ce dependencies: check for Math BigInt/BigRat Perl module
As for MakeMaker Perl module, add a check for Math BigInt/BigRat Perl
Core module, which are needed by mpv.

Fixes:
Waf: Entering directory `output/build/mpv-0.17.0/build'
[  1/198] Compiling version.sh
[ 12/198] Processing ebml_types.h: demux/ebml.c demux/demux_mkv.c -> build/ebml_types.h
[ 13/198] Compiling demux/ebml.c
Can't locate Math/BigRat.pm in @INC (you may need to install the Math::BigRat module)

Waf: Entering directory `output/build/mpv-0.17.0/build'
[  1/198] Compiling version.sh
[  2/198] Compiling TOOLS/osxbundle/mpv.app/Contents/Resources/icon.icns
[  3/198] Compiling video/out/x11_icon.bin
[  4/198] Compiling etc/input.conf
[  5/198] Compiling sub/osd_font.otf
[  6/198] Compiling video/out/opengl/nnedi3_weights.bin
[  7/198] Compiling player/lua/defaults.lua
[  8/198] Compiling player/lua/assdraw.lua
[  9/198] Compiling player/lua/options.lua
[ 10/198] Compiling player/lua/osc.lua
[ 11/198] Compiling player/lua/ytdl_hook.lua
[ 12/198] Processing ebml_types.h: demux/ebml.c demux/demux_mkv.c -> build/ebml_types.h
[ 13/198] Compiling demux/ebml.c
[ 14/198] Compiling DOCS/man/mpv.rst
Can't locate Math/BigInt.pm in @INC (you may need to install the Math::BigInt module)

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: François Perrad <francois.perrad@gadz.org>
[Thomas et al.: make those perl modules only required when mpv is enabled.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-16 12:09:49 +02:00
Romain Naour
026d47c7c5 dependencies: check for MakeMaker Perl module
ExtUtils::MakeMaker is one of the Perl Core modules usually packaged in
Perl package for a Debian/Ubuntu based system.
For a Fedora based system, each Perl Core modules have their own RPM
package. So install only Perl package is not enough.

Fixes:
>>> host-libxml-parser-perl 2.41 Configuring
[...]
perl `which perl` Makefile.PL
Can't locate ExtUtils/MakeMaker.pm in @INC (you may need to install the ExtUtils::MakeMaker module)

Add a new Perl module check in dependency.sh.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: François Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-16 12:07:54 +02:00
Thomas Petazzoni
183d9b654c support/scripts/get-developers: add new script
This script, and its companion library, is more-or-less Buildroot's
equivalent to the kernel get_maintainer.pl script: it allows to get the
list of developers to whom a set of patches should be sent to.

To do so, it first relies on a text file, named DEVELOPERS, at the root
of the Buildroot source tree (added in a followup commit) to list the
developers and the files they are interested in. The DEVELOPERS file's
format is simple:

N:     Firstname Lastname <email>
F:     path/to/file
F:     path/to/another/file

This allows to associate developers with the files they are looking
after, be they related to a package, a defconfig, a filesystem image, a
package infrastructure, the documentation, or anything else.

When a directory is given, the tool assumes that the developer handles
all files and subdirectories in this directory. For example
"package/qt5/" can be used for the developers looking after all the Qt5
packages.

Conventional shell patterns can be used, so "package/python-*" can be
used for the developers who want to look after all packages matching
"python-*".

A few files are recognized specially:

 - .mk files are parsed, and if they contain $(eval
   $(<something>-package)), the developer is assumed to be looking after
   the corresponding package. This way, autobuilder failures for this
   package can be reported directly to this developer.

 - arch/Config.in.<arch> files are recognized as "the developer is
   looking after the <arch> architecture". In this case, get-developer
   parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH
   values. This way, autobuilder failures for this package can be
   reported directly to this developer.

 - pkg/pkg-<infra>.mk are recognized as "the developer is looking after
   the <infra> package infrastructure. In this case, any patch that adds
   or touches a .mk file that uses this infrastructure will be sent to
   this developer.

Examples of usage:

$ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch
git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>"

$ ./support/scripts/get-developers -p imx-lib
Arnout Vandecappelle <arnout@mind.be>
Gary Bisson <gary.bisson@boundarydevices.com>

$ ./support/scripts/get-developers -a bfin
Waldemar Brodkorb <wbx@openadk.org>

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-09-21 09:02:13 +02:00
Yann E. MORIN
d4db4277ff manual: get rid of the lists of packages
We currently have four lists of packages in the manual:
  - the non-virtual target packages,
  - the virtual target packages,
  - the host packages,
  - the deprecated features.

Those list take more than half of the manual. They do not serve much
purpose except to show off.

After the recent discussion on the list [0], remove them all.

We can now get rid of our biggish and complex generating script (and its
companion library kconfiglib).

[0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-09-19 19:32:59 +02:00
Luca Ceresoli
c2d80a8c5d core: don't build host-cmake if it is available on the build host
Currently all cmake packages depend on host-cmake. Unfortunately
host-cmake takes a long time to configure and build: almost 7 minutes
on a dual-core i5 with SSD. The time does not change even with ccache
enabled.

Indeed, building host-cmake is avoidable if it is already installed on
the build host: CMake is supposed to be quite portable, and the only
patch in Buildroot for the CMake package seems to only affect
target-cmake.

Thus we automatically skip building host-cmake and use the one on the
system if:
 - cmake is available on the system and
 - it is recent enough.

First, we leverage the existing infrastructure in
support/dependencies/dependencies.mk to find out whether there's a
suitable cmake executable on the system. Its path can be passed in the
BR2_CMAKE environment variable, otherwise it defaults to "cmake". If
it is enabled, found and suitable then we set BR2_CMAKE_HOST_DEPENDENCY
to empty; otherwise we set BR2_CMAKE_HOST_DEPENDENCY to 'host-cmake' and
override BR2_CMAKE with "$(HOST_DIR)/usr/bin/cmake" to revert to using
our own cmake (the old behaviour).

Then in pkg-cmake.mk we replace the hard-coded dependency on host-cmake
to using the BR2_CMAKE_HOST_DEPENDENCY variable, and we use $(BR2_CMAKE)
instead of $(HOST_DIR)/usr/bin/cmake.

Unlike what we do for host-tar and host-xzcat, for host-cmake we do
not add host-cmake to DEPENDENCIES_HOST_PREREQ. If we did, host-cmake
would be a dependency for _any_ package when it's not installed on the
host, even when no cmake package is selected.

Cmake versions older than 3.0 are affected by the bug described and
fixed in Buildroot in ef2c1970e4 ("cmake: add patch to fix Qt mkspecs
detection"). The bug was fixed in upstream CMake in version 3.0 [0].

Amongst all the cmake packages currently in Buildroot, the currently
highest version mentioned in cmake_minimum_required() is 3.1 (grantlee
and opencv3).

Thus we use 3.1 as the lowest required cmake for now, until a package is
bumped, or a new package added, with a higher required version.

[0] https://cmake.org/gitweb?p=cmake.git;h=e8b8b37ef6fef094940d3384df5a1d421b9fa568

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Davide Viti <zinosat@tiscali.it>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
[yann.morin.1998@free.fr:
  - simplify logic in check-host-cmake.mk;
  - set and use BR2_CMAKE_HOST_DEPENDENCY, drop USE_SYSTEM_CMAKE;
  - bump to cmake 3.1 for grantlee and opencv;
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-09-17 16:12:06 +02:00
Peter Korsgaard
7353967690 Merge branch 'next'
Quite some conflicts, so here goes ..

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-09-02 16:20:33 +02:00
Peter Korsgaard
78e9914628 Update for 2016.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-09-01 11:30:19 +02:00
Yann E. MORIN
242e0087e5 core: move pkg-utils.mk to support/
pkg-utils.mk contains various definitions that are used in the package
infrastructures and packages themselves.

However, those definitions can be useful in other parts of Buildroot,
and are already used in a few places that are not related to the package
infrastructure. Also, $(sep) will be needed early in the Makefile when
we eventually support multiple br2-external trees.

Since this file only contains definitions, we can include it anytime.

So, consider that file to no longer be specific to the package infras:
  - move it to support and rename it,
  - move a few similar definitions from the main Makefile to that file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-27 16:03:35 +02:00
Romain Perier
ff559846fd support/download: Add support to pass options directly to downloaders
This adds support to pass options to the underlying command that is used
by downloader. Useful for retrieving data with server-side checking for
user login or passwords, use a proxy or use specific options for cloning
a repository via git and hg.

Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-23 14:59:01 +02:00
Romain Naour
9c6b9a2bb0 docs: libtool patches are not handled by legal-info infrastructure.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: slightly reword with Yann's suggestion.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-19 23:16:23 +02:00
Samuel Martin
e8c3755676 pkg-cmake.mk: set CMAKE_SYSROOT variable in toolchainfile.cmake
This change enforces the CMAKE_SYSROOT value set in the toolchainfile.cmake.

This fix overrides the CMake heuristics used to guess it, and turns off some
non-desirable behavior adding "-isystem ..." flags to the compiler command
line, misleading the compiler and making the build failed due to some
unfound standard headers.

Fixes:
  http://autobuild.buildroot.net/results/f7e/f7e92678e91a6cb15ccf32d4a7d75b39f49d6000/defconfig
  (and others)

Cc: Ben Boeckel <mathstuf@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-18 22:27:19 +02:00
Thomas Petazzoni
34b1bdeba4 size-stats: fix bug when parsing file names with comma
The size-stats script fails when the usb_modeswitch_data is enabled,
because this package installs files that contain commas in their
name. However, the size-stats script also uses comma as a separator for
its CSV files, causing a "ValueError: too many values to unpack" in:

  pkg, fpath = l.split(",")

Fix this by splitting only the two fields that need to be split.

The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested
a fix.

Fixes bug #9136.

Reported-by: Matthias <porto.rio@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-09 12:25:11 +02:00
Enrique Ocaña González
1e39f08733 support/download/git: Fix compatibility issue with git older than 1.8.4
The "--no-patch" option used by the git downloader appeared on git
1.8.4. Systems with older git versions show an error and fall back to
the wget downloader, which isn't suitable for all the cases.

Signed-off-by: Enrique Ocaña González <eocanha@igalia.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-28 22:02:13 +02:00
Romain Naour
bd3c4d8c7f support: pkg-stats: add efl package in packages stats
Since efl update to 1.15 version, the efl package is a "real"
Buildroot package. It doesn't contain any subdirectories anymore.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-24 23:25:31 +02:00
Yegor Yefremov
28eae03f5e scanpypi: include LICENCE in the list of supported license files
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-07 11:48:52 +02:00
Yann E. MORIN
d5ae67b4f4 support/apply-patches: re-instate set -e
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.

Use "set -e" to exit immediately if a command return an error.

Be sure to ignore any expected error: when we check if a patch to be
applied has the same basename as an already applied patch, the grep
would fail when no such patch was already applied. We should not fail
in this case.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-06 14:47:30 +02:00
Thomas Petazzoni
75d8977ba8 Revert "apply-patches: catch unexpected failure"
This reverts commit 9cf1ad6cdb.

This breaks many packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-05 17:42:27 +02:00
Romain Naour
9cf1ad6cdb apply-patches: catch unexpected failure
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.

Use "set -e" to exit immediately if a command return an error.

[1] http://patchwork.ozlabs.org/patch/626196

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-05 17:36:57 +02:00
Francois Perrad
28a001fc7b scancpan: improve message when bad host perl version
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[Thomas:
 - add comment in scancpan about the version dependency, suggested by
   Yann E. Morin.
 - add comment in perl.mk about the need to sync any version change with
   scancpan, also suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 17:41:54 +02:00
Yegor Yefremov
2f4bd0f339 scanpypi: change hash file comments
sha256 checksum will be computed locally either by scanpypi at package
creation or by hand by package updates. Define this checksum as
'computed locally' so that one doesn't need to change this comment by
package updates. Also put comments for both md5 and sha256 in one line.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 16:57:10 +02:00
Yann E. MORIN
d1830ca877 support/download: don't over-remove files from git archives
When we now manually create git archives, we removed all .git-related
files. However, we also exclude empty directories.

This means that a directory which only had a .gitignore file is excluded
from the archive.

Fixes:
    http://autobuild.buildroot.org/results/2aa/2aa8954311f009988880d27b6e48af91bc74c346/
    http://autobuild.buildroot.org/results/b45/b45cceea99b9860ccf1c925eeda498a823b30903/
    http://autobuild.buildroot.org/results/5ae/5ae336052fd32057d9631649279e142a81f5651f/
    http://autobuild.buildroot.org/results/5fc/5fc3abf4a1aea677f576e16c49253d00720a8bef/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-04 10:04:22 +02:00
Yegor Yefremov
f49295cc42 scanpypi: fix help text handling
Make sure a help text is terminated with a full stop.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 00:35:16 +02:00
Samuel Martin
019ba1dc52 pkg-cmake.mk: export the fortran compiler path in the CMake toolchain file
Since the fortran support is conditional, only enable it when needed.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Cc: Benjamin Kamath <bkamath@spaceflight.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-03 16:58:11 +02:00
Yann E. MORIN
ec3e057f79 core/pkg-infra: download git submodules if the package wants them
Add a new package variable that packages can set to specify that they
need git submodules.

Only accept this option if the download method is git, as we can not get
submodules via an http download (via wget).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Aleksandar Simeonov <aleksandar@barix.com>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-02 19:11:14 +02:00
Yann E. MORIN
f109e7eeb5 support/download/git: add support for submodules
Some git repositories may be split into a master repository and
submodules. Up until now, we did not have support for submodules,
because we were using bare clones, in which it is not possible to
update the list of submodules.

Now that we are using plain clones with a working copy, we can retrieve
the submdoules.

Add an option to the git download helper to kick the update of
submodules, so that they are only fetched for those packages that
require them. Also document the existing -q option at the same time.

Submodules have a .git file at their root, which contains the path to
the real .git directory of the master repository. Since we remove it,
there is no point in keeping those .git files either.

Note: this is currently unused, but will be enabled with the follow-up
patch that adds the necessary parts in the pkg-generic and pkg-download
infrastructures.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-02 19:11:14 +02:00
Yann E. MORIN
3abd5ba424 support/download/git: do not use git archive, handle it manually
We currently use git-archive to generate the tarball. This is all handy
and dandy, but git-archive does not support submodules. In the follow-up
patch, we're going to handle submodules, so we would not be able to use
git-archive.

Instead, we manually generate the archive:
  - extract the tree to the requested cset,
  - get the date of the commit to store in the archive,
  - store only numeric owners,
  - store owner and group as 0 (zero, although any arbitrary value would
    have been fine, as long as it's a constant),
  - sort the files to store in the archive.

We also get rid of the .git directory, because there is no reason to
keep it in the context of Buildroot. Some people would love to keep it
so as to speed up later downloads when updating a package, but that is
not really doable. For example:
  - use current Buildroot
  - it would need foo-12345, so do a clone and keep the .git in the
    generated tarball
  - update Buildroot
  - it would need foo-98765
For that second clone, how could we know we would have to first extract
foo-12345 ? So, the .git in the archive is pretty much useless for
Buildroot.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-02 19:11:14 +02:00
Yann E. MORIN
13c89c2f89 support/download/git: do not use bare clones
Currently, we are using bare clones, so as to minimise the disk usage,
most notably for largeish repositories such as the one for the Linux
kernel, which can go beyond the 1GiB barrier.

However, this precludes updating (and thus using) the submodules, if
any, of the repositories, as a working copy is required to use
submodules (becaue we need to know the list of submodules, where to find
them, where to clone them, what cset to checkout, and all those is
dependent upon the checked out cset of the father repository).

Switch to using /plain/ clones with a working copy.

This means that the extra refs used by some forges (like pull-requests
for Github, or changes for gerrit...) are no longer fetched as part of
the clone, because git does not offer to do a mirror clone when there is
a working copy.

Instead, we have to fetch those special refs by hand. Since there is no
easy solution to know whether the cset the user asked for is such a
special ref or not, we just try to always fetch the cset requested by
the user; if this fails, we assume that this is not a special ref (most
probably, it is a sha1) and we defer the check to the archive creation,
which would fail if the requested cset is missing anyway.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-07-02 19:11:14 +02:00
Yegor Yefremov
8a64ade2f3 scanpypi: rework runtime dependency handling
In most cases Python's package dependencies found in setup.py are
runtime dependencies and hence don't need to be mentioned in *.mk
file.

Also add '# runtime' tag to select statements in Config.in.

__create_mk_requirements() itself is left for future uses (cffi backend
handling etc.).

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-01 00:03:54 +02:00
Luca Ceresoli
d271f89b08 legal-info: explicitly state how patches are licensed
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr: slightly tweak after Arnout's review]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 16:23:14 +02:00
Yann E. MORIN
1273636fc6 core/legal-info: also save patches
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.

We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.

However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.

So, always include patches alongside the source archive.

To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.

We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.

Update the legal-info header accordingly.

Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 16:18:16 +02:00
Yann E. MORIN
1924159814 support/apply-patches: bail-out on duplicate patch basenames
Patches we save can come from various locations:
  - bundled with Buildroot
  - downloaded
  - from one or more global-patch-dir

It is possible that two patches lying into different locations have the
same basename, like so (first is bundled, second is from an hypothetical
global-patch-dir):
    package/foo/0001-fix-Makefile.patch
    /path/to/my/patches/foo/0001-fix-Makefile.patch

In that case, when running legal-info, we'd save only the second patch,
overwriting the first. That would be problematic, because:

  - either the second patch depends on the first, and thus would no longer
    apply (this is easy to detect, though),

  - or the second patch does not depend on the first, and the compliance
    delivery will not be complete (this is much harder to detect).

We fix that by checking that no two patches have the same same basename.
If we find that the basename of the patch to be applied collides with
that of a previously applied patch, we error out and report the duplicate.

The unfortunate side-effect is that existing setups will now break in
that situation, but that's a minor, corner-case issue that is easily
fixed.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: adjust coding style, fix minor typos in the commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 16:04:24 +02:00
Yann E. MORIN
a3165461bb core/apply-patches: store full path of applied patches
Currently, we only store the filename of the applied patches.

However, we are soon to want to install those patches in the legal-info
directory, so we'll have to know where those patches come from.

Instead of duplicating the logic to find the patches (bundled,
downloaded, from a global patch dir...), just store the full path to
each of those patches so we can retrieve them more easily later on.

Also always create the list-file, even if empty, so that we need not
test for its existence before reading it.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
[Tested only with patches in the Buildroot sources]
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: used $PWD instead of $(pwd), as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 15:58:38 +02:00
Denis THULIN
1008ac4406 scanpypi: new utility
A utility for creating python package from the python package index.
It fetches packages info from http://pypi.python.org and generates
corresponding packages files.

Signed-off-by: Denis THULIN <denis.thulin@openwide.fr>
Tested-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: minor tweaks.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-11 16:13:31 +02:00
Julien Floret
de2fff2174 build: cleanup remaining references to BR2_STRIP_sstrip
sstrip option has been removed in commit
106f5b8c50 ("build: remove deprecated sstrip option").

Signed-off-by: Julien Floret <julien.floret@6wind.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-10 20:15:15 +02:00
Bernd Kuhls
b571dbf347 support/gnuconfig: bump version to 2016-05-15
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-06-05 22:26:57 +02:00
Peter Korsgaard
577021e81b Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-06-01 17:55:16 +02:00
Peter Korsgaard
aa6fd11fee Update for 2016.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-05-31 23:52:36 +02:00
Yegor Yefremov
549bbba67f python/python3: globalize *.pyc files compilation
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.

To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.

The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.

The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.

A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
 - rework for python 3.5
 - integrate Samuel proposal that allows to detect compilation
   failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-26 22:07:35 +02:00
Thomas Petazzoni
8bf947e0ef support/scripts: fix graph-build-time help text
The graph-build-time help text currently looks like this:

usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER]
                        [--alternate-colors] [--input OUTPUT] --output OUTPUT

Obviously, naming the parameter for --input as OUTPUT is not a very
good idea, so this commit fixes that to name it "INPUT", as expected.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-24 23:26:25 +02:00
Yann E. MORIN
88bad8f462 support/scripts: add helper to hardlink-or-copy
When preparing the legal-info, the source archives are copied in the
legal-info/ output directory. When the archives are big, it can take
quite a bit of time and unnecessarily uses disk space. When the
legal-info output directory is on the same filesystem as the BR2_DL_DIR,
we can easily reduce copy time and disk usage by just using hardlins
instead of copying. However, the BR2_DL_DIR may be on a different
filesystem, so we must fallback to copying in this case

Introduce a helper script that copies a source file into a destination
directory, by first attempting to hard-link, and falling back to a
plain copy in case the hardlink fails.

In case the destination already exists, it is forcibly removed first, to
avoid clobering any existing target file (and especially any hardlink to
it), since cp -f does not remove the destination file, but clobbers it.

In some situations, it will be necessary that the destination file is
named differently than the source, so if a third argument is specified,
it is treated as the basename of the destination file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-11 23:14:37 +02:00
Samuel Martin
bf8f0136e9 support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}
At least syslinux is installing stuff in HOST_DIR/sbin.

Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-04-21 22:17:36 +02:00
Samuel Martin
ac031cfb97 support/scripts: fix graph-depends when run with python3
Make graph-depends script opening the output file in text mode since
only ascii characters will be written.

This change fixes the following error occuring when the default host
python interpreter is python3:

  make: Entering directory '/opt/buildroot'
  Getting targets
  Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...]
  Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...]
  Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...]
  Getting version for ['toolchain-external', 'toolchain', 'busybox', ...]
  Traceback (most recent call last):
    File "/opt/buildroot/support/scripts/graph-depends", line 425, in <module>
      outfile.write("digraph G {\n")
  TypeError: a bytes-like object is required, not 'str'
  Makefile:807: recipe for target 'graph-depends' failed
  make[1]: *** [graph-depends] Error 1
  Makefile:84: recipe for target '_all' failed
  make: *** [_all] Error 2
  make: Leaving directory '/opt/buildroot'

While with python2, adding 'b' to the openning mode has no effect on
Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]).

Therefore, just open the outfile in default (i.e. text) mode.

[1] https://docs.python.org/3/library/functions.html#open
[2] https://docs.python.org/2/library/functions.html#open

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-04-18 11:34:33 +02:00
Francois Perrad
746a91ab31 scancpan: use recommend & test flags only at first level
Currently, these flags are recursively propagated. This behavior is
not expected by users, because it can cause dependencies explosively.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-03-15 23:16:33 +01:00
Francois Perrad
d6cff4abdc scancpan: handle recommended dependencies as optional packages
Currently, without the flag -recommend, scancpan takes as dependency
only one which has the relationship "requires"; this mode works fine.
And, with the flag -recommend, scancpan takes all ones (ie. with
relationship "requires" or "recommends") in the same way; this mode
never works fine, because it is too simplistic.

With this commit, the "not required" dependencies are handled as
optional BR package or skipped if a cyclic dependency is detected.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-03-15 23:16:29 +01:00
Peter Korsgaard
aaf6c28a5c Update for 2016.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-03-01 21:47:30 +01:00
Yann E. MORIN
8773182092 support/download: really, really make git archives reproducible
The way we use it, gzip will store the current time in the header, which
leads to unreproducible archives.

Fix that by telling gzip to not store the name and date of the file it
compresses, with the -n option. Since it compresses its stdin, there was
already no filename stored; now there's even no date stored.

Note: gzip has had -n since at least 1.2.4, released in 1993, so
virtually every gzip out there nowadays has it.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-02-27 23:13:55 +01:00
Thomas Petazzoni
ee8e4c5c25 graph-depends: add support for excluding host packages
Just like the --stop-on and --exclude options allow to stop on or
exclude virtual packages from the list by passing the "virtual" magic
value, this commit extends the graph-depends logic to support a "host"
magic value for --stop-on and --exclude. This will allow to draw the
graph by stopping on host packages, or by excluding host packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: minor code beautification suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-08 21:29:06 +01:00
Thomas Petazzoni
7771bb93b2 graph-depends: fix handling of "virtual" in exclude_list
The condition to determine if a virtual package should be excluded
from the list due to "virtual" being passed in --exclude is under a
loop iterating over each entry of the exclude_list, but it doesn't use
the iterator of this list.

Indeed, the condition contains:

	"virtual" in exclude_list

which checks automatically if "virtual" was passed in the list. Due to
this, there is no need for this check to be within the "for p in
exclude_list" iteration. This commit fixes that by moving the check
outside of the loop.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-08 21:28:19 +01:00
Yann E. MORIN
90551bfac0 support/graph-depends: teach it to only check dependencies
Add an option to graph-depends to only do the dependency checks and not
generate the dot program.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:21:40 +01:00
Yann E. MORIN
f48c08f0f5 support/graph-depends: detect circular dependencies
Currently, if there is a circular dependency in the packages, the
graph-depends script just errors out with a Python RuntimeError which is
not caught, resulting in a very-long backtrace which does not provide
any hint as what the real issue is (even if "RuntimeError: maximum
recursion depth exceeded" is a pretty good hint at it).

We fix that by recursing the dependency chain of each package, until we
either end up with a package with no dependency, or with a package
already seen along the current dependency chain.

We need to introduce a new function, check_circular_deps(), because we
can't re-use the existing ones:

  - remove_mandatory_deps() does not iterate,

  - remove_transitive_deps() does iterate, but we do not call it for the
    top-level package if it is not 'all'

  - it does not make sense to use those functions anyway, as they were
    not designed to _check_ but to _act_ on the dependency chain.

Since we've had time-related issues in the past, we do not want to
introduce yet another time-hog, so here are timings with the circular
dependency check:

    $ time python -m cProfile -s cumtime support/scripts/graph-depends
    [...]
             28352654 function calls (20323050 primitive calls) in 87.292 seconds

       Ordered by: cumulative time

       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            1    0.012    0.012   87.292   87.292 graph-depends:24(<module>)
           21    0.000    0.000   73.685    3.509 subprocess.py:473(_eintr_retry_call)
            7    0.000    0.000   73.655   10.522 subprocess.py:768(communicate)
            7   73.653   10.522   73.653   10.522 {method 'read' of 'file' objects}
          5/1    0.027    0.005   43.488   43.488 graph-depends:164(get_all_depends)
            5    0.003    0.001   43.458    8.692 graph-depends:135(get_depends)
            1    0.001    0.001   25.712   25.712 graph-depends:98(get_version)
            1    0.001    0.001   13.457   13.457 graph-depends:337(remove_extra_deps)
         1717    1.672    0.001   13.050    0.008 graph-depends:290(remove_transitive_deps)
    9784086/2672326    5.079    0.000   11.363    0.000 graph-depends:274(is_dep)
    2883343/1980154    2.650    0.000    6.942    0.000 graph-depends:262(is_dep_uncached)
            1    0.000    0.000    4.529    4.529 graph-depends:121(get_targets)
      2883343    1.123    0.000    1.851    0.000 graph-depends:246(is_dep_cache_insert)
      9784086    1.783    0.000    1.783    0.000 graph-depends:255(is_dep_cache_lookup)
      2881580    0.728    0.000    0.728    0.000 {method 'update' of 'dict' objects}
            1    0.001    0.001    0.405    0.405 graph-depends:311(check_circular_deps)
    12264/1717    0.290    0.000    0.404    0.000 graph-depends:312(recurse)
    [...]
    real    1m27.371s
    user    1m15.075s
    sys     0m12.673s

The cumulative time spent in check_circular_deps is just below 0.5s,
which is largely less than 1% of the total run time.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:20:38 +01:00
Yann E. MORIN
a3f75bcc76 support/graph-depends: add option to specify output file
Currently, graph-depends outputs the dotfile program to stdout, and uses
stderr to trace the dependencies it is currently looking for.

Redirection was done because the output was directly piped into the dot
program to generate the final PDF/SVG/... dependency graph, but that
meant that an error in the graph-depends script was never caught
(because shell pipes only return the final command exit status, and an
empty dot program is perfectly valid so dot would not complain).

Add an option to tell graph-depends where to store the generated dot
program, and keep stdout as the default if not specified.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with
the rest of the new option naming.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:15:30 +01:00
Thomas De Schampheleire
8c5afd120e support/scripts: add size-stats-compare script
Leverage the CSV files produces by size-stats (make graph-size) to allow
for a comparison of rootfs size between two different buildroot
compilations.

The script takes the file-size CSV files of two compilations as input, and
produces a textual report of the differences per package.
Using the -d/--detail flag, the report will show the file size changes
instead of package size changes.
The -t/--threshold option allows to ignore file size differences smaller
or equal than the given threshold (in bytes).

Example output is:

Size difference per package (bytes), threshold = 0
--------------------------------------------------------------------------------
       -8192         busybox
      228572   added dmalloc
      301584   added jq
--------------------------------------------------------------------------------
      521964         TOTAL

or with detailed view:

Size difference per file (bytes), threshold = 0
--------------------------------------------------------------------------------
       -8192         bin/busybox
       18152   added usr/bin/jq
       39252   added usr/bin/dmalloc
       46968   added usr/lib/libdmalloc.so
       47288   added usr/lib/libdmallocxx.so
       47316   added usr/lib/libdmallocth.so
       47748   added usr/lib/libdmallocthcxx.so
      283432   added usr/lib/libjq.so.1.0.4
--------------------------------------------------------------------------------
      521964         TOTAL

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 15:52:25 +01:00
Angelo Compagnucci
778026b94d support/misc: Adding Vagrant file for provisioning
This patch adds a Vagrant file to buildroot. With this file
you can provision a complete buildroot developing environment
in minutes on all major platforms (Linux/Mac/Windows).

[Peter: bump to 2GB RAM, hardcode Buildroot release, add unzip,
	drop website update and tweak manual text as suggested by Yann]
Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-02-04 17:25:54 +01:00
Yann E. MORIN
6a8baa17a8 core/legal-info: update the legal-info report header
In some cases, the toolchain sources are now recovered and available in
the legal-info output. So, adapt the header to use conditional instead
of an definitive negation.

Also update the part about saving the sources: it's not the license list
that defines whether sources are installed, but rather whether the
package is redistributable or not.

Update the header accordingly.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-01 22:04:39 +01:00
Peter Korsgaard
53703ee67a support/scripts: drop ancient build-ext3-img script
It hasn't been updated since it was added in 2008, and nowadays things kind
of stuff should be handled with genimage.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-02-01 11:51:08 +01:00
Henrique Marks
0f56304521 merge_config.sh: create temporary files in /tmp
Creating temporary files in /tmp (or the path pointed by $TMPDIR) allows the
buildroot top directory to be read-only and shareable between multible builds.
This follows what other scripts do, e.g. check-kernel-headers.sh.

Signed-off-by: Henrique Marks <henrique.marks@datacom.ind.br>
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-31 09:09:52 +01:00
Joao Mano
4fad12598d support/download: alternative access methods to CVS
Allows user to specify other access methods than :pserver:anonymous@
on CVS repositories. This shall be defined in the <pkg>_SITE variable.

[Thomas:
 - as suggested by Yann, quote the variable expansion
 - as suggested by Yann, use a regexp match
 - tweak commit log]

Signed-off-by: Joao Mano <joao@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-01-20 23:26:03 +01:00
Yann E. MORIN
b9be0f65ac support/download: support older bazaar versions
In efe7f68 (support/download: generate reproducible Bazaar archives),
bzr was instructed to store files with the timestamp set to the date
they were last modified in the repository, instead of the current date,
using the --per-file-timestamp option.

However, this option has been added only in bzr-2.2 (August 2010) which
is not available on older distros.

We fix that by not using --per-file-timestamp when the bzr version is
older than 2.2, in which case we just generate the archive with the
current date set on files.

This means the archive is thus non-reproducible, and if a hash is
available for that archive, the hash will not match, and Buildroot will
try to download from the mirror (if any) or fail (if no mirror).

Fixes:
    http://autobuild.buildroot.org/results/51f/51f4ff5462c15a85937d411f457096224d00fdcd
    http://autobuild.buildroot.org/results/b88/b8828b5fbc16128408c2f44169ac23de7e34d770
    http://autobuild.buildroot.org/results/fb4/fb4b0fb2131b40c18273dbe5e51b393cb6df18ec
    ...

[Peter: simplify sed invocation]
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-18 20:46:04 +01:00
Arnout Vandecappelle
c1a674f994 support/scripts/apply-patches.sh: fix whitespace
The apply-patches.sh script was using a mix of tabs and spaces, and
some three-space indentation. Normalize everything to four-space
indentation.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-13 22:09:08 +01:00
Thomas Petazzoni
a059055c52 support/scripts/size-stats: fix copy/paste error in ArgumentParser
A copy/paste error in the ArgumentParser() constructor call disclosed
the fact that the author of the script has shamefully based his work
on the existing graph-build-time script. This commit fixes this
mistake, therefore hiding in a better way how size-stats was
vampirized from graph-build-time.

Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-01-13 16:35:19 +01:00
Yann E. MORIN
efe7f684a6 support/download: generate reproducible Bazaar archives
Similarly to what has previously been done for the Hg download backend,
instruct bzr to generate the archive on stdout, so that we can generate
reproducible archives.

When instructing bzr to generate the output file by itself, it uses a
temporary file that is then fed to gzip, which in turn stores the
timestamp of that file in the generated archive, whereas when the output
is generated on stdout, there is no timestamp, so the archive is then
reproducible.

Bizarely enough, we can tell 'bazaar' not to generate a bazaar in the
archive. Cool, uh? ;-]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-03 21:51:50 +01:00
Yann E. MORIN
76b51f90c0 support/download: generate reproducible Hg archives
When hg directly creates the output file, the hash for that file changes
everytime.

However, if we just tell hg to output the archive on stdout and we do
the redirect to the file, then the archive is reproducible.

(The reason is that in the first case, a temporary file is created and
then compressed, and gzip is adding the filename and its timestamp in
the gzip header, while in the second case, there is no temporary file,
and thus no timestamp and thus it is reproducible.)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-03 21:50:38 +01:00
Peter Korsgaard
f62ce058be graph-depends: correct is_dep() comment
The uncached variant is called is_dep_uncached(), not is_dep_full().

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-12-29 23:46:10 +01:00
Thomas Petazzoni
5ed60cee1e graph-depends: optimize remove_transitive_deps()
For large configurations, the execution time of
remove_transitive_deps() becomes really high, due to the number of
nested loops + the is_dep() function being recursive.

For an allyespackageconfig, the remove_extra_deps() function takes 334
seconds to execute, and the overall time to generate the .dot file is
6 minutes and 39 seconds. Here is a timing of the different
graph-depends steps and the overall execution time:

  Getting dependencies:   42.5735 seconds
  Turn deps into a dict:   0.0023 seconds
  Remove extra deps:     334.1542 seconds
  Get version:            22.4919 seconds
  Generate .dot:           0.0197 seconds

  real	6m39.289s
  user	6m16.644s
  sys	0m8.792s

By adding a very simple cache for the results of is_dep(), we bring
down the execution time of the "Remove extra deps" step from 334
seconds to just 4 seconds, reducing the overall execution time to 1
minutes and 10 seconds:

  Getting dependencies:  42.9546 seconds
  Turn deps into a dict:  0.0025 seconds
  Remove extra deps:      4.9643 seconds
  Get version:           22.1865 seconds
  Generate .dot:          0.0207 seconds

  real	1m10.201s
  user	0m47.716s
  sys	0m7.948s

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Gustavo Zacarias <gustavo.zacarias@free-electrons.com>
[yann.morin.1998@free.fr:
  - rename is_dep() to is_dep_uncached(), keep existig code as-is
  - add is_dep() as a cached-version of is_dep_uncached()
  - use constructs more conform with 2to3
  - use exceptions (EAFP) rather than check-before-use (LBYL) to be more
    pythonist; that even decreases the duration yet a little bit more!
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2015-12-29 23:26:22 +01:00
Yann E. MORIN
3f2bdd0701 support/download: protect from custom commands with spaces in args
Some users may provide custom download commands with spaces in their
arguments, like so:
    BR2_HG="hg --config foo.bar='some space-separated value'"

However, the way we currently call those commands does not account
for the extra quotes, and each space-separated part of the command is
interpreted as separate arguments.

Fix that by calling 'eval' on the commands.

Because of the eval, we must further quote our own arguments, to avoid
the eval further splitting them in case there are spaces (even though
we do not support paths with spaces, better be clean from the onset to
avoid breakage in the future).

We change all the wrappers to use a wrapper-function, even those with
a single call, so they all look alike.

Note that we do not single-quote some of the variables, like ${verbose}
because it can be empty and we really do not want to generate an
empty-string argument. That's not a problem, as ${verbose} would not
normally contain space-separated values (it could get set to something
like '-q -v' but in that case we'd still want two arguments, so that's
fine).

Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-12-12 17:00:46 +01:00
Ryan Barnett
aab481c14e apply-patches: only use first field of line for series file
A series file for quilt has a valid syntax of:

  fixes/autoconf.diff -p1
  fixes/doc-html-local-css.diff -p1
  fixes/gnu-inline.diff -p1

However, with the current way that a series file is handled, it will
error out because the -p1 is tried as a file. This is because in the
for loop that iterates the files, we only look for comment lines. Then
each line is used within a bash for loop which uses spaces a
delimiter. In order to fix this, we should only use the string that
comes before a space in the series file.

Note that the format allows for any arbitrary depth to the -pN field.
But since we'll have only one package with -pN fields, and all will be
-p1, we for now always assume -p1. This will have to be fixed whenever
we get a package with other values.

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
[yann.morin.1998@free.fr: expand comment about the format of a series
file and how we interpret it]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
CC: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-12-12 14:34:47 +01:00
Yann E. MORIN
f42e262199 support/check-host-rpath: remove trailing '/' in host dir
Despite the comment saying so, the trailing '/' in the host directory is
not removed. Note however that it is properly removed from extracted
RPATH tags.

This is not visible when the host directory is our default $(O)/host
location, but breaks for user-supplied external host directory, when
the user leaves a trailing slash in the path.

Fix that.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-12-02 23:28:14 +01:00
Peter Korsgaard
cc257ee493 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-12-01 23:24:07 +01:00
Vivien Didelot
f8b8251a92 support/download: fetch all refs on full git clone
When specifying BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION, a user may want to
specify the SHA of a reference different than a branch or tag.

For instance, Gerrit stores the patchsets under refs/changes/xx/xxx, and
Github stores the pull requests under refs/pull/xxx/head.

When cloning a repository with --bare, you don't fetch these references.
This patch uses --mirror for a full clone, in order to give the user
access to all references of the Git repository.

Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: "Maxime Hadjinlian" <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-11-29 18:52:44 +01:00
Yann E. MORIN
5c0c385226 core: check host executables have appropriate RPATH
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.

We currently ensure that in two ways:
  - we add a RPATH tag that points to our host library directory,
  - we export LD_LIBRARY_PATH to point to that same directory.

With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.

However, it turns our that not all the host programs we build end up
with an RPATH tag:
  - some packages do not use our $(HOST_LDFLAGS)
  - some packages' build system are oblivious to those LDFLAGS

In this case, there are two situations:
  - the program is not linked to one of our host libraries: it in fact
    does not need an RPATH tag [0]
  - the program actually uses one of our host libraries: in that case it
    should have had an RPATH tag pointing to the host directory.

For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.

We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.

[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-18 22:47:10 +01:00
Arnout Vandecappelle
7caf46439c apply-patches.sh: use everything from the series file
When a series file exists, we should use every file mentioned in it,
not just the ones ending with .patch or .diff. Also, there's no need
to uncompress anything if it's mentioned in a series file (the tools
that manipulate series files don't support compressed patches).

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Doug Kehn <rdkehn@yahoo.com>
Tested-by: Doug Kehn <rdkehn@yahoo.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-17 09:24:53 +01:00
Cédric Marie
474d39a1ff Makefile: Remove 'quiet' variable
'quiet' variable is set and exported, but it is not used. We can safely
remove it.

This variable is inherited from the Makefile of the Linux kernel, and
is not used in Buildroot.

In support/scripts/mkmakefile, 'quiet' value is checked, but the test
is always true ('quiet' is never set to silent_), so the test can be
removed as well.

Signed-off-by: Cédric Marie <cedric.marie@openmailbox.org>
Reviewed-by: "James Knight" <james.d.knight@live.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-29 00:04:05 +01:00
Francois Perrad
e2aff3d7f5 scancpan: README as default license file
add this heuristic when no specific license file is found

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-26 22:56:05 +01:00
Yann E. MORIN
f0c4456530 support/mkusers: allow comments in users tables
The format of the users table files is non trivial, so it is sometimes
handy to add comments explaining the syntax (or simply the reason for
the user) inline in the files.

Ignore empty lines and comment lines prefixed with '#' similar to shell
or makedevs files.

Packages that defined no user (the vast majority) would cause an empty
line to be present in the internal users table, hence the reason we
skipped empty usernames. Now that we ignore empty lines, we no longer
need to check for empty usernames.

Reported-by: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-18 17:52:21 +02:00
Thomas Petazzoni
598c80be8f support/scripts: add size-stats script
This new script uses the data collected by the step_pkg_size
instrumentation hook to generate a pie chart of the size contribution
of each package to the target root filesystem, and two CSV files with
statistics about the package size and file size. To achieve this, it
looks at each file in $(TARGET_DIR), and using the
packages-file-list.txt information collected by the step_pkg_size
hook, it determines to which package the file belongs. It is therefore
able to give the size installed by each package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-17 16:12:59 +02:00
Vicente Olivert Riera
c4339a0823 dependencies.sh: improve the missing perl modules detection
[Thomas:
 - Check for Thread::Queue, not Thread:Queue.
 - Use 'printf' instead of 'echo -e', since printf is POSIX, but not
   'echo -e'.]

Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-15 22:51:50 +02:00
Arnout Vandecappelle
e31ad0e809 package-cmake: remove now-redundant target ccache support
All the complexity with the different ways that CMAKE_C_COMPILER and
CMAKE_C_COMPILER_ARG1 can be set are no longer needed, it's all handled
by the toolchain wrapper now.

Note that it is still necessary to handle this for the host build.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-04 18:22:20 +02:00
Francois Perrad
2e39ffcc6a pkg-perl: refactor perl infrastructure
the perl dependency of cpan module is no longer generated by scancpan,
but added at the infrastructure level

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-02 20:56:52 +01:00
Francois Perrad
ceb1214632 scancpan: fix detection of native module
When a module is native or depends of a native module, it must be
disabled for static builds via its Config.in

We detect native modules by looking at the filenames listed in the
MANIFEST. If there is a file which looks like it contains code that
much be compiled (e.g. .c, .h and so on...), then we exclude that
module (and its dependencies) from static builds.

That's what we tried to do so far, but failed when there was a
comment on the same line as the filename in the manifest, like so:
    foo-bar.c # Bla bla bla

Fix that by detecting either endof-line (as currently done) or
end-of-string.

For an example of failed build of perl-html-parser, see
http://autobuild.buildroot.net/results/128/128671dfa23d843698a63220c2fac1f44e1d5845/

[Thomas: use better commit log proposed by Yann E. Morin.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-09-20 15:14:10 +02:00
Francois Perrad
8498474ce3 scancpan: remove hack for Module-Build
with Perl 5.22, Module-Build is no longer a core module

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-09-06 22:48:49 +02:00
Francois Perrad
072986df1c perl: bump to version 5.22.0
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-09-06 22:48:42 +02:00
Yann E. MORIN
697e2b7d94 support/download: fix the Hg backend for tags
When the version of a package is a Mercurial tag, the download fails,
with:
    abort: unknown revision 'X.Y.Z'!

This is because, in Mercurial, tags are commits like the others, and
when we clone, we actively request a tag. But then, the server
"dereferences" that tag and sends us the revision pointed to by that
tag. Of course, since the tag is a commit after the revision we got,
we do not have the revision adding the tag.

So, we just have to download the full repository to be sure we have
the tags in our local clone.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-08-29 11:38:00 +02:00
Yann E. MORIN
9d0d4c3cd2 docs/manual: fix generation of deprecated list
Since commit 5f117c3 (webkit: mark as deprecated), generation of the
manual has been broken.

This is because that commit added a deprecated dependency on a
prompt-less symbol, BR2_PACKAGE_WEBKIT_ARCH_SUPPORTS. However, the
generation script does not check that a symbol has a prompt before
it attempts to add it to the deprecated list. So, we end up with
traceback:

    Writing the virtual-packages list in:
            /home/ymorin/dev/buildroot/O/build/docs/manual/virtual-package-list.txt
    Traceback (most recent call last):
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 510, in <module>
        buildroot.print_list(list_name, dry_run=args.dry_run, output=output)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 466, in print_list
        item_label=item_label)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 126, in format_asciidoc_table
        enable_choice=enable_choice))
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 350, in _format_symbol_prompt_location
        return "| {0:<40} <| {1}\n".format(get_label_func(symbol),
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 458, in <lambda>
        get_label = lambda x: self._get_symbol_label(x, mark_depr)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 313, in _get_symbol_label
        label = symbol.get_prompts()[0]
    IndexError: list index out of range

However, we can not use the existing _is_deprecated filter function to
filter out symbols without prompts, because this function is also used
to add a '(deprecated)' tag in the man package list (not that it would
not work, but it does not seem /right/). Furthermore, it could also be
used (but is currently not) to build the list of virtual packages, which
do not have a prompt.

So, introduce a filter function, aptly named _is_deprecated_feature(),
to be used as the filter to find deprecated feature, and keep the
existing _is_deprecated() that can be used in any context to decide
whether a symbol is deprecated or not.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-08-02 19:18:51 +02:00
Yann E. MORIN
3dea23cf53 core/download: don't be needlessly verbose in backends
In 50c8b7e (support/download: support -q in all download backends), the
backend were made to respect the quietness of the main Makefile, when -s
is poassed on the 'make' command line. In doing so, they were all made
to be verbose by default.

However, the verbosity of some of the tools, like scp, is very high, and
is in fact intended for debug purposes.

Drop being verbose by default, just use whatever each tool deems normal
output. Only respect the quietness requested by the user.

Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-26 15:18:39 +02:00
Francois Perrad
6e9d322a11 scancpan: adjust package name
underscore is not allowed in BR package name.
this problem was found with the Perl module DB_File
which must give the BR package perl-db-file.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-23 22:36:18 +02:00
Yann E. MORIN
6bae1ac575 core/out-of-tree: fix Makefile wrapper
Commit 971faf8 (Makefile: fix out-of-tree builds with multiple targets
with 'all') renamed the default target to '_all' to avoid name-clashing.

In doing so, I forgot to also fix the instance in the .PHONY rule.

Fix that now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-16 23:38:52 +02:00
Maxime Hadjinlian
f53d69614e graph-depends: Strip skeleton from dependency
skeleton being a mandatory dependency, we don't want all our packages to
have a link back to that node, the graph would be awful.

Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-14 18:10:58 +02:00
Guido Martínez
fd13247a22 scripts: mkmakefile: set umask before calling BR's makefile
Small optimization so we don't have another 'make' level (caused by the
umask fix) when running the generated makefile.

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-13 17:19:56 +02:00
Thomas Petazzoni
ef7cc99c7a pkg-generic: fix fallout of <pkg>_STRIP_COMPONENTS introduction
The introduction of <pkg>_STRIP_COMPONENTS broke the build of the
target tar package, because support/dependencies/check-host-tar.mk
defines TAR_STRIP_COMPONENTS to --strip-components. Which leads to
have the package infrastructure do:

 $$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS)

which for the tar package evaluates to:

 $$(TAR_STRIP_COMPONENTS)=$$(TAR_STRIP_COMPONENTS)

which evalutes to:

 --strip-components=--strip-components

Which obviously doesn't work really well. And in fact the
TAR_STRIP_COMPONENTS definition in
support/dependencies/check-host-tar.mk is no longer necessary: it was
needed in the days where we were trying to support old tar versions
that did not support --strip-components. But nowadays, when such an
old tar version is encountered, we build our own host-tar which
supports --strip-components.

Fixes:

  http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-12 13:35:55 +02:00
Fabio Porcedda
6c0d6592f4 support/download/cvs: add support to use a date instead of a tag
This is useful when a tag is not avaiable.

Also fix support for Fedora where the command "cvs -r :<version>" doesn't work.

Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-10 11:09:20 +02:00
Ulf Magnusson
954c09ae62 support: remove outdated note re. Python 3 support
Kconfiglib now runs as either Python 2 or Python 3.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-12 21:19:52 +02:00
Ulf Magnusson
b4250efb2f support: get prompts via official Kconfiglib APIs
These weren't available when gen-manual-lists.py was first written.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:21 +02:00
Ulf Magnusson
aba6fa137a support: gen-manual-lists.py base directory simplification
Buildroot doesn't use $srctree from what I could tell.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:07 +02:00
Ulf Magnusson
00cc243170 support: update Kconfiglib to the latest version
Corresponds to a95f477 in https://github.com/ulfalizer/Kconfiglib.

Fixes:

 - Unset user values when loading a zero-byte .config. (5e54e2c)

 - Ignore indented .config assignments. (f8a7510)

 - Do not require $srctree to be set for non-kernel projects. (d56e9c1)

 - Allow digits in $-references to symbols. (ecacdd5)

 - Add Symbol.is_allnoconfig_y(). (deaa624)

 - Fix small output issue with Comments inside Choices.

Also adds Python 3 support and has a lot of internal cleanup and
optimization.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:01 +02:00
James Knight
8d972df185 scripts/mkusers: allow users with no password value set
The following allows a user definition to specify that a created user
entry should not have a password value set. Original implementation
allowed a user definition to provide a password value of "=" (no quotes)
to generate a crypt-encoded empty string value. In some cases, it may be
desired to have no value specified for a user's password. By using a
value "-" for a password, no value will be set in the shadow value.

An example when this can be used is when logging into a terminal.
Logging into a session with an encoded empty password will prompt a user
to enter a password since it does not know the password is empty. If the
password field blank, a login session will not prompt for a password.

Signed-off-by: James Knight <james.knight@rockwellcollins.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-09 23:13:41 +02:00
Baruch Siach
ef92d32f19 support/download: relocate the git clone comment
Following commit 95a572282e (pkg-infra: move the git download helper to a
script, 2014-07-02), move the comment describing the shallow clone trickery as
well. Merge this comment with the existing helper comment that was added in
7e40a1103a (support/download: convert git to use the wrapper, 2014-08-03).

Rename $($(PKG)_DL_VERSION) to ${cset} to match the helper code context.

Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-05-31 11:56:56 +02:00
Gergely Imreh
965e3b96b6 fix typo in package install suggestion for dependencies
Signed-off-by: Gergely Imreh <imrehg@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-05-08 07:16:20 +02:00
Yann E. MORIN
f75a9f07a6 support/download: restore mandatory check of hashes
Now that custom external toolchains to be downloaded properly instruct
to not fail on a missing hash, restore the mandatory hash check for
everything else.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-25 11:51:44 +02:00
Yann E. MORIN
8d2f4e6246 support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).

So, add a way to avoid failing in that case.

>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

changes v6 -> v7:
  - /beautify/ the pattern in the case clause

Changed v5 -> v6:  (Arnout)
  - fix the pattern in the case clause

Changes v4 -> v5:
  - micro-optimisation, use case-esac instead of a for-loop  (Arnout)
  - typoes  (Arnout)

Changes v3 -> v4:
  - drop the magic value, use a list of excluded files  (Arnout)

Changes v1 -> v2:
  - fix typoes in commit log

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-25 11:51:08 +02:00
Thomas Petazzoni
8a58e0238e Makefile: rename TARGETS to PACKAGES
For clarity, this commit renames the TARGETS variable to the more
meaningful PACKAGES variable. Indeed, only packages (handled by one of
the package infrastructures) should be listed in this variable, and
not other random non-package targets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2015-04-14 10:16:51 +02:00
Masahiro Yamada
5b686a0674 Fix typos in comment blocks
[Thomas: fix issues noticed by Arnout:
  - Rewrap the linux/Config.in paragraph
  - Revert the "is a toolchain dependency" -> "has a toolchain
    dependency" change from pkg-generic.mk, as the original was
    correct.]

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-08 22:44:43 +02:00
Yann E. MORIN
aa3a409b05 support/download: do not warn about missing hash file for repositories
When downloading from a repository, we explicitly pass no hash file,
because we can't check hashes in that case.

However, we're still printing a message that there is a missign hash
file.

Beside being a bit annoying (since we can't do anything about it), it
may also be wrong, especially for packages for which we support multiple
versions, with some being downloaded via a git clone and others as
tarballs.

Just print no warning when the path to the hash file is empty.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-06 23:29:02 +02:00
Yann E. MORIN
209b754996 support/download: quick fix to avoid breaking on custom toolchains
When the user selects a custom toolchain to be downloaded, there's no
hash for that toolchain, so the download fails, now that hashes are
mandatory.

Fix that by simply exiting as if there was no error, until we have a
better fix...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:21:47 +02:00
Yann E. MORIN
b6b833e7bc support/download: warn when there's no .hash file
Instead of silently accepting a missing .hash file, print a warning.

This can be grepped from a build log, to find packages that still have
no hash, with the long-term goal of adding hashes for all packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:54 +02:00
Yann E. MORIN
3275315432 support/download: always fail when there's no hash
At the time we introduced hashes, we did not want to be too harsh in the
beginning, and give people some time to adapt and accept the hashes. So
we so far only whined^Wwarned about a missing hash (when the .hash file
exists).

Some time has passed now, and people are still missing updating hashes
when bumping packages.

Let's make that warning a little bit more annoying...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:54 +02:00
Yann E. MORIN
3c19d511d7 support/download: properly catch missing hashes
When checking hashes reports no hash for a file, and this is treated as
an error (now: because BR2_ENFORCE_CHECK_HASH is set; later: because
that will be the new and only behaviour), exit promptly in error.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:54 +02:00
Yann E. MORIN
0012b36c14 support/download: return different exit codes for different failures
Return different exit codes depending on the error that occured:

  0: no error (hash file missing, or all hashes match)
  1: unknown option
  2: hash file exists, but at least one hash in error
  3: hash file exists, but no hash for file to check
  4: hash file exists, but at least one hash type unknown

This will be used in a later patch to decide whether the downloaded file
should be kept or removed.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:50 +02:00
Yann E. MORIN
1ba85b7f87 support/download: add explicit no-hash support
Add support to explicitly state that an archive has no hash.

This can be used for archives downloaded from a repository, like a
git-clone or a subversion checkout, or using the github helper.

This will come in handy when we'll eventually make hashes mandatory as
soon as a .hash file exists: for some packages, like gcc, some versions
are downloaded as archives from upstream, while other versions may come
from a GitHub repository (via the github herlper).

In this case, a .hash file would exist, that contains hashes for the
downloaded tarballs, but archives downloaded from the repository would
not have a hash (since it is currently not possible to have reproducible
such archives). So, we'd need a way to explicitly state there is no
hash, on purpose, for those archives.

So, add 'none' as a new type of hash.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:00 +02:00
Yann E. MORIN
c22fe04a78 support/download: make hash file optional
Currently, specifying a hash file for our download wrapper is mandatory.

However, when we download a git, svn, bzr, hg or cvs tree, there's by
design no hash to check the download against.

Since we're going to have hash checking mandatory when a hash file
exists, this would break those downloads from a repository.

So, make specifying a hash file optional when calling our download
wrapper and bail out early from the check-hash script if no hash file is
specified.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-03 14:08:00 +02:00
Yann E. MORIN
5bcfaaaf4e support/graph-depends: allow excluding virtual packages
Like for --stop-on, make --exclude recognise the keyword 'virtual',
to stop on virtual packages (as explained in the help...).

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-01 20:02:54 +02:00
Yann E. MORIN
64b70f55cb support/graph-depends: add option to completely exclude a package
Similar to --stop-on, but also omits the package from the graph.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:55:31 +02:00
Yann E. MORIN
979267e8f7 support/graph-depends: accepts globs to stop on package
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:52:31 +02:00
Yann E. MORIN
5558a34e9a support/graph-depends: add option to stop on specific packages
Add a new option to graph-depends, that users can set to stop the graph
on a specific (set of) package(s).

This accepts any actual package name, or the 'virtual' keyword to stop
on virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:50:11 +02:00
Yann E. MORIN
96a8321a3c support/mkusers: enhance error messages
The current error messages are a bit terse, and do not provide all the
required information.

Expand them to provide more context.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Tom Elliott <tommygunsster@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-29 13:39:25 +02:00
Samuel Martin
252e57cf16 toolchainfile.cmake: only search the sysroot for CMake module
This change prevents CMake from searching outside the sysroot location
for CMake modules when cross-compiling.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-16 22:16:45 +01:00
Alexey Brodkin
0b32791f00 graph-depends: remove absent targets from TARGET_EXCEPTIONS
Most of targets listed in TARGET_EXCEPTIONS these days are long
gone, so why still keep them?

Most of those targets were removed in this commit:
http://git.buildroot.net/buildroot/commit/?id=02b88600312554bf166f6cfd71f7f2ede783096a

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@sunsite.dk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-03-11 17:18:24 +01:00
Francois Perrad
ef0b091ca4 graph-depends: display virtual package with italic style
virtual packages are found by their version,
so we retrieve the version of all packages

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-08 22:17:11 +01:00
Fabio Porcedda
ff82572a7f pkg-stats: ignore linux-ext-fbtft.mk and doc-asciidoc.mk
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-02-15 23:28:55 +01:00
Fabio Porcedda
cd0c7c37c2 pkg-stats: add kconfig and rebar infrastructures
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-02-15 23:28:44 +01:00
Bjørn Forsman
8868051262 kconfig/lxdialog: get ncurses CFLAGS with pkg-config
This makes "make menuconfig" also work on systems where ncurses is not
installed in a standard location (such as on NixOS).

This patch changes ccflags() so that it tries pkg-config first, and only
if pkg-config fails does it go back to the fallback/manual checks. This
is the same algorithm that ldflags() already uses.

[This patch is already applied upstream (is part of linux v3.18):
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=be8af2d54a66911693eddc556e4f7a866670082b

I'm adding this instead of doing a full upstream kconfig sync because
there was a conflict in one of the Buildroot kconfig patches (against
linux 3.18-rc1), which I was unable to resolve. Just drop this patch next time
Buildroot kconfig is synced against upstream.
]

Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-01-07 22:26:53 +01:00
Yann E. MORIN
4bf1174a83 support/download: further silence the git helper
We expresely call printf in the git helper, calls which were not
addresed in the previous silent-build patchset.

Just redirect stdout to oblivion when being silent.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Tested-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-01-05 15:15:09 +01:00
Fabio Porcedda
0b600227b0 pkg-download: silence downloads if make is silent
If doing a silent build (make -s -> QUIET=-q), silence all downloads,
by passing the -q flag downward to backends as well as to check-hash.

Change a printf to use the trace functions.

Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-01-04 18:39:46 +01:00
Yann E. MORIN
50c8b7e947 support/download: support -q in all download backends
Add an option flag to all backends, as well as the check-hash script, so
as to silence download helpers when the user wants a silent build.

Additionaly, make the default be verbose.

Inspired by Fabio's patch on git/svn.

[Thomas: fix a typo "Environemnt" -> "Environment"

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-01-04 18:38:36 +01:00
Gustavo Zacarias
4a28e3523d support/libtool: add patch for newer versions
Add patch to support newer (>2.4.2) versions.
Adjust patch logic to check for patchlevel greater than 2 (apply new patch) or
not (apply current patch).
Some people/distributions used unreleased versions, with the string being
2.4.2.x, this packages are AUTORECONFed and have to be kept like this since
the up-to-2.4.2 patch doesn't work, neither does the from-2.4.3 version patch.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-21 13:21:56 +01:00
Yann E. MORIN
9bb7d10eea pkg-download: check hashes for locally cached files
In some cases, upstream just update their releases in-place, without
renaming them. When that package is updated in Buildroot, a new hash to
match the new upstream release is included in the corresponding .hash
file.

As a consequence, users who previously downloaded that package's tarball
with an older version of Buildroot, will get stuck with an old archive
for that package, and after updating their Buildroot copy, will be greeted
with a failed download, due to the local file not matching the new
hashes.

Also, an upstream would sometime serve us HTML garbage instead of the
actual tarball we requested, like SourceForge does from time for as-yet
unknown reasons.

So, to avoid this situation, check the hashes prior to doing the
download. If the hashes match, consider the locally cached file genuine,
and do not download it. However, if the locally cached file does not
match the known hashes we have for it, it is promptly removed, and a
download is re-attempted.

Note: this does not add any overhead compared to the previous situation,
because we were already checking hashes of locally cached files. It just
changes the order in which we do the checks. For the records, here is the
overhead of hashing a 231MiB file (qt-everywhere-opensource-src-4.8.6.tar.gz)
on a core-i5 @2.5GHz:

            cache-cold  cache-hot
    sha1      1.914s      0.762s
    sha256    2.109s      1.270s

But again, this overhead already existed before this patch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-11 23:59:41 +01:00
Yann E. MORIN
9b88c60484 pkg-download: verify the hashes from the download wrapper
Instead of repeating the check in our download rules, delegate the check
of the hashes to the download wrapper.

This needs three different changes:

  - add a new argument to the download wrapper, that is the full path to
    the hash file; if the hash file does not exist, that does not change
    the current behaviour, as the existence of the hash file is checked
    for in the check-hash script;

  - add a third argument to the check-hash script, to be the basename of
    the file to check; this is required because we no longer check the
    final file with the final filename, but an intermediate file with a
    temporary filename;

  - do the actual call to the check-hash script from within the download
    wrapper.

This further paves the way to doing pre-download checks of the hashes
for the locally cached files.

Note: this patch removes the check for hashes for already downloaded
files, since the wrapper script exits early. The behaviour to check
localy cached files will be restored and enhanced in the following
patch.

[Thomas: fix minor typo in comment.]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-11 23:59:25 +01:00
Yann E. MORIN
2685937e06 pkg-download: check for already downloaded file in the download wrapper
Instead of repeating the same test again and again in all our download
rules, just delegate the check for an already downloaded file to the
download wrapper.

This clears up the path for doing the hash checks on a cached file
before the download.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-11 23:58:44 +01:00
Yann E. MORIN
78b92e5055 support/download: add option parsing to the download wrapper
Instead of relying on argument ordering, use actual options in the
download wrapper.

Download backends (bzr, cp, hg...) are left as-is, because it does not
make sense to complexify them, since they are almost very trivial shell
scripts, and adding option parsing would be really overkill.

This commit also renames the script to dl-wrapper so it looks better in
the traces, and it is not confused with another wrapper.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-11 23:58:37 +01:00
Thomas Petazzoni
665e13c85e Rename BR2_PREFER_STATIC_LIB to BR2_STATIC_LIBS
Since a while, the semantic of BR2_PREFER_STATIC_LIB has been changed
from "prefer static libraries when possible" to "use only static
libraries". The former semantic didn't make much sense, since the user
had absolutely no control/idea of which package would use static
libraries, and which packages would not. Therefore, for quite some
time, we have been starting to enforce that BR2_PREFER_STATIC_LIB
should really build everything with static libraries.

As a consequence, this patch renames BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS, and adjust the Config.in option accordingly.

This also helps preparing the addition of other options to select
shared, shared+static or just static.

Note that we have verified that this commit can be reproduced by
simply doing a global rename of BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS plus adding BR2_PREFER_STATIC_LIB to Config.in.legacy.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2014-12-11 22:48:13 +01:00
Volker Krause
c1e41153e6 pkg-cmake.mk: Set CMAKE_SYSTEM_PROCESSOR.
This is rarely needed by packages, but convenient to have when it is.

[Thomas:
  - don't define ARM_VARIANT as this name is too global, use
    CMAKE_SYSTEM_PROCESSOR_ARM_VARIANT instead.
  - don't use ifndef, but a more traditional else clause, for the
    non-ARM cases.]

Signed-off-by: Volker Krause <volker.krause@kdab.com>
Reviewed-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-09 23:00:12 +01:00
Francois Perrad
315d29e159 support/script/scancpan: remove duplicated dependency
Dependencies from metacpan comes as a list of modules which is
transformed in a list of distribution for BR.  Different modules could
be included in the same distribution, so duplication is possible.

This can for example be seen with the HTTP-Daemon module, which would
get two times the dependencies on HTTP-Message without this commit.

[Thomas: slightly extend commit log.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-09 21:15:09 +01:00
Jens Stimpfle
94c6417e70 support/scripts/dependencies.sh: cleanup the output of the 32 bits gcc test
The screen is cluttered when we build for 32 bit target and 32 bit gcc
is missing.

  ~/buildroot$ make
  [...]
  /usr/bin/ld: cannot find crt1.o: No such file or directory
  /usr/bin/ld: cannot find crti.o: No such file or directory
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a when searching for -lgcc
  /usr/bin/ld: cannot find -lgcc
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so when searching for -lgcc_s
  /usr/bin/ld: cannot find -lgcc_s
  /usr/bin/ld: cannot find -lc
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a when searching for -lgcc
  /usr/bin/ld: cannot find -lgcc
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so when searching for -lgcc_s
  /usr/bin/ld: cannot find -lgcc_s
  /usr/bin/ld: cannot find crtn.o: No such file or directory
  collect2: error: ld returned 1 exit status
  [...]
  Your Buildroot configuration needs a compiler capable of building 32 bits binaries.

The final note is enough, and adding 2>/dev/null to the gcc test
invocation is also more consistent with the rest of the script. The
patch makes the '/usr/bin/ld:' and 'collect2:' lines go away.

Signed-off-by: Jens Stimpfle <debian@jstimpfle.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-08 22:32:35 +01:00
Jens Stimpfle
d1607dc7df support/scripts/dependencies.sh: improve g++ test to avoid misleading output
When g++ is not installed, a misleading error message turns up because
of a bad combination of an unquoted shell variable and control flow.

 ~/buildroot$ make
 You may have to install 'g++' on your build machine
 /home/testuser/buildroot/support/dependencies/dependencies.sh: 136: [: -lt: unexpected operator

[Thomas:
  - fixed commit log, as per the suggestion of Yann E. Morin.
  - don't change existing empty new lines, suggested by Yann.
  - use positive logic in the newly added test, suggested by Yann.]

Signed-off-by: Jens Stimpfle <debian@jstimpfle.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-08 22:32:21 +01:00
Francois Perrad
18033effad support/script/scancpan: fix corelist check
module could be removed of the core,
so check if the module is currently in the core,
but not if the module was once time included in the core.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-07 23:04:39 +01:00
Yann E. MORIN
dab47af5d9 pkg-download: fix arguments to hash checking script
The argument are correctly used, but incorrectly documented.
Inverse the comments to match the actual usage.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-07 22:08:03 +01:00
Peter Korsgaard
fe1b2ef1d3 Merge branch 'next'
Conflicts:
	Makefile
	package/flac/0001-fix-altivec-logic.patch
	package/grantlee/Config.in

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-12-01 11:16:42 +01:00
Yann E. MORIN
514d7cab61 infra: do not append an empty LD_LIBRARY_PATH
When we set LD_LIBRARY_PATH when building our host tools, we append any
pre-existing value to our custom path:
        LD_LIBRARY_PATH="$(HOST_DIR)/usr/lib:$(LD_LIBRARY_PATH)"

But then if LD_LIBRARY_PATH was previously empty, we end up with an
LD_LIBRARY_PATH that ends with a colon.

Also, when we check that an existing LD_LIBRARY_PATH does not contain
CWD, we previously did not look for a zero-length prefix.

Since 'man ld.so' says of LD_LIBRARY_PATH:
    A colon-separated list of directories in which to search for ELF
    libraries at execution-time. Similar to the PATH environment
    variable.

And POSIX states about PATH:
    A zero-length prefix is a legacy feature that indicates the current
    working directory.

And bash also recognises a zero-length prefix to search in CWD:
    A zero-length (null) directory name in the value of PATH indicates
    the current directory.

We may thus end up on a system where a zero-length prefix in
LD_LIBRARY_PATH is interpreted as CWD.

Do not append the previous LD_LIBRARY_PATH if it was empty, and check
for a zero-length prefix when checking dependencies.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-11-30 23:56:14 +01:00
Francois Perrad
6199f0f823 support/scripts/scancpan: generate hash file
retrieve MD5 and SHA256 from metacpan.org, and store them in the hash
file for each package.

[Thomas: remove the odd indentation of the filename for the md5 hash
lines in the hash file.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-11-22 19:43:00 +01:00
Bjørn Forsman
acc76a6ef9 Change /bin/bash shebangs into /usr/bin/env bash
Not all systems have /bin/bash (e.g. NixOS[1] doesn't). Buildroot
already uses /usr/bin/env shebangs for other interpreters (perl,
python), so why not bash?

This changes only the shebangs used by Buildroot itself; stuff installed
to the target system is left unchanged.

With this applied I can run Buildroot unmodified on NixOS.

[1]: http://nixos.org/

Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-25 01:55:37 +02:00
Fabio Porcedda
9a4e217b73 apply-patches.sh: don't print anything when "make -s" is used
The make "-s" option is used to enable the "Silent operation" so if that
option is used don't print anything as far as there isn't any error.

Add the "-s" option to "apply-patches.sh" to enable silent operation.

[Peter: use the existing QUIET variable]
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-25 01:42:01 +02:00
Jerzy Grzegorek
774ff0d13f package: indentation cleanup
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-25 01:11:26 +02:00
Gustavo Zacarias
2ff8f54e79 scripts/mkusers: allow the creation of groups alone
Let mkusers create groups alone, useful for supplementary permissions in
udev/systemd for example where users can be added to later at runtime.
Use a magic string "-" to signal that user creation should be skipped.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-22 21:39:12 +02:00
Thomas Petazzoni
256db411db pkg-stats: add statistics about hash files
As discussed during the Buildroot meeting, this commit extends the
pkg-stats script to include statistics about the number of packages
having vs. not having the hash file.

As of today, we have 104 packages with the hash file, and 1274
packages without.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-19 17:20:42 +02:00
Luca Ceresoli
9722381c0d scripts/graph-build-time: properly warn about missing modules
Currently the graph-build-time script prints a python exception if a
needed module cannot be imported. Catch the exception and tell the user
which packages are missing, as we do for other missing dependencies.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-12 17:23:06 +02:00
Luca Ceresoli
96183af9af scripts/graph-build-time: clarify backend selection
This instruction in the middle of 'import' lines looks very strange.

Also, it was not obvious to me what the 'Agg' backend is.

Both things are actually correct, but it took a while to find out why.
So clarify with a comment to save someone else's time.

[Peter: fix s/soe/some/ typo]
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Sascha Arthur <sascha.arthur@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-12 17:21:00 +02:00
Thomas De Schampheleire
f268f7131b .mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.

This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.

Alignment of line continuation characters (\) is kept as-is.

The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'

Brief explanation of this command:
    ^\([A-Z0-9a-z_]\+\)     a regular variable at the beginning of the line
    \([?:+]\?=\)            any assignment character =, :=, ?=, +=
    \([^\\]\+\)             any string not containing a line continuation
    \([^\\ \t]\+\s*\\\)     string, optional whitespace, followed by a
                            line continuation character
    \(\s*\\\)               optional whitespace, followed by a line
                            continuation character

Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.

This expression was tested on following test text: (initial tab not
included)

	FOO     = spaces before
	FOO     =   spaces before and after
	FOO	= tab before
	FOO	  = tab and spaces before
	FOO =	tab after
	FOO =	   tab and spaces after
	FOO =   	spaces and tab after
	FOO =    \
	FOO = bar \
	FOO = bar space    \
	FOO   =		   \
	GENIMAGE_DEPENDENCIES   = host-pkgconf libconfuse
	FOO     += spaces before
	FOO     ?=   spaces before and after
	FOO     :=
	FOO     =
	FOO	=
	FOO	  =
	FOO =
	   $(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
	AT91BOOTSTRAP3_DEFCONFIG = \
	AXEL_DISABLE_I18N=--i18n=0

After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
  expression leaves the number of whitespace between the value and line
  continuation character intact, but the whitespace before that could have
  changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
  actually makes the code more readable.

Finally, the end result was manually reviewed.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 15:00:28 +02:00
Francois Perrad
1fd8d9e4dd support/script/scancpan: add -test option
Perl modules can have three different types of dependencies:

 - configure/build time dependency which becomes host dependency
 - runtime dependency which becomes target dependency
 - test time dependency which is useless in a cross-compiling context like BR

Before this patch, test time dependencies are handled like runtime
dependencies.

After this patch, test time dependencies are ignored by default. The
newly added -test option allows to add them anyway if needed.

[Thomas: reword commit log using Francois proposal.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-04 14:42:44 +02:00
Francois Perrad
f7ef4c2ed4 perl: bump to version 5.20.1
[Peter: squash 'add hash' patch into it]
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-02 21:03:31 +02:00
Samuel Martin
47544e43a5 toolchainfile.cmake.in: do not force the CMAKE_{C, CXX}_FLAGS to the cache
Fix #7280 [1]

When the FORCE option is passed to the set command, the variable is
added/updated in the CMake cache every single time CMake processes this
command.

Because the toolchainfile.cmake prepends architecture/toolchain flags
to the CMAKE_{C,CXX}_FLAGS, this makes the CFLAGS being updated in the
generated Makefiles each time one reconfigures its project. So it
forces the compilation of everything, even when nothing has changed.

[1] https://bugs.busybox.net/show_bug.cgi?id=7280

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(tested the SimpleApp reproduction scenario described in the bug report)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-28 20:57:26 +02:00
Samuel Martin
474a0e113e toolchainfile.cmake.in: set linker flags
The linker flags are part of the toolchain configuration, so set them for
the CMake-based packages.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-28 20:52:52 +02:00
Thomas De Schampheleire
53223cbb4e check-kernel-headers: mktemp --tmpdir not available on RedHat RHEL5
mktemp --tmpdir is not available on older Redhat RHEL5 machines. The
alternative that has the same behavior is 'mktemp -t'.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-28 20:40:57 +02:00
Sascha Arthur
28ed63bae2 scripts/graph-build-time: remove X dependency
[Thomas: tweak commit title.]

Signed-off-by: Sascha Arthur <sascha.arthur@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-09-21 22:44:42 +02:00
Peter Korsgaard
5aedb8be68 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-01 15:22:07 +02:00
Francois Perrad
56e7d0b886 support/script/scancpan: don't generate the trailing slash in <PKG>_SITE variable
The trailing slash was removed in all packages, so don't generate it.
See http://git.buildroot.net/buildroot/commit/?id=c7f4b964718bc5a3329bb730f59e24091ecbc7a2

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-08-30 09:01:32 +02:00
Bernd Kuhls
7fa43f911b dependencies: Reject gcj in BR2_NEEDS_HOST_JAVA check
Xbmc uses the Groovy java package to generate API code, for details see
http://wiki.xbmc.org/index.php?title=Codegeneration#How_it_works

Groovy itself is broken when used with gcj:
http://jira.codehaus.org/browse/GROOVY-3701

Xbmc recommends to use another java package instead, like openjdk:
http://forum.xbmc.org/showthread.php?tid=199742

Until we come up with a host package to provide a java setup and
in order to make autobuilder happy let´s reject gcj when we check
for a java installation on the host.

Currently Xbmc is the only package using BR2_NEEDS_HOST_JAVA
so changing the behaviour of dependencies.sh has no side effects.

Fixes
http://autobuild.buildroot.net/results/213/213d7454a1dc68cfb8f4c8d9508f52ebf8e58e33/
http://autobuild.buildroot.net/results/639/639cda872cf36c122cbd6139dd76f6b51ac1864e/
http://autobuild.buildroot.net/results/c9e/c9eb12c95b2939c64590692a82af8d84bbd90956/

Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-17 21:01:10 +02:00
Yann E. MORIN
8f76366c0f support/download: fix the git helper output file format
The git helper uses gzip to compress the intermediate tarball. But gzip
removes the source file, and create a new file named by appending .gz to
the original file name.

Thus, we end up with output.gz, while the download wrapper expects jsut
output, and thus believes the downlaod failed.

Fix that by storing the tar from git to a temporary file, then pipe this
file to gzip's stdin, and redirect gzip's stdout to the output file.

Reported-by: Graham Newton <gnewton@peavey-eu.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-06 19:41:05 +02:00
Peter Seiderer
9d1ab43211 support/download: fix the git helper
Re-add the git_done variable (lost in commit [1]).

Fixes download problem reported by Rohit Kumar [2].

[1] http://git.buildroot.net/buildroot/commit/?id=7e40a1103a919a8177f00ddca2b46b4439953511
[2] http://lists.busybox.net/pipermail/buildroot/2014-August/103733.html

Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-06 19:40:34 +02:00
Yann E. MORIN
9a8470c9ee support/download: convert wget to use the wrapper
This drastically simplifies the wget helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make busybox-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:25 +02:00
Yann E. MORIN
a92290aa20 support/download: convert svn to use the wrapper
This drastically simplifies the svn helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make open2300-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:25 +02:00
Yann E. MORIN
5b3880e174 support/download: convert scp to use the wrapper
This drastically simplifies the scp helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by setting a primary site to 'scp://localhost:/tmp' and
running 'make vim-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:25 +02:00
Yann E. MORIN
93e678b767 support/download: convert Hg to use the wrapper
This drastically simplifies the hg helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make vim-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Yann E. MORIN
7e40a1103a support/download: convert git to use the wrapper
This drastically simplifies the git helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make fmc-fsl-sdk-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Yann E. MORIN
389f50dfb6 support/download: convert cvs to use the wrapper
This drastically simplifies the cvs helper, as it no longer has to deal
with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Yann E. MORIN
84469226c1 support/download: convert localfiles to use the wrapper
This drastically simplifies the localfiles helper, as it no longer has
to deal with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by setting BUSYBOX_SITE = file:///tmp and running 'make busybox-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Yann E. MORIN
c48d45e342 support/download: convert bzr to use the wrapper
This drastically simplifies the bzr helper, as it no longer has to
deal with atomically saving the downloaded archive.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Yann E. MORIN
b5a9470bdf support/download: add download wrapper
The download wrapper is responsible for ensuring the atomicity
of saving into $(BR2_DL_DIR).

It calls the appropriate download helper, telling it to save the
downloaded content to a temporary file in $(BUILD_DIR) (so it does
not clutter $(BR2_DL_DIR) with partial, failed downloads.

Then, only if the download helper was successful, does the wrapper
save the downloaded content to the final location, yet still in a
temporary file, and finally atomically renames it to the final output
file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-04 20:12:24 +02:00
Gustavo Zacarias
8b691cec63 support/libtool: improve version 1.5 static patching
Alter the libtool 1.5.x support patch to accomodate for wildly different
versions of ltmain.sh
Just make it alter incoming args from -static to -all-static which seems
to apply to all the different variants out there since argument parsing
is unlikely to change much.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-31 16:35:49 +02:00
Thomas Petazzoni
97703978ac support/libtool: make -static behave like -all-static
After switching TARGET_LDFLAGS from --static to -static, one issue
appears: from the point of view of libtool, -static only means to link
statically against the 'uninstalled libtool libraries' (i.e the
libraries that libtool has built in the current package), but
otherwise links dynamically with the other libraries. To really get a
completely static build, you need to pass -all-static to
libtool. Unfortunately, -all-static is only a valid option for
libtool, not as a general LDFLAGS, so we cannot to TARGET_LDFLAGS =
-all-static without breaking virtually all packages.

As pointed out 10 years ago on the libtool mailing list, the current
naming of the options is very confusing and the source of issues, and
there was a proposal to change -static to have the behavior of
-all-static, and instead introduce a separate -lt-static to have the
current behavior of -static. But that never got merged, because it was
breaking the current behavior. See:

  http://lists.gnu.org/archive/html/libtool/2004-11/msg00017.html

However, in Buildroot, when we pass -static, we really mean it, and we
want a completely static build. Therefore, this patch adapts our
ltmain.sh patches so that they alter the behavior of -static to make
it work like -all-static. The changes are small and quite easy to
understand, and have been tested to work fine with a small selection
of packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-30 23:27:19 +02:00
Yann E. MORIN
11c1076db9 toolchain: add option to copy the gconv libraries
The gconv libraries are used to translate between different character sets
('charsets', even 'csets' sometimes). Some packages need them to present
text to the user (eg. XBMC Gotham).

In (e)glibc they are implemented by the internal implemenation of iconv,
called gconv, and are provided as dlopen-able libraries.

Note that some gconv modules need extra libraries (shared by more than
one gconv module), so we must, when adding a subset of modules, scan the
installed modules in search of the missing libraries.

[Thomas: add general explanation in expunge-gconv-modules and fix
coding style.]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Cc: Eric Limpens <limpens@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-27 23:15:48 +02:00
Francois Perrad
827ba10035 support/script/scancpan: more license name ajustment
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-24 22:31:46 +02:00
Thomas De Schampheleire
60714bb2ed trivial: fix typo 'informations'
In English, unlike in French, almost all usages of the word 'information'
are uncountable, meaning that 'informations' is invalid.
This patch fixes this typo throughout the tree, except in CHANGES and
docs/news.html (historic text).

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-24 22:31:20 +02:00
Vincent
14c1641c1a dependencies.sh: fix typo
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 18:04:38 +02:00
Yann E. MORIN
1c4498082b support/gen-manual-lists.py: bail out early for legacy symbols
When searching for virtual package providers, there's no need to
handle legacy symbols at all, so just bail out early.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 18:03:26 +02:00
Yann E. MORIN
34363f0df1 support/gen-manual-lists.py: remove unneeded arg in private function
No need to pass as argument to a function, members of the class it's in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 18:02:07 +02:00
Francois Perrad
ed8ab0ce3d support/script/scanpcan: small fix
- typo in comment
- remove trailing space in _HOST_DEPENCENCIES when no dependency

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 11:38:31 +02:00
Francois Perrad
6d3962f0bd support/scripts/scancpan: automatically populate LICENSE_FILES
This commit improves the scancpan script to automatically populate the
LICENSE_FILES variable using informations available in the Perl
package MANIFEST file.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-18 19:45:20 +02:00
Francois Perrad
1c7ab72386 support/script/scancpan: add support for BR2_PREFER_STATIC_LIB
Perl extensions are loaded at runtime with dlopen(), so it does not
make sense to even build extensions that are written in C when
BR2_PREFER_STATIC_LIB is enabled. A Perl module written in C or with a
dependency on a module written in C is not available when doing a
static build.

Therefore, this commit adapts the scancpan script to automatically
generate a dependency on !BR2_PREFER_STATIC_LIB when the Perl module
would not work in a static-only configuration.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-17 20:49:39 +02:00
Karoly Kasza
c9b4b57f84 dependencies.sh: add other Debian package names for 32 bits libraries
[Thomas: implemented Arnout suggestion.]

Signed-off-by: Karoly Kasza <kaszak@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-15 20:58:54 +02:00
Yann E. MORIN
ebe6154ff4 support/download: fix the git helper
When switching the git helper over to a shell script, a special case was
not carried over: in case the remote has the required reference, we
attempt a shallow clone, using --depth 1. However, this is not supported
when the remote is accessed with the http protocol.

Therefore, the download fails.

What happened before the conversion to a shell script was that the helper
in the Makefile would fallback to doing a full-clone.

This is the case and behaviour that were lost in the conversion.

To avoid making the script too complex, we only attempt a full clone if
needed. And we decide that a full clone is needed by default; we decide
it is unnecessary if the remote has the needed reference *and* the
shallow clone was successful.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-08 23:26:48 +02:00
Yann E. MORIN
84d7986940 support/download: fix the bzr helper
bzr uses the name of the extension of the output file to known what
output format to use: tar, tgz, tar.bz2... If no extension is
recognised, bzr will output to a directory.

Since we use 'mktemp .XXXXXX' to generate temporary files, it obviously
never ends with a recognised extension. Thus, bzr expects the output to
be a directory, and fails since it is a file.

Fix that by forcing the output format.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-08 10:53:39 +02:00
Yann E. MORIN
051876936f manual: add documentation about packages' hashes
Although md5 is, for legacy reasons, a supported hash type,
it is not documented on purpose, since it is now known to
be weak.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-04 23:39:26 +02:00
Yann E. MORIN
9bd8b59526 pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...

Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.

In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.

So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.

Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-04 23:35:19 +02:00
Yann E. MORIN
19afad50f5 pkg-infra: don't use DL_DIR as scratchpad for temporary downloads
DL_DIR can be a very precious place for some users: they use it to store
all the downloaded archives to share across all their Buildroot (and
maybe non-Buildroot) builds.

We do not want to trash this location with our temporary downloads (e.g.
git, Hg, svn, cvs repository clones/checkouts, or wget, bzr tep tarballs).

Turns out that we already have some kind of scratchpad, the BUILD_DIR.
Although it is not really a disposable location, that's the best we have
so far.

Also, we create the temporary tarballs with mktemp using the final tarball,
as template, since we want the temporary to be on the same filesystem as
the final location, so the 'mv' is just a plain, atomic rename(2), and we
are not left with a half-copied file as the final location.

Using mktemp ensures all temp file names are unique, so it allows for
parallel downloads from different build dirs at the same time, without
cloberring each downloads.

Note: we're using neither ${TMP} nor ${TMPDIR} since they are shared
locations, sometime with little place (eg. tmpfs), and some of the
repositories we clone/checkout can be very big.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
[tested a particular scenario that used to fail: two separate builds
using a shared DL_DIR, ccache enabled, so that they run almost
synchronously. These would download the same file at the same time,
corrupting each other. With the patches in this series, all works
fine.]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-04 23:28:13 +02:00
Yann E. MORIN
283b8b7fea pkg-infra: move the cp (aka localfiles) download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 00:07:57 +02:00
Yann E. MORIN
54adff55d5 pkg-infra: move the scp download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 00:07:12 +02:00
Yann E. MORIN
45261f1fb7 pkg-infra: move the bzr download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 00:03:59 +02:00
Yann E. MORIN
4bee7629ee pkg-infra: move the wget download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 00:00:15 +02:00
Yann E. MORIN
2fd4b959bd pkg-infra: move the hg download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
[tested a particular scenario that used to fail, when the 'hg archive'
step is interrupted, now working fine]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-02 23:59:10 +02:00
Yann E. MORIN
f4526c053f pkg-infra: move the cvs download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-02 23:54:16 +02:00
Yann E. MORIN
daf034f881 pkg-infra: move the svn download helper to a script
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.

[Peter: redirect pushd/popd output to /dev/null]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-02 23:51:56 +02:00
Yann E. MORIN
95a572282e pkg-infra: move the git download helper to a script
The git download helper is getting a bit more complex. Fixing it in the
Makefile when it breaks (like the recent breakage with a non-existing
sha1-cset) proves to be challenging, to say the least.

Move it into a shell script in support/download/git, which will make
it much easier to read, maintain, fix and enhance in the future.

[Peter: redirect pushd/popd output to /dev/null]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-02 23:47:13 +02:00
Francois Perrad
6b8376d8f0 support/script/cpan: improve Makefile generation
- remove trailing space after perl when it's the only dependency
- license: substitution of perl name by BR name
- add a tabulation before source

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:45:25 +02:00
Vivien Didelot
7cf527426f graph-build-time: support python3
This patch is the result of 2to3.

Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:44:41 +02:00
Arnout Vandecappelle
cfb6bf5ee9 graph-depends: remove unnecessary redirect of stderr
It hides any error messages reported by make.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:44:00 +02:00
Arnout Vandecappelle
b82f66cf40 graph-depends: support calling from recursive make
The graph-depends script calls make. If the outer make was called
recursively, or if it was called with '-C <somedir>', then the
environment will contain "MAKEFLAGS=w --". Therefore, the recursive
make prints 'Entering' and 'Leaving' messages, which clobbers the
output for dot.

To avoid this, add "--no-print-directory" to the recursive make
arguments. Since we require GNU make 3.81, we can be sure that this
option is available.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:43:37 +02:00
Francois Perrad
0bf13117aa support/script/scancpan: populates Config.in with homepage
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-22 19:25:43 +02:00
Arnout Vandecappelle
63454b1c07 graph-depends.py: support python3
This patch is the result of 2to3.

In addition, universal_newlines=True is added to the Popen calls. In
python3, this makes sure that the output is decoded so that we get a
string instead of a buffer object.

Cc: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-22 19:05:18 +02:00
Maxime Hadjinlian
8f9da282b2 classpath: Use generic check for host program
Remove the specific check that was done in dependencies.sh to use the
generic one that were introduced by the previous patch.

Also, introduce, BR2_NEEDS_HOST_JAVAC and BR2_NEEDS_HOST_JAR as it is
needed by classpath.

Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-14 19:55:28 +02:00
Maxime Hadjinlian
8575332f0d infra: Add generic check_prog_host function
Avoid copy/pasting the same block of code to check if a program is
available on the host machine.

Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-14 19:54:26 +02:00
Yann E. MORIN
4cce4ac990 graphs/depends: do not draw transitive dependencies by default
The transitive dependencies make the graphs barely readable for large
configs, with a large number of packages.

So, just switch to not drawing the transitive dependencies by default.

By popular demand... ;-)

[Peter: reword]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc; Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-13 14:39:48 +02:00
Yann E. MORIN
3fbd9887b3 filesystems: also chown symlinks
Currently, the symlinks in the generated filesystems will have the
UID of the user running the build, because 'chown' does not change
the ownership of symlinks, by default.

Although the implications are limited, some may not want that UID
to leak in the generated filesystems.

So, use 'chown -h' so even symlinks get properly chowned.

Reported-by: Angelo Dureghello <angelo@barix.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-09 11:28:57 +02:00
Yann E. MORIN
4d922634da support/scripts: generate a list of virtual packages
Generate an asciidoc table that can be included in the manual, that
lists the existing virtual packages, the corresponding symbols, and
their providers (and sub-options thereof).

The core of this change is the addition of a new formatter for virtual
packages. This formatter is a bit tricky, as it has to catter for a
bunch of corner cases:
  - provider is not a package, but is sub-options of a package
  - such a sub-option may be itself 'select'-ed by one or more
    other sub-options
  - legacy packages should not be considered as a provider

Those cases are real:
  - sub-options of mesa3d provide EGL or GLES
  - selected sub-options of mesa3d provide GL
  - udev is a legacy package, but it provides udev

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:11:05 +02:00
Yann E. MORIN
fe696d7c65 support/scripts: introduce a symbol formatter to generate package lists
Currently, we can generate two different tables of packages:
  - a single-column table with the symbols' prompts,
  - a two-column table with the symbols' prompts and locations in the
    menuconfig.

For virtual packages, this is not enough, since we will have to display
more columns, with different content:
  - the virtual package name (but such symbols do not have a prompt)
  - the symbol name
  - the providers for the virtual package

So, instead of having a single function that knows how to generate any
table, introduce a formatter function that is passed as argument to,
and called by format_asciidoc_table(). Such formatter functions are
responsible for providing:
  - the layout of the table (number of columns, column arrangement),
  - the formatted header line,
  - a formatted line for a symbol.

What the formatter should ouput depends on its arguments:
  - if none are passed, the layout is returned,
  - if the header label is passed, it returns the formatted header line,
  - otherwise, it returns the formatted line for a symbol.

Two formatter functions are introduced in this changeset, to replace the
current 'sub_menu' feature:
  - _format_symbol_prompt() to display a one-column table with only the
    symbols' prompts,
  - _format_symbol_prompt_location() to display a two-column table with
    the symbols' prompts and locations.

This will help us to later introduce a new formatter to generate a table
for virtual packages.

[Thanks to Samuel for his pythonistic help!]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:10:32 +02:00
Yann E. MORIN
b2c75ecc5d support/scripts: who's responsible to decide what is a package
When generating the package lists, the responsibility to decide what is
actually a package symbol is currently split between the _is_package(),
the get_symbol_subset() and the format_asciidoc_table() functions.

The two latter functions check that an item is really a symbol, and that
is has a prompt.

While this is currently correct for real packages, this will no longer
be the case when we also generate a list of virtual packages, since they
do not have a prompt.

Move the responsibility to verify that a symbol is indeed a package symbol
to _is_package(), so it's all in one place, and makes it easier to change
for virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:09:57 +02:00
Yann E. MORIN
f93edd823f support/scripts: do not display virtual packages in generated lists
If a package has both a 'real' and a 'virtual' definition, consider it
is a virtual package and do not display it in the generated package list.

This is the case for jpeg and cryptodev, that are virtual packages, but
also real (but empty) packages used to provide a prompt to enable/disable
a choice to select an implementation. In this case, we do not want to
list the virtual packages, but only their implementations.

So, consider packages that are both real and virtual as virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:09:38 +02:00
Yann E. MORIN
e49d4f0c37 support/scripts: prepare handling virtual packages in generated lists
Prepare to tell apart real packages from virtual packages.

Currently, the code implicitly recognises only real packages, and
discards virtual packages, because of the heuristic used to recognise
whether a symbol is a package:

  - for real package:
    - symbols  : BR2_PACKAGE_FOO
    - .mk files: foo.mk
  - for virtual packages:
    - symbols  : BR2_PACKAGE_HAS_FOO
    - .mk files: foo.mk

The current heuristic is to check for each symbol if a corresponding .mk
file exists, by stripping 'BR2_PACKAGE_' from the beginning of the symbol,
converting the result to lowercase, and checking if a .mk file exists.

So, as a side effect, it completely misses the virtual packages [*], which
is pretty nice since we get a list with only real packages that the user
can indeed select and see in the menuconfig.

[*] Except for 'cryptodev' and 'jpeg' which are both virtual packages and
normal packages. Except they are not normal packages, they are used to
display a choice of the implementation to use. This case will be fixed in
follow-up patches.

Since we'll soon need to also output the table of virtual packages, we
need to teach the _is_package() function to recognise them as well.

This patch is the first step into that direction: it introduces a new
function _is_real_package() that is just a wrapper to _is_package(), which
gains a new parameter, being the type of packages to filter on.

No behavioural change is made in this patch, it is just a preparatory
patch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:08:36 +02:00
Yann E. MORIN
6cffe52b77 support/scripts: document args to _is_package() in gen-manual-lists
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:08:12 +02:00
Yann E. MORIN
d3fa2bebc7 support/scripts: prepare expanding the packages lists
Move to a function the code generating the package name from a
symbol's name, to avoid code duplication.

This is not used currently, but will be in a subsequent patch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:08:01 +02:00
Yann E. MORIN
47aeebd956 graph-depends: rename the mode constants
This is ugly, since Python does not have enum constructs, so by moving
the 'type' of the constant ('MODE' here) to the beginning, we get an
artificial 'namespace' for the constants.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:27 +02:00
Yann E. MORIN
4a2db6d193 graph-depends: document the 'transitive' variable
Although unnecessary (we already have initialisation via the parser),
initialise the 'transitive' option, and document it at the same time.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:22 +02:00
Yann E. MORIN
61dbf4d306 graph-depends: add option to set the colours
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Francois Perrad <fperrad@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:17 +02:00
Yann E. MORIN
e347bee02c graph-depends: handle the depth argument in a more pythonic way
Add some comment as well, enhance help text.

[thanks to Samuel for the hints to make it even more pythonic]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:13 +02:00
Yann E. MORIN
95ab905ba7 graphs-depends: merge redundant-dependencies elimination
Merge the redundant-dependencies elimination into the newly introduced
transitive-dependencies elimination.

This makes the code cleaner and much shorter, because:

  - the ('all',pkg) redundant dependency is in fact a transitive
    dependency, and we now have code to deal with that

  - the (pkg,'toolchain') dependency is easy enough to deal with that
    having a separate function for that is overkill

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:08 +02:00
Yann E. MORIN
67957967a5 graphs: add option to remove transitive dependencies in dependency graph
Currently, all the dependencies of a package are drawn on the dependency
graph, including transitive dependencies (e.g. A->B->C and A->C).

For very big graphs, with lots of packages with lots of dependencies, the
dependency graph can be very dense, and transitive dependencies are
cluttering the graph.

In some cases, only getting the "build-order" dependencies is enough (e.g.
to see what impact a package rebuild would have).

Add a new environment variable to disable drawing transitive dependencies.

Basically, it would turn this graph:

    pkg1 ---> pkg2 ---> pkg3 -------------------.
         |\__________/                 \         \
         |\____________________         \         \
         |                     \         \         \
          `-> pkg4 ---> pkg5 ---> pkg6 ---> pkg7 ---> pkg8
                    \__________/

into that graph:

    pkg1 ---> pkg2 ---> pkg3 -----------.
         |                               \
          `-> pkg4 ---> pkg5 ---> pkg6 ---> pkg7 ---> pkg8

[Thanks to Samuel for the parser hints]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:43:04 +02:00
Francois Perrad
b689264d38 graph-depends: refactor with more colors
Do not use the same colors for toolchain, host and target packages.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr rephrase commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 16:42:58 +02:00
Samuel Martin
c8f2d248f6 pkg-cmake.mk: enable ccache for cmake packages
This patch updates the generated toolchainfile.cmake to use ccache.

When toolchainfile.cmake is used inside Buildroot, using ccache during
the build is driven by a CMake knob: USE_CCACHE, automatically set by
the cmake-package infrastructure and reflecting the BR2_CCACHE value.

Since this toolchainefile.cmake file can be used outside Buildroot, and
this file also set a couple of things (among these: the sysroot cflag,
some pkg-config environment variables), it is important to set the
compiler variables as well to keep the consistency of the
cross-compilation configuration.
So, when it is used outside Buildroot, using ccache for the build is
driven by the ccache program availability.

Note that using ccache for the build is achieved by setting the *_ARG1
CMake variables to let CMake use ccache without failing in detecting
the compiler.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-01 22:16:24 +02:00
Samuel Martin
9033d90118 pkg-cmake.mk: do not hardcode absolute path in toolchainfile.cmake
The patch allows sharing or moving the toolchains.

This is a step toward making the toolchain/sdk relocatable.

Closes #6818

[Peter: reword comment as suggested by Thomas]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Uwe Strempel <u.strempel@googlemail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-01 22:15:43 +02:00
Samuel Martin
85fea5d953 pkg-cmake.mk: refactor the toolchainfile.cmake generation
This patch introduces a toolchainfile.cmake.in template which is filled
by Buildroot.

Using a toolchainfile.cmake.in template file allows to avoid overloading
quoting and/or escaping and it becomes much more similar to the
resulting file.

This patch also cleans up the quoting style.

[Peter: drop stdin redirect as suggested by Thomas]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-01 22:11:38 +02:00
Peter Korsgaard
27a5414804 Merge branch 'next'
Conflicts:
	package/gdb/Config.in.host

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-01 09:58:54 +02:00
Heiko Abraham
fc90fa9417 Improve tar check if bsdtar is installed
If bsdtar is installed, fix script error for tar version detection.

bsdtar does not provide all expected command line (long) options
like "--hard-dereference". To ensure compatibility, mark version
of tar as 'invalid' and trigger build of 'host-tar'.

[Peter; slightly reworded commit text]
Signed-off-by: Heiko Abraham <abrahamh@web.de>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-31 09:39:40 +02:00
Francois Perrad
9c2be26ed8 scancpan: sort dependencies
in order to avoid spurious diff when updating packages

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-18 23:54:09 +02:00
Fabio Porcedda
f3f3de0966 pkg-stats: fix whitespaces
Remove some spaces before tabs and add the empty line at end of file.

Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Cc: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-15 23:38:20 +02:00
Fabio Porcedda
a9463d24d4 pkg-stats: add luarocks perl python virtual infrastructures
Recognize the recently added infrastructures: luarocks, perl, python,
virtual.

Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Cc: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-15 23:37:55 +02:00
Yann E. MORIN
38b81eb172 support: properly check bash is available
The current check is broken, as it only checks if the user's login
shell is bash, not what the system shell is.

Mimick the sequence found in the top-level Makefile to search for
bash, except for the fallback case, where we explicitly check that
'sh' is bash, by checking if it sets $BASH, so we know the fallback
case, in the top-level Makefile, to use 'sh' will indeed use bash.

Remove superfluous semi-colons ';' at the end of lines, they are
not needed in a shell script (this is not C!)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Andrew Barnes <andy@outsideglobe.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-20 11:13:36 +02:00
Yann E. MORIN
a89f14e0f1 support/graphs: fix comparisons agains None
As Samuel said:
    In Python, None is a singleton, and it is recommended to use "is" or
    "is not" for testing them [1].

    [1] http://legacy.python.org/dev/peps/pep-0008/#programming-recommendations

Reported-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-14 22:50:24 +02:00
Yann E. MORIN
ef8b8d950f graphs: make graphs with lots of packages nicer to look at
Some magic numbers obtained with trial-and-error and successive
iterations, to eventually get a nice graph.

[Thomas: remove excessive spaces in expressions.]

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-14 22:13:04 +02:00
Yann E. MORIN
ede3e5fec3 support/graph-depends: don't show toolchain dependency for all packages
Al packages depend on 'toolchain'. Currently, 'graph-depends' graphs this
dependency. The resulting graph is thus cluttered with less-than-useful
information.

Instead, do not graph the 'toolchain' dependency for any package, save
for the fake 'all' package. The graph is now a bit more readable.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-14 21:00:38 +02:00
Yann E. MORIN
12c5e68ea8 support/graph-depends: add option to limit the depth of the graph
Currently, the complete dependency chain of a package is used to
generate the dependency graph. When this dependency chain is long,
the generated graph becomes almost unreadable.

However, it is often sufficient to get the first few levels of
dependency of a package.

Add a new variable BR2_GRAPH_DEPTH, that the user can set to limit
the depth of the dependency list.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-14 20:56:20 +02:00
Yann E. MORIN
3d37950ec1 support/graph-depends: use argparse to parse argv[]
Currently, we are using a crude, ad-hoc parsing of argv[].
This is a limiting factor to adding new options.

Use argparse instead, and introduce a single argument for now:
  --package, -p PACKAGE

In the (near) future, we'll be able to add more option arguments,
such as depth-limiting for big graphs.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-14 20:56:18 +02:00
Yann E. MORIN
2a82bb8a90 support/check-kernel-headers: fix old custom toolchains without -print-sysroot
Old toolchains, with old gcc that do not support -print-sysroot, break the
kernel-headers version check script: it fails to find the sysroot of the
toolchain, and thus ends up including the host's linux/version.h.

Most of the time, this will break early, since the host's kernel headers
will not match the toolchain settings.

But it can happen that the check is succesful, although the configuration
of the toolchain is wrong:

  - the custom toolchain has kernel headers vX.Y
  - the user selected vX.Z  (Z!=Y)
  - the host has headers vX.Y

In this case, the check passes OK, but the build of some packages later on
will break (which is exactly what those _AT_LEAST_XXX options were added to
avoid).

Fix that by passing the sysroot to the check script, instead of the cross
compiler.

We get the sysroot as thus:

  - for custom toolchains, we use the macro toolchain_find_sysroot. We can
    do that, because we already have a complete sysroot with libc.a at that
    time.

  - for internal toolchain using a custom kernel headers version, we just
    use $(STAGING_DIR). We can't use the macro as for custom toolchains
    above, because at the time we install the kernel headers, we do not yet
    have a complete sysroot with a libc.a.  But we can just use
    $(STAGING_DIR), since we're only interested in the kernel headers.

For all other types of toolchains, we already have the _AT_LEAST_XXX options
properly set, so we need not add a check in this case.

Fixes:
    http://autobuild.buildroot.net/results/f33/f331a6eff0b0b93c73af52db3a6b43e4e598577e/
    http://autobuild.buildroot.net/results/a57/a5797c025bec50c10efdcff74945aab4021d05e4/
    [...]

[Thanks to Thomas for pointing out the toolchain_find_sysroot macro!]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-04-09 01:38:10 +02:00
Yann E. MORIN
78b8621865 support/check-kernel-headers: use mktemp instead of hand-crafted temp file
Instead of creating a temporary files with a dubious scheme, use mktemp,
which purpose is exactly that: creating temporary files

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-06 17:12:04 +02:00
Yann E. MORIN
0aa9019c2d toolchain: print actual version of kernel headers when checking
Since we introduced the _AT_LEAST_XXX for the kernel headers, people
using pre-built custom toolchain now have to specify the version of
the kernel headers their custom toolchain uses.

So, when we detect that there is a mismatch between the selection in
the menuconfig, and the actual version of the headers, we currently
only bail out with a terse message "Incorrect selection of kernel
headers".

This could be confusing some, and getting the version of the headers
used by the toolchain is not trivial (well, it's very easy, but not
trivial.)

This patch changes the way we report the error by moving the message
into the test-code, and by printing the expected and actual versions
of the kernel headers.

BUT! To get this pretty error message, we need to run the
test-program, so we can not use the cross-toolchain, we have to use
the native one.

BUT! The native one has its own linux/version.h header, so we can not
simply include it.

So, we ask the cross-compiler where its default sysroot is, and use
that to then force-feed the cross linux/version.h to the native
toolchain.

[Thomas: augment commit log with a message provided by Yann, fix
coding style to not have spaces after opening parenthesis and before
closing parenthesis, reformatted the message "Incorrect selection..."
to make it fit on one line.]

Reported-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-06 14:26:34 +02:00
Yann E. MORIN
4955a31cec support/mkusers: fix warnings when adding a new user
When adding a new user (or a new group), we would get warnings, like:
  [...]/support/scripts/mkusers: line 145: [: too many arguments

This is because we're checking if a UID (or a GID) is already defined,
and/or is different from the requested one, both checks in the same
test.

Of course, if a UID (or a GID) is not defined, it does not have a value,
so we can not compare it to an integer.

Fix that by splitting the test in two, so the second is only executed if
the first is sucessful.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-05 18:39:16 +02:00
Francois Perrad
0f33003a54 script/scancpan: add -host & -target options
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-04 23:31:06 +02:00