Commit Graph

351 Commits

Author SHA1 Message Date
Ricardo Martincoski
493a86a7b7 size-stats: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
253e016616 pycompile: fix code style
Fix these warnings:
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
cd04833926 graph-build-time: fix code style
Fix these warnings:
E201 whitespace after '['
E202 whitespace before ']'
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Ignore these warnings:
E402 module level import not at top of file

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
236bede631 check-uniq-files: fix code style
Fix these warnings:
E128 continuation line under-indented for visual indent
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Ricardo Martincoski
9ff44852c4 graph-depends: fix code style
Fix these warnings:
E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
E202 whitespace before ']'
E221 multiple spaces before operator
E225 missing whitespace around operator
E231 missing whitespace after ','
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E502 the backslash is redundant between brackets
E713 test for membership should be 'not in'

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Thomas De Schampheleire
a3f37c53d5 support/scripts: sort pie charts
Make sure that the pie charts produced by 'graph-build' and 'graph-size'
targets are sorted on the size of each piece of the pie. Otherwise, making
visual analysis is difficult, as one needs to look at the legends of each
piece and do the sorting manually in their head.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-02 22:40:09 +01:00
Luca Ceresoli
6887ad7003 support/scripts/genimage.sh: show usage when invoked incorrectly
[Peter: use ${0}]
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-12-11 20:14:39 +01:00
Thomas Petazzoni
1c8dda3e43 Merge branch 'next'
This merges the next branch accumulated during the 2017.11 release
cycle back into the master branch.

A few conflicts had to be resolved:

 - In the DEVELOPERS file, because Fabrice Fontaine was added as a
   developer for libupnp in master, and for libupnp18 in
   next. Resolution is simple: add him for both.

 - linux/Config.in, because we updated the 4.13.x release used by
   default in master, while we moved to 4.14 in next. Resolution: use
   4.14.

 - package/libupnp/libupnp.hash: a hash for the license file was added
   in master, while the package was bumped into next. Resolution: keep
   the hash for the license file, and keep the hash for the newest
   version of libupnp.

 - package/linux-headers/Config.in.host: default version of the kernel
   headers for 4.13 was bumped to the latest 4.13.x in master, but was
   changed to 4.14 in next. Resolution: use 4.14.

 - package/samba4/: samba was bumped to 4.6.11 in master for security
   reasons, but was bumped to 4.7.3 in next. Resolution: keep 4.7.3.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-01 21:56:44 +01:00
Andrey Yurovsky
88af7d330d support/scripts/size-stats: avoid divide-by-zero
Some packages (ex: skeleton-init-systemd) have a zero size so we cannot
divide by the package size. In that case make their percent zero
explicitly and avoid a ZeroDivisionError exception.

Signed-off-by: Andrey Yurovsky <yurovsky@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-29 21:36:34 +01:00
Yann E. MORIN
1b8ad2d08e core: check files are not touched by more than one package
Currently, we do nothing about packages that touch the same file: given
a specific configuration, the result is reproducible (even though it
might not be what the user expected) because the build order is
guaranteed.

However, when we later introduce top-level parallel build, we will no
longer be able to guarantee a build order, by the mere way of it being
parallel. Reconciliating all those modified files will be impossible to
do automatically. The only way will be to refuse such situations.

As a preliminary step, introduce a helper script that detects files that
are being moified by two or more packages, and reports them and the
impacted packages, at the end of the build.

The list being reported at the end of the build will make it prominently
visible in autobuilder results, so we can assess the problem, if any.

Later on, calling that helper script can be done right after the package
installation step, to bail out early.

Thanks Arnout for the pythonist way to write default dictionaries! ;-)

Note: doing it in python rather than a shell script is impressively
faster: where the shell script takes ~1.2s on a minimalist build, the
python script only takes ~0.015s, that is about 80 times faster.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Peter Seiderer <ps.report@gmx.net>
[Thomas: rename script without .py extension.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-27 23:02:19 +01:00
Yann E. MORIN
c1cfee290e support/check-rpath: recognise PIE
We sanity-check the host executables that they have a correct RPATH
pointing to the host libraries.

This is currently done by looking for all files in $(HOST_DIR) that
match the 'ELF executable' pattern (a bit more complex, but that's
idea).

However, when an executable is built with -fPIE of -fpie, it no longer
appears to be an 'ELF executable', but it rather looks like an 'ELF
sheard object' (like if it were an library.

So, we miss those files.

It turns out that the problem is a real one, because quite a few
mainline distros, expecially those based on Debian for example, have
already switched to generating PIE code by default, and thus we miss on
a whole class of systems..

We fix that by simply looking if we can find an ELF interpreter in each
file. If we there is one, this is an ELF executable; if not, it may be
anything else: we don't care (not even about ELF libraries).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-09-27 00:13:55 +02:00
Thomas Petazzoni
85f11d08c3 support/scripts/apply-patches.sh: do not apply patches with renames
Patches with renames apply properly with patch >= 2.7, but not with
older patch versions. Since "git format-patch" by default generates
patches with renames, Buildroot developers often don't realize that
their patches will not apply properly on build machines that have
patch < 2.7. In order to prevent such a situation from happening
again, this commit adds some logic in apply-patches.sh to refuse
applying patches that contain renames.

Note that just searching for '^rename' is not sufficient, since the
patch commit message may contain the words "rename from" or "rename to"
as well. Therefore, the grep expression is made as accurate as possible,
checking both.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Arnout: spaces instead of tabs (suggested by Yann);
 extend commit message.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-09-19 22:51:25 +02:00
Thomas Petazzoni
cb0f3fa13f support/scripts/check-bin-arch: fix symbolic link check
Commit c96b8675ea
("support/scripts/check-bin-arch: ignore symbolic links") was bogus,
because it tested ${f}, which is the relative path of the file inside
${TARGET_DIR}, so we end up testing if ${f} on the system is a
symbolic link.

This commit fixes that by testing ${TARGET_DIR}/${f}.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-09-05 20:59:04 +02:00
Thomas Petazzoni
c96b8675ea support/scripts/check-bin-arch: ignore symbolic links
Since commit da32b49f00
("instrumentation: extend packages-file-list.txt with symlinks and
directories"), the packages-file-list.txt also contains symbolic
links. Therefore, check-bin-arch is now also checking symbolic links.

However, symbolic links in $(TARGET_DIR) can have absolute path as
targets, such as:

$ ls -l output/target/sbin/ifdown
lrwxrwxrwx 1 thomas thomas 10 Sep  3 15:55 output/target/sbin/ifdown -> /sbin/ifup

Therefore, we are now potentially checking a host binary, which
obviously makes check-bin-arch fail.

This commit changes check-bin-arch to ignore symbolic links. Indeed,
we have two cases:

 - The symbolic link really points to something that will in the
   rootfs (such as /sbin/ifup above). In this case, /sbin/ifup will be
   checked separately by check-bin-arch.

 - The symbolic link doesn't point to something that will be in the
   rootfs, and that is not a problem from the perspective of
   check-bin-arch, which checks the architecture of target binaries.

Fixes:

  http://autobuild.buildroot.net/results/16d384a0183d477646ac7692feb65f00dde7d068/
  (vim)

  http://autobuild.buildroot.net/results/50429c0f63a8befff9e20899327b9a8d754d99be/
  (ifupdown)

  http://autobuild.buildroot.net/results/1db65973e782bfa61abcbccd3501bfd235f77288/
  (gawk)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-09-04 16:01:51 +02:00
Wolfgang Grandegger
0bc81f528b support/scripts/fix-rpath: typo fix and unneeded "; " removed
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 14:38:43 +02:00
Wolfgang Grandegger
17dd0d49da support/scripts/fix-rpath: check if the patchelf utility is executable
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 14:35:55 +02:00
Samuel Martin
022b8adbd6 support/scripts: add fix-rpath script to sanitize the rpath
This commit introduces the script "fix-rpath" able to scan a tree,
detect ELF files, check their RPATH and fix it in a proper way.
The RPATH fixup is done by the patchelf utility using the option
"--make-rpath-relative <root-directory>".

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-20 22:47:04 +02:00
Thomas Petazzoni
67409d9303 pkg-stats: add exception for nvidia-tegra23.mk
This file is not a package per-se, it includes other .mk files that
are packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-06 21:46:04 +02:00
Thomas Petazzoni
df3bf7bccd pkg-stats: take into account waf packages
The waf package infrastructure was not known by the pkg-stats script,
so let's add it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-06 21:45:41 +02:00
Ricardo Martincoski
2f762fb620 pkg-stats: allow to sort by column
With 2000+ packages it's not trivial to identify i.e.:
- all packages that don't have a hash file;
- all packages that have patches;
- all packages that have code style warnings;

User experience can be improved by dynamically sorting the resulting
table.

There is an open-source solution that does that in the client-side and
requires minimal changes to our script: sorttable.js.  The script is
MIT licensed as stated in its website.

Also add a hint to the user that the table can be sorted.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-06 21:38:43 +02:00
Arnout Vandecappelle
1d11d6314c check-host-rpath: no longer allow $(HOST_DIR)/usr
Now all packages have been updated to install things in $(HOST_DIR)/lib
instead of $(HOST_DIR)/usr/lib, there should no longer be any reason
to have $(HOST_DIR)/usr/lib in the RPATH, so we don't allow it any more.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Wolfgang Grandegger <wg@grandegger.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 16:54:12 +02:00
Arnout Vandecappelle
4db7d1f355 eclipse-register-toolchain: toolchain is no longer installed in $(HOST_DIR)/usr
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 16:06:27 +02:00
Arnout Vandecappelle
2295780a95 check-host-rpath: no longer check $(HOST_DIR)/usr/{bin, sbin}
Since $(HOST_DIR)/usr/{bin,sbin} are now symlinks to
$(HOST_DIR)/{bin,sbin}, it makes no sense to check them - they are
already covered.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:45:58 +02:00
Arnout Vandecappelle
4c790b8864 Move $(HOST_DIR)/usr/lib to $(HOST_DIR)/lib
This is a step towards eliminating $(HOST_DIR)/usr. It allows us to
convert all packages installing things into $(HOST_DIR)/usr/lib without
affecting the rest.

To allow compatibility with packages that still use $(HOST_DIR)/usr as
the prefix, create a symlink from usr/lib to ../lib.

Note that the symlink creation will break when $(HOST_DIR)/usr/lib
already exists as a directory, i.e. when rebuilding in an existing
output directory. This is necessary: if we don't break it now, the
following commits (which remove the usr part from various variables)
_will_ break it.

At the same time as creating this symlink, we also have to update the
check-host-rpath script to accept both $(HOST_DIR)/usr/lib and
$(HOST_DIR)/lib, because depending on how the package derives the
path, it may be different.

Since there are some dependency chains that involve $(STAGING_DIR),
$(STAGING_DIR) may in fact be created before $(HOST_DIR). Since
$(STAGING_DIR) is a subdirectory of $(HOST_DIR), it is possible that the
newly added rule for $(HOST_DIR) never triggers. To make sure that the
rule does trigger, add an order-only dependency from $(STAGING_DIR) to
$(HOST_DIR).

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:39:58 +02:00
Thomas De Schampheleire
ec019bcf64 setlocalversion: fix detection of hg revision for untagged versions
By default, cut prints the entire line if the specified delimiter is not
present at all:

    $ printf "foo bar" | cut -d' ' -f2
    bar
    $ printf "foobar" | cut -d' ' -f2
    foobar

In setlocalversion, cut is presented with the output of 'hg id' which has
the format:

    "<revision> <tags-if-any>"

If the current revision is not tagged, the output of 'hg id' does not
contain the delimiter (space), cut prints the entire string, and
setlocalversion thinks the version is the tag.
As setlocalversion does not print anything for tagged versions, there is no
output overall, and no correct indication of the mercurial revision.

Fix by passing the extra cut option '--only-delimited', which suppresses
output if no delimiter is found.

This problem likely went unnoticed for so long, because the tag 'tip' (i.e.
most recent revision of the branch) is treated specially: in this case the
mercurial revision _is_ printed, i.e. the situation is treated as
'untagged'.
The problem is only seen when you are _not_ at the most recent revision in
your branch.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-07-03 11:40:49 +02:00
Wolfgang Grandegger
824f3d9beb support/scripts: check-host-rpath now handles $ORIGIN as well
"$ORIGIN/../../usr/lib" is also a valid RPATH for binaries in
"$hostdir/usr/bin". After RPATH sanitation, all RPATH
directories start with "$ORIGIN".

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-02 15:49:25 +02:00
Thomas Petazzoni
7ca9fc3170 tools: rename to 'utils'
After some discussion, we found out that "tools" has the four first
letters identical to the "toolchain" subfolder, which makes it a bit
unpractical with tab-completion. So, this commit renames "tools" to
"utils", which is more tab-completion-friendly.

This has been discussed with Arnout and Yann.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 18:07:00 +02:00
Yann E. MORIN
cb4184d905 tools: move scanpypi out of support/scripts/
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:56:03 +02:00
Yann E. MORIN
bb5a6878c1 tools: move scancpan out of support/scripts/
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: "François Perrad" <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:55:18 +02:00
Yann E. MORIN
b1cbf3b9f9 tools: move size-stat-compare out of support/scripts/
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:54:29 +02:00
Yann E. MORIN
e518b38936 tools: move check-package out of support/scripts/
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:54:19 +02:00
Yann E. MORIN
27b7bd6ad0 tools: move get-developers out of support/scripts/
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:53:04 +02:00
Yann E. MORIN
afff1ce5b0 tools: move test-pkg out of support/scripts
Move it to the top-level tools/ directory, so that it is easier to
find for users.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 17:50:52 +02:00
Jens Maus
82f64203a8 support/scripts: allow /etc/shadow to be symlinked
This commit fixes a problem where it was not possible to replace
/etc/shadow with a symlink to a e.g. a user partition where the
shadow file is placed. This is required, e.g. for systems where the
rootfs is mounted read-only but users should still be able to be
added. Thus, if within an filesystem overlay setup a user tries
to replace /etc/shadow with a symlink to the real file on a user
partition a buildroot build stops with an error message because
sed is called on the symlink instead of following the symlink.
This commit fixes this shortcoming.

Signed-off-by: Jens Maus <mail@jens-maus.de>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-24 17:49:48 +02:00
Bernd Kuhls
9eebeb4482 scanpypi: fix comment typo
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-06-17 13:15:05 +02:00
Francois Perrad
c3839e048e scancpan: refactor with $mirror
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:32:01 +02:00
Francois Perrad
e27c0b5acb scancpan: refactor with MetaCPAN::API source()
instead of get_manifest()

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:31:50 +02:00
Francois Perrad
8710996bb7 scancpan: update with MetaCPAN API v1
The API v0 is shutdown.
see https://bugs.busybox.net/show_bug.cgi?id=9951

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-15 22:31:44 +02:00
Francois Perrad
9cbe364f57 support/scripts: fix shebang with bash
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-06-08 21:43:05 +02:00
Francois Perrad
ed0ce49537 support/scripts: fix shebang with python
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-06-07 21:55:10 +02:00
Abhimanyu Vishwakarma
e73cf8a228 genimage.sh: fix calling from BR2_ROOTFS_POST_IMAGE_SCRIPT
When called from BR2_ROOTFS_POST_IMAGE_SCRIPT, this script
ends up with following error:

Error: Missing argument

This is because, an extra positional argument is also passed
along with BR2_ROOTFS_POST_SCRIPT_ARGS. genimage.sh didn't
have support to parse positional and optional arguments
together.

Signed-off-by: Abhimanyu Vishwakarma <Abhimanyu.V@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-04-27 22:55:52 +02:00
Ricardo Martincoski
7b394c4926 check-package: move parts to subdirectory
Currently the check-package script uses many files in the same
directory. This commit keeps the main script in support/scripts/ and
moves the rest into a subdirectory.

The modules were previously prefixed to make it easy to identify which
script they belong to. This is no longer needed when using a
subdirectory, so the prefix is removed.

Note: if this commit is checked out and the script is run, and later on
a previous version is checked out, the file
support/scripts/checkpackagelib/__init__.pyc needs to be manually
removed to prevent Python interpreter to look for checkpackagelib
package when only the checkpackagelib module is available.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-20 22:29:41 +02:00
Yann E. MORIN
37308b979a support/test-pkg: run legal-info
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-17 15:54:33 +02:00
Arnout Vandecappelle
92b10f63c5 support/test-pkg: calculate toolchain name only once
The toolchain name was calculated in main() for reporting to the user,
and again in build_one() for creating the build directory. Calculate
it only once, in main(), and pass the build directory as an argument
to build_one().

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-17 15:54:22 +02:00
Thomas Petazzoni
c0274ae8b5 pkg-stats: also parse toolchain packages
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-09 09:57:44 +02:00
Thomas Petazzoni
33b78b2ecf pkg-stats: ignore additional package infra .mk files
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-09 09:45:10 +02:00
Thomas Petazzoni
b1e102d1ab pkg-stats: add new column with check-package warnings
This allows the page at http://autobuild.buildroot.net/stats/ to show
how many warnings returned by check-package affect each package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 21:35:33 +02:00
Ricardo Martincoski
40ef732490 check-package: check *.mk for typo in variable
Warn when a variable is defined in a .mk file and it don't start with
the package name.

This function generates false warnings and the maintenance of the
whitelist can be an extra burden, but it catches some typos really hard
to see:
- POPLER_CONF_OPTS [1]
- BALELD_LICENSE [2]
- DRDB_UTILS_DEPENDENCIES [3]
- PERL_LIBWWW_LICENSE_FILES [4]
- AVRDUDR_LICENSE_FILES [5]
- GST1_PLUGINS_ULGY_HAS_GPL_LICENSE [6]
- ON2_8170_LICENSE [7]
- LIBFDTI_CONF_OPTS [8][9]
- IPSEC_DEPENDENCIES [10]

[1] http://patchwork.ozlabs.org/patch/681533
[2] http://patchwork.ozlabs.org/patch/643293
[3] http://patchwork.ozlabs.org/patch/449589
[4] http://patchwork.ozlabs.org/patch/464545
[5] http://patchwork.ozlabs.org/patch/305060
[6] http://patchwork.ozlabs.org/patch/253089
[7] http://patchwork.ozlabs.org/patch/250523
[8] http://patchwork.ozlabs.org/patch/394125
[9] fe7a4b524b
[10] dff1d590b2

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:39 +02:00
Ricardo Martincoski
203b33fab2 check-package: check Config.* files
Warn when help text is larger than 72 columns, see [1].
Warn for wrongly indented attributes, see [1].
Warn when the convention of attributes order is not followed, see [2].

[1] http://nightly.buildroot.org/#writing-rules-config-in
[2] http://nightly.buildroot.org/#_config_files

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:38 +02:00
Ricardo Martincoski
8e87d76c12 check-package: check *.mk files
Warn when there are obvious indentation errors:
- the number of expect tabs is not yet checked since it is more complex
  to achieve;
- the content inside define ... endef should be indented with tab(s),
  see [1];
- line just after a backslash should be indented with tab(s), see [2];
- other lines should not be indented, see [3];
- ignore empty lines and comments.
Warn when there is no well-formatted header in the file:
- 80 hashes at lines 1 and 5;
- 1 hash at lines 2 and 4;
- empty line at line 6;
- see [4];
- ignore files that only include other mk files.
Warn when there are more than one space before backslash, see [5].
Warn when there is a trailing backslash [6].
Warn for flags set to default value YES or NO, see [7], [8], [9].

[1] http://patchwork.ozlabs.org/patch/681429/
[2] http://patchwork.ozlabs.org/patch/681430/
[3] http://patchwork.ozlabs.org/patch/559209/
[4] http://nightly.buildroot.org/#writing-rules-mk
[5] http://patchwork.ozlabs.org/patch/649084/
[6] http://patchwork.ozlabs.org/patch/535550/
[7] http://patchwork.ozlabs.org/patch/704718/
[8] http://nightly.buildroot.org/#_infrastructure_for_packages_with_specific_build_systems
[9] http://nightly.buildroot.org/#_infrastructure_for_autotools_based_packages

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:35 +02:00
Ricardo Martincoski
23b0b4c028 check-package: check *.patch files
Warn when the name of the patch file does not start with number (apply
order), see [1].
Warn when the patch was generated using git format-patch without -N, see
[2].
Warn when the patch file has no SoB, see [3].

[1] http://nightly.buildroot.org/#_providing_patches
[2] http://patchwork.ozlabs.org/patch/704753/
[3] http://nightly.buildroot.org/#_format_and_licensing_of_the_package_patches

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:34 +02:00
Ricardo Martincoski
6c931bbf5f check-package: check *.hash files
Check each hash entry (see [1]) and warn when:
- it does not have three fields;
- its type is unknown;
- its length does not match its type;
- the name of the file contains a directory component.

[1] http://nightly.buildroot.org/#adding-packages-hash

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:33 +02:00
Ricardo Martincoski
c6bf3239cb check-package: check whitespace and empty lines
Create 3 new check functions to warn when:
- there are consecutive empty lines in the file, see [1];
- the last line of the file is empty, see [2];
- there are lines with trailing whitespace, see [3].

Apply these functions to Config.*, *.mk and *.hash, but not for *.patch
files since they can contain any of these and still be valid.

[1] http://patchwork.ozlabs.org/patch/682660/
[2] http://patchwork.ozlabs.org/patch/643288/
[3] http://patchwork.ozlabs.org/patch/398984/

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:32 +02:00
Ricardo Martincoski
111132903d support/scripts/check-package: new script
Create the infra to check the style of new packages before submitting.
The overall function of the script is described inside a txt file.
It is designed to process the actual files and NOT the patch files
generated by git format-patch.

Also add the first check function, to warn if a file (Config.*, *.mk,
*.hash, *.patch) has no newline at the last line of the file, see [1].

Basic usage for simple packages:
support/scripts/check-package -vvv package/newpackage/*

Basic usage for packages with subdirs:
support/scripts/check-package -vvv $(find package/newpackage/ -type f)

See "checkpackage" in [2].

[1] http://patchwork.ozlabs.org/patch/631129/
[2] http://elinux.org/Buildroot#Todo_list

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:31 +02:00
Yann E. MORIN
762cd7cdcb support/test-pkg: print number of toolchains and progress
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-06 21:51:29 +02:00
Phelip Etienne
c9062dcdf6 support/scripts: add generic genimage script
This script is a wrapper for the genimage tool used by most boards.
The board postimage script can now call this script instead of invoking
genimage command themselves.

Signed-off-by: Etienne Phelip <etienne.phelip@savoirfairelinux.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:50:22 +02:00
Rahul Bedarkar
f8ff7ab0be scripts/scanpypi: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:43:36 +02:00
Rahul Bedarkar
e261d807f3 scripts/scancpan: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:35:14 +02:00
Yann E. MORIN
c5fa9308ea core/br2-external: properly report unexpected errors
Unextected error in the br2-external script are properly caught, but
they are not reported properly, and we end up in either of two
situations:

  - the .br2-external.mk file is not generated, in which case make will
    try to find a rule to generate it (because the 'include' directive
    tries to generate missing files);

  - the .br-external.mk file is generated but does not contain the error
    variable, and thus the build might not get interrupted.

We fix that by using a trap on the pseudo ERR signal, to emit the error
variable on unexpected errors.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-26 23:44:04 +02:00
Thomas Petazzoni
d5d642a43f support/scripts/check-bin-arch: fix after "fix for filenames with spaces"
In commit 2f6c5e513c
("support/check-bin-arch: fix for filenames with spaces"), Yann
adjuste the check-bin-arch script to properly handle filenames with
spaces.

However, he also did a subtle change of the regexp that extracts the
path of the files. It was:

  "/^${package},(.+)$/!d; s//\1/;"

and Yann changed it to:

  "/^${package},\.(.+)$/!d; s//\1/;"

So the file paths used to start with a dot (like "./usr/share/foo"),
and now they no longer start with a dot (like "/usr/share/foo"). While
this modification is good and makes sense, the match for
/lib/firmware/ was not adjusted accordingly, and the follow-up patch
also ignoring /usr/share was not adjusted as well.

This commit fixes those /lib/firmware/ and /usr/share/ special cases,
which will fix:

  http://autobuild.buildroot.net/results/76a1475f4cdedb80426fb022ef2e644aa5625660/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-22 21:43:54 +01:00
Yegor Yefremov
3b627c89dc graph-depends: rename pkgutil.py to brpkgutil.py
pkgutil.py is also part of Python itself. Placing pkgutil.py as is
in a folder with other scripts that require original pkgutil will
break them. This is the case with scanpypi. So rename pkgutil.py
to brpkgutil.py to avoid naming collision.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=9766

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-21 23:11:04 +01:00
Thomas Petazzoni
ba72d71d79 support/script/check-bin-arch: ignore /usr/share
/usr/share normally should not contain binaries executable for the
target platform. However, it might contain ELF binaries for other
platforms, such as firmware files installed by Qemu or
pru-software-support.

Instead of special-casing each package, let's simply ignore /usr/share.

Fixes:

  http://autobuild.buildroot.net/results/6f3fea9f6adaef1573fbb0dd6903b5d99e470610/
  (pru-software-support)

  http://autobuild.buildroot.net/results/fe8892bc22a03299fc41e30bfea5e42166838f88/
  (qemu)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:50 +01:00
Yann E. MORIN
2f6c5e513c support/check-bin-arch: fix for filenames with spaces
Filenames with spaces will break the current for loop.

Fix that by using a while-read loop, fed with the list of files on
stdin, using process substitution.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:17 +01:00
Jérôme Pouiller
6e45c5500b pycompile: allow to force compilation
By default, compile_dir() relies on the modification time to know if a
python file has to be built again. However in some circumstances (when
doing reproducible builds), modification times are not reliable. Thus,
this patch adds a way to force the rebuild of all python sources.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 23:00:25 +01:00
Thomas Petazzoni
bbb7f6c16c Makefile: add check of binaries architecture
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.

This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.

Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.

Example output with the firejail package enabled, when building for an
ARM target:

ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM

Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 22:22:22 +01:00
Andrey Smirnov
61fcd08247 scripts/pycompile: Accomodate latest Python 3 codebase
As of the version 3.6.0 compile_dir() call will treat its 'quiet'
argument as a full blown integer rather than a boolean value and perform
integer comparison operations such as '<' or '>='.

To account for that convert ReportProblem type to be a true derivative
of built-in int() and override all of int's rich comparison operators in
order to be able to "sniff" for PyCompileError in all possible use-cases

The integer value ReportProblem pretends to be is teremined by class
variable VALUE which is set to 1.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-19 15:04:32 +01:00
Benoît Allard
ac53042bf2 support/scripts/test-pkg: extend logfile with olddefconfig output
Signed-off-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-09 21:34:33 +01:00
Yann E. MORIN
00d34e8a6f support/mkusers: rename variable to avoid name clashing
The LINES variable is automatically set by bash to represent the number
of lines in the terminal. That variable can be set when the shell
receives SIGWINCH.

If the shell does receive SIGWINCH after our LINES array is filled, the
content of the array is mangled.

Rename the variable to avoid that.

Fixes #9456

Reported-by: George Y. <georgebrmz@oss3d.com>
Reported-by: Paul Stewart <paulstewartis@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-04 11:56:51 +01:00
Yann E. MORIN
5bf12ad141 support/test-pkg: report number and types of failures
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
c3c69c4dbb support/test-pkg: be less verbose
Requested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
aeabb58dc5 support/test-pkg: simplify the config check
Use comm(1) to check that all our config options are properly set in the
resulting configuration, rather than our canned and fragile code.

Reported-by: Cam Hutchison <camh@xdna.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
55bbbe04c3 support/test-pkg: check the config snippet exists
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:49 +01:00
Yann E. MORIN
67a221be1b support/test-pkg: fix code style
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:46 +01:00
Yann E. MORIN
9e7885d5f7 support/test-pkg: add option to limit the number of tests
Sometimes, it interesting to have a global overview of whether the
package builds at all or not, rather than test on all toolchains.

Add an option that allows testing on a limited set of randomly choosen
toolchains.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 23:12:36 +01:00
Yann E. MORIN
d5c58ceec2 support/test-pkg: store lines missing from resulting configuraiton
When a build is skipped, store the lines from the config snippet, that
are missing in the resulting configuration, in a file in the build
directory, for the user to inspect.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:57:13 +01:00
Yann E. MORIN
47d5da8bcc support/scripts: add script to test a package
This script helps in testing that a package builds fine on a wide range
of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr:
 - completely rewrite the script from Thomas, with help from Luca
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:42:38 +01:00
Frank Hunleth
2ec0a7d11c size-stats: don't count hard links
This change adds inode tracking to the size-stats script so that hard
links don't cause files to be double counted. This has a significant
effect on the size computation for some packages. For example, git has
around a dozen hard links to a large file. Before this change, git would
weigh in at about 170 MB with the total filesystem size reported as
175 MB. The actual rootfs.ext2 size was around 16 MB. With the change,
the git package registers at 10.5 MB with a total filesystem size of
15.8 MB.

Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 19:38:53 +01:00
Thomas De Schampheleire
412e6c80ad graph-depends: split off get_rdepends from get_depends
The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.

Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:28:00 +01:00
Thomas De Schampheleire
12683184b1 graph-depends: split off get_version/get_depends into pkgutil.py
Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:26:55 +01:00
Thomas De Schampheleire
bd8b531d24 graph-depends: avoid use of global var 'rule' in get_depends
Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.

As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.

Instead, pass the global as an argument.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-05 15:20:02 +01:00
Yann E. MORIN
05576fca13 core/br2-external: fix use of relative paths
Fixes #9576

When the path to a br2-external tree is relative, make enters an endless
recursive loop (paths elided for brevity):

    $ make BR2_EXTERNAL=.. foo_defconfig
    make[1]: stat: ../configs/../configs/../configs[...]/toto_defconfig: Filename too long
    make[1]: *** No rule to make target '../configs/../configs/../configs[...]/toto_defconfig',
    needed by '../configs/../configs/../configs[...]/toto_defconfig'.  Stop.
    Makefile:79: recipe for target '_all' failed
    make: *** [_all] Error 2

It is a bit complex to understand the actual technical reason for this
never-ending expansion; it seems it happens in the code generated by the
percent_defconfig macro. Not sure why, though...

But the root cause is the relative path.

Just use absolute, canonical paths to br2-external trees. Always.

[Peter: add bugzilla reference]
Reported-by: outtierbert@gmail.com
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-16 13:55:09 +01:00
Francois Perrad
ec45554cfd perl: bump to version 5.24.0
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-04 17:06:27 +01:00
Danomi Manchego
1ac9d83f55 mkusers: fix spelling in intro comment
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-20 09:32:06 +01:00
Sam bobroff
41444d7f89 pkg-autotools: generic configure fix for powerpc64
Many (100+) packages supported by buildroot contain old configure
scripts (or build them from old versions of autotools) that are unable
to determine how to link shared libraries on powerpc64 and
powerpc64le. This causes that test to erroneously fail on toolchains
that are not "bi-endian" (which is the case for toolchains built by
buildroot), which causes configure to build static libraries instead
of dynamic ones. Although these builds succeed, they tend to cause
linker failures in binaries later linked against them.

Because affected configure files can be discovered automatically, this
patch introduces a hook (enabled only when building for powerpc64 and
powerpc64le) that uses a script to scan and fix each package.

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-12-05 22:51:17 +01:00
Yann E. MORIN
b14b02698e core/br2-external: restore compatibility with old distros
Currently, the br2-external script uses bash-4's associative arrays.

However, some oldish enterprise-class distros like RHEL5 still use
bash-3.1 which lacks associative arrays.

We restore compatibility with those oldish distros using 'eval' to
emulate associative arrays, as suggested by Arnout.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 23:31:20 +01:00
Gaël PORTAY
c29f023fc6 get-developers: fix parentheses for print
Python3 complains about missing parentheses.

$ ./support/scripts/get-developers
  File "./support/scripts/get-developers", line 45
    print f
          ^
SyntaxError: Missing parentheses in call to 'print'

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-05 11:49:05 +01:00
David Lechner
7d4a2f2ca4 linux: add ev3dev extension
This adds an ev3dev Linux drivers extension that provides Linux kernel
drivers for LEGO MINDSTORMS EV3 from the ev3dev project.

Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-02 17:52:24 +01:00
Yann E. MORIN
2a2eb55ca7 core/graph-depends: add option to graph reverse dependencies
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.

It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 22:59:05 +02:00
Rahul Bedarkar
1b0df8f23c scripts/get-developers: correct type of patches argument
Current type for 'patches' argument is str. It supposed to only
contain names of files.

If we specify FileType as type, then we don't need to open file ourself
and it allows script to read patch from standard input as well.

e.g.
$ git show -1 | ./support/scripts/get-developers -

Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 12:43:27 +02:00
Rahul Bedarkar
a1d12fe939 get-developers: use --cc instead of --to for developers
Output of get-developers script in our manual uses --cc for developers,
but actual output of get-developers script uses --to. This patch makes
code consistent with documentation, by using --cc for developers.

Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:38:16 +02:00
Yann E. MORIN
e30fa61d80 core/br2-external: fix link to manual for converting old trees
To be noted: that link will only be valid once we have a released
manual. In the meantime, it's accessible on the nightly manual:
    http://nightly.buildroot.org/#br2-external-converting

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:20:18 +02:00
Yann E. MORIN
5feff44038 core/br2-external: export paths and descriptions
Now that we support multiple br2-external trees, BR2_EXTERNAL is no
longer exported in the environment.

This means that post-build scripts in a br2-external tree can no longer
find their own files (well, they could re-invent the path by stripping
their known-relative path, but that'd be ugly, especially since we can
very well provide it).

Export the path for each br2-external trees as environment variables.
Do so for the description as well, as a courtesy.

Also, re-order variable definitions to be more logical: first, purely
internal variables, then exported variables.

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:19:14 +02:00
Yann E. MORIN
49117c1028 core: support description for br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
20cd497387 core: add support for multiple br2-external trees
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.

A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.

Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.

Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.

What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:

  - the list of all br2-external names, BR2_EXTERNAL_NAMES,

  - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
    point each to the actual location of the corresponding tree,

  - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,

  - the space-separated list of absolute paths to the external trees,
    BR2_EXTERNAL_DIRS.

Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.

This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".

Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.

If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
a71e311df6 docs/manual: add appendix to convert old br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
[Peter: slightly reword]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
fc34cf772c core: introduce per br2-external NAME
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.

This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.

Also, display the NAME and its path as a comment in the menuconfig.

This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.

The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.

Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).

Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
f834b0ef21 core/br2-external: validate even more
A br2-external tree must provide external.mk and Config.in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
6bd19ccf0d core: offload handling of BR2_EXTERNAL into the script
Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have one
(BR2_EXTERNAL is not empty).

There is now no reason to treat those two cases differently:

  - the kconfig snippet is always generated appropriately (i.e. it would
    include the br2-external tree if set, or include nothing otherwise);

  - we no longer have a dummy br-external tree either.

Also, the Makefile code to handle BR2_EXTERNAL is currently quite
readable if at least a little bit tricky.

However, when we're going to add support for using multiple br2-external
trees simultaneously, this code would need to get much, much more complex.

To keep the Makefile (rather) simple, offload all of the handling of
BR2_EXTERNAL to the recently added br2-external helper script.

However, because of Makefiles idiosyncracies, we can't use a rule to
generate that Makefile fragment.

Instead, we use $(shell ...) to call the helper script, and include the
fragment twice: once before the $(shell ...) so we can grab a previously
defined BR2_EXTERNAL value, a second time to use the one passed on the
command line, if any.

Furthermore, we can't error out (e.g. on non-existent br2-external tree)
directly from the fragment or we'd get that error on subsequent calls,
with no chance to override it even from command line.

Instead, we use a variable in which we store the error, set it to empty
before the second inclusion, so that only the one newly generated, if
any, is taken into account.

Since we know the script will always be called from Makefile context
first, we know validation will occur in Makefile context first. So we
can assume that, if there is an error, it will be detected in Makefile
context. Consequently, if the script is called to generate the kconfig
fragment, validation has already occured, and there should be no error.
So we change the error function to generate Makefile code, so that
errors are caught as explained above.

Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget'
about the previous value of the BR2_EXTERNAL_MK variable, especially in
the case where BR2_EXTERNAL is now set to empty, so that we do not try
to include it later. That's why we first generate empty version of
BR2_EXTERNAL_MK, and then assign it the new value, if any.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
64e12a370c core: get rid of our dummy br2-external tree
Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no needed.

As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer needed.

Ditch both of those files, and:

  - only generate actual content in the Kconfig snippet when we actually
    do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
    empty);

  - add a variable that contains the path to the external.mk provided by
    the user, or empty if none, and include the path set in that variable
    (make can 'include' nothing without any problem! ;-) )

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
0f4435e950 core: do not hard-code inclusion of br2-external in Kconfig
Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.

This will ultimately allow us to use more than one br2-external tree.

Offload the "User-provided options" menu to the generated Kconfig
snippet. We can also move the definition of the Kconfig-version of
BR2_EXTERNAL into this snippet.

We introduce an extra check that was not present in the previous code,
to check that we do have permission on that directory. Prevciously, it
was handled as a side effect of not being able to cd into there, but it
is cleaner to check it expressly.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00