Commit Graph

348 Commits

Author SHA1 Message Date
Ricardo Martincoski
111132903d support/scripts/check-package: new script
Create the infra to check the style of new packages before submitting.
The overall function of the script is described inside a txt file.
It is designed to process the actual files and NOT the patch files
generated by git format-patch.

Also add the first check function, to warn if a file (Config.*, *.mk,
*.hash, *.patch) has no newline at the last line of the file, see [1].

Basic usage for simple packages:
support/scripts/check-package -vvv package/newpackage/*

Basic usage for packages with subdirs:
support/scripts/check-package -vvv $(find package/newpackage/ -type f)

See "checkpackage" in [2].

[1] http://patchwork.ozlabs.org/patch/631129/
[2] http://elinux.org/Buildroot#Todo_list

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-08 16:15:31 +02:00
Yann E. MORIN
762cd7cdcb support/test-pkg: print number of toolchains and progress
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-06 21:51:29 +02:00
Phelip Etienne
c9062dcdf6 support/scripts: add generic genimage script
This script is a wrapper for the genimage tool used by most boards.
The board postimage script can now call this script instead of invoking
genimage command themselves.

Signed-off-by: Etienne Phelip <etienne.phelip@savoirfairelinux.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:50:22 +02:00
Rahul Bedarkar
f8ff7ab0be scripts/scanpypi: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:43:36 +02:00
Rahul Bedarkar
e261d807f3 scripts/scancpan: use SPDX short identifier for license
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-04-01 15:35:14 +02:00
Yann E. MORIN
c5fa9308ea core/br2-external: properly report unexpected errors
Unextected error in the br2-external script are properly caught, but
they are not reported properly, and we end up in either of two
situations:

  - the .br2-external.mk file is not generated, in which case make will
    try to find a rule to generate it (because the 'include' directive
    tries to generate missing files);

  - the .br-external.mk file is generated but does not contain the error
    variable, and thus the build might not get interrupted.

We fix that by using a trap on the pseudo ERR signal, to emit the error
variable on unexpected errors.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-26 23:44:04 +02:00
Thomas Petazzoni
d5d642a43f support/scripts/check-bin-arch: fix after "fix for filenames with spaces"
In commit 2f6c5e513c
("support/check-bin-arch: fix for filenames with spaces"), Yann
adjuste the check-bin-arch script to properly handle filenames with
spaces.

However, he also did a subtle change of the regexp that extracts the
path of the files. It was:

  "/^${package},(.+)$/!d; s//\1/;"

and Yann changed it to:

  "/^${package},\.(.+)$/!d; s//\1/;"

So the file paths used to start with a dot (like "./usr/share/foo"),
and now they no longer start with a dot (like "/usr/share/foo"). While
this modification is good and makes sense, the match for
/lib/firmware/ was not adjusted accordingly, and the follow-up patch
also ignoring /usr/share was not adjusted as well.

This commit fixes those /lib/firmware/ and /usr/share/ special cases,
which will fix:

  http://autobuild.buildroot.net/results/76a1475f4cdedb80426fb022ef2e644aa5625660/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-22 21:43:54 +01:00
Yegor Yefremov
3b627c89dc graph-depends: rename pkgutil.py to brpkgutil.py
pkgutil.py is also part of Python itself. Placing pkgutil.py as is
in a folder with other scripts that require original pkgutil will
break them. This is the case with scanpypi. So rename pkgutil.py
to brpkgutil.py to avoid naming collision.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=9766

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-21 23:11:04 +01:00
Thomas Petazzoni
ba72d71d79 support/script/check-bin-arch: ignore /usr/share
/usr/share normally should not contain binaries executable for the
target platform. However, it might contain ELF binaries for other
platforms, such as firmware files installed by Qemu or
pru-software-support.

Instead of special-casing each package, let's simply ignore /usr/share.

Fixes:

  http://autobuild.buildroot.net/results/6f3fea9f6adaef1573fbb0dd6903b5d99e470610/
  (pru-software-support)

  http://autobuild.buildroot.net/results/fe8892bc22a03299fc41e30bfea5e42166838f88/
  (qemu)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:50 +01:00
Yann E. MORIN
2f6c5e513c support/check-bin-arch: fix for filenames with spaces
Filenames with spaces will break the current for loop.

Fix that by using a while-read loop, fed with the list of files on
stdin, using process substitution.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-03-21 22:14:17 +01:00
Jérôme Pouiller
6e45c5500b pycompile: allow to force compilation
By default, compile_dir() relies on the modification time to know if a
python file has to be built again. However in some circumstances (when
doing reproducible builds), modification times are not reliable. Thus,
this patch adds a way to force the rebuild of all python sources.

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 23:00:25 +01:00
Thomas Petazzoni
bbb7f6c16c Makefile: add check of binaries architecture
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.

This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.

Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.

Example output with the firejail package enabled, when building for an
ARM target:

ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM

Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 22:22:22 +01:00
Andrey Smirnov
61fcd08247 scripts/pycompile: Accomodate latest Python 3 codebase
As of the version 3.6.0 compile_dir() call will treat its 'quiet'
argument as a full blown integer rather than a boolean value and perform
integer comparison operations such as '<' or '>='.

To account for that convert ReportProblem type to be a true derivative
of built-in int() and override all of int's rich comparison operators in
order to be able to "sniff" for PyCompileError in all possible use-cases

The integer value ReportProblem pretends to be is teremined by class
variable VALUE which is set to 1.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-19 15:04:32 +01:00
Benoît Allard
ac53042bf2 support/scripts/test-pkg: extend logfile with olddefconfig output
Signed-off-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-09 21:34:33 +01:00
Yann E. MORIN
00d34e8a6f support/mkusers: rename variable to avoid name clashing
The LINES variable is automatically set by bash to represent the number
of lines in the terminal. That variable can be set when the shell
receives SIGWINCH.

If the shell does receive SIGWINCH after our LINES array is filled, the
content of the array is mangled.

Rename the variable to avoid that.

Fixes #9456

Reported-by: George Y. <georgebrmz@oss3d.com>
Reported-by: Paul Stewart <paulstewartis@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-04 11:56:51 +01:00
Yann E. MORIN
5bf12ad141 support/test-pkg: report number and types of failures
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
c3c69c4dbb support/test-pkg: be less verbose
Requested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
aeabb58dc5 support/test-pkg: simplify the config check
Use comm(1) to check that all our config options are properly set in the
resulting configuration, rather than our canned and fragile code.

Reported-by: Cam Hutchison <camh@xdna.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 16:00:01 +01:00
Yann E. MORIN
55bbbe04c3 support/test-pkg: check the config snippet exists
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:49 +01:00
Yann E. MORIN
67a221be1b support/test-pkg: fix code style
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-12 15:59:46 +01:00
Yann E. MORIN
9e7885d5f7 support/test-pkg: add option to limit the number of tests
Sometimes, it interesting to have a global overview of whether the
package builds at all or not, rather than test on all toolchains.

Add an option that allows testing on a limited set of randomly choosen
toolchains.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 23:12:36 +01:00
Yann E. MORIN
d5c58ceec2 support/test-pkg: store lines missing from resulting configuraiton
When a build is skipped, store the lines from the config snippet, that
are missing in the resulting configuration, in a file in the build
directory, for the user to inspect.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:57:13 +01:00
Yann E. MORIN
47d5da8bcc support/scripts: add script to test a package
This script helps in testing that a package builds fine on a wide range
of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[yann.morin.1998@free.fr:
 - completely rewrite the script from Thomas, with help from Luca
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-09 22:42:38 +01:00
Frank Hunleth
2ec0a7d11c size-stats: don't count hard links
This change adds inode tracking to the size-stats script so that hard
links don't cause files to be double counted. This has a significant
effect on the size computation for some packages. For example, git has
around a dozen hard links to a large file. Before this change, git would
weigh in at about 170 MB with the total filesystem size reported as
175 MB. The actual rootfs.ext2 size was around 16 MB. With the change,
the git package registers at 10.5 MB with a total filesystem size of
15.8 MB.

Signed-off-by: Frank Hunleth <fhunleth@troodon-software.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 19:38:53 +01:00
Thomas De Schampheleire
412e6c80ad graph-depends: split off get_rdepends from get_depends
The use of a 'rule' variable that can contain 'show-depends' or
'show-rdepends' is not logical if get_depends is considered as a reusable
function from various scripts. The name of these rules are too much an
implementation detail.

Therefore, split the existing get_depends into two separate functions
get_depends and get_rdepends, while keeping code duplication to a minimum.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:28:00 +01:00
Thomas De Schampheleire
12683184b1 graph-depends: split off get_version/get_depends into pkgutil.py
Functions to obtain the version and dependencies of a package from Python
can be useful for several scripts. Extract this logic out of graph-depends
into pkgutil.py.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: remove shebang from pkgutil.py, noticed by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-06 12:26:55 +01:00
Thomas De Schampheleire
bd8b531d24 graph-depends: avoid use of global var 'rule' in get_depends
Function get_depends was recently changed to support both normal
dependencies as reverse dependencies, via a global variable 'rule' that
equals 'show-depends' or 'show-rdepends'.

As a subsequent function will extract this function get_depends to a
separate file, the use of globals is problematic.

Instead, pass the global as an argument.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-02-05 15:20:02 +01:00
Yann E. MORIN
05576fca13 core/br2-external: fix use of relative paths
Fixes #9576

When the path to a br2-external tree is relative, make enters an endless
recursive loop (paths elided for brevity):

    $ make BR2_EXTERNAL=.. foo_defconfig
    make[1]: stat: ../configs/../configs/../configs[...]/toto_defconfig: Filename too long
    make[1]: *** No rule to make target '../configs/../configs/../configs[...]/toto_defconfig',
    needed by '../configs/../configs/../configs[...]/toto_defconfig'.  Stop.
    Makefile:79: recipe for target '_all' failed
    make: *** [_all] Error 2

It is a bit complex to understand the actual technical reason for this
never-ending expansion; it seems it happens in the code generated by the
percent_defconfig macro. Not sure why, though...

But the root cause is the relative path.

Just use absolute, canonical paths to br2-external trees. Always.

[Peter: add bugzilla reference]
Reported-by: outtierbert@gmail.com
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-16 13:55:09 +01:00
Francois Perrad
ec45554cfd perl: bump to version 5.24.0
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-01-04 17:06:27 +01:00
Danomi Manchego
1ac9d83f55 mkusers: fix spelling in intro comment
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-12-20 09:32:06 +01:00
Sam bobroff
41444d7f89 pkg-autotools: generic configure fix for powerpc64
Many (100+) packages supported by buildroot contain old configure
scripts (or build them from old versions of autotools) that are unable
to determine how to link shared libraries on powerpc64 and
powerpc64le. This causes that test to erroneously fail on toolchains
that are not "bi-endian" (which is the case for toolchains built by
buildroot), which causes configure to build static libraries instead
of dynamic ones. Although these builds succeed, they tend to cause
linker failures in binaries later linked against them.

Because affected configure files can be discovered automatically, this
patch introduces a hook (enabled only when building for powerpc64 and
powerpc64le) that uses a script to scan and fix each package.

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-12-05 22:51:17 +01:00
Yann E. MORIN
b14b02698e core/br2-external: restore compatibility with old distros
Currently, the br2-external script uses bash-4's associative arrays.

However, some oldish enterprise-class distros like RHEL5 still use
bash-3.1 which lacks associative arrays.

We restore compatibility with those oldish distros using 'eval' to
emulate associative arrays, as suggested by Arnout.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-23 23:31:20 +01:00
Gaël PORTAY
c29f023fc6 get-developers: fix parentheses for print
Python3 complains about missing parentheses.

$ ./support/scripts/get-developers
  File "./support/scripts/get-developers", line 45
    print f
          ^
SyntaxError: Missing parentheses in call to 'print'

Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-05 11:49:05 +01:00
David Lechner
7d4a2f2ca4 linux: add ev3dev extension
This adds an ev3dev Linux drivers extension that provides Linux kernel
drivers for LEGO MINDSTORMS EV3 from the ev3dev project.

Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-11-02 17:52:24 +01:00
Yann E. MORIN
2a2eb55ca7 core/graph-depends: add option to graph reverse dependencies
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.

It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 22:59:05 +02:00
Rahul Bedarkar
1b0df8f23c scripts/get-developers: correct type of patches argument
Current type for 'patches' argument is str. It supposed to only
contain names of files.

If we specify FileType as type, then we don't need to open file ourself
and it allows script to read patch from standard input as well.

e.g.
$ git show -1 | ./support/scripts/get-developers -

Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-25 12:43:27 +02:00
Rahul Bedarkar
a1d12fe939 get-developers: use --cc instead of --to for developers
Output of get-developers script in our manual uses --cc for developers,
but actual output of get-developers script uses --to. This patch makes
code consistent with documentation, by using --cc for developers.

Signed-off-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:38:16 +02:00
Yann E. MORIN
e30fa61d80 core/br2-external: fix link to manual for converting old trees
To be noted: that link will only be valid once we have a released
manual. In the meantime, it's accessible on the nightly manual:
    http://nightly.buildroot.org/#br2-external-converting

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:20:18 +02:00
Yann E. MORIN
5feff44038 core/br2-external: export paths and descriptions
Now that we support multiple br2-external trees, BR2_EXTERNAL is no
longer exported in the environment.

This means that post-build scripts in a br2-external tree can no longer
find their own files (well, they could re-invent the path by stripping
their known-relative path, but that'd be ugly, especially since we can
very well provide it).

Export the path for each br2-external trees as environment variables.
Do so for the description as well, as a courtesy.

Also, re-order variable definitions to be more logical: first, purely
internal variables, then exported variables.

Reported-by: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Benoît Allard <benoit.allard@greenbone.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-10-19 23:19:14 +02:00
Yann E. MORIN
49117c1028 core: support description for br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
20cd497387 core: add support for multiple br2-external trees
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.

A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.

Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.

Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.

What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:

  - the list of all br2-external names, BR2_EXTERNAL_NAMES,

  - the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
    point each to the actual location of the corresponding tree,

  - the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,

  - the space-separated list of absolute paths to the external trees,
    BR2_EXTERNAL_DIRS.

Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.

This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".

Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.

If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
a71e311df6 docs/manual: add appendix to convert old br2-external trees
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
[Peter: slightly reword]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
fc34cf772c core: introduce per br2-external NAME
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.

This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.

Also, display the NAME and its path as a comment in the menuconfig.

This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.

The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.

Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).

Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
f834b0ef21 core/br2-external: validate even more
A br2-external tree must provide external.mk and Config.in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
6bd19ccf0d core: offload handling of BR2_EXTERNAL into the script
Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have one
(BR2_EXTERNAL is not empty).

There is now no reason to treat those two cases differently:

  - the kconfig snippet is always generated appropriately (i.e. it would
    include the br2-external tree if set, or include nothing otherwise);

  - we no longer have a dummy br-external tree either.

Also, the Makefile code to handle BR2_EXTERNAL is currently quite
readable if at least a little bit tricky.

However, when we're going to add support for using multiple br2-external
trees simultaneously, this code would need to get much, much more complex.

To keep the Makefile (rather) simple, offload all of the handling of
BR2_EXTERNAL to the recently added br2-external helper script.

However, because of Makefiles idiosyncracies, we can't use a rule to
generate that Makefile fragment.

Instead, we use $(shell ...) to call the helper script, and include the
fragment twice: once before the $(shell ...) so we can grab a previously
defined BR2_EXTERNAL value, a second time to use the one passed on the
command line, if any.

Furthermore, we can't error out (e.g. on non-existent br2-external tree)
directly from the fragment or we'd get that error on subsequent calls,
with no chance to override it even from command line.

Instead, we use a variable in which we store the error, set it to empty
before the second inclusion, so that only the one newly generated, if
any, is taken into account.

Since we know the script will always be called from Makefile context
first, we know validation will occur in Makefile context first. So we
can assume that, if there is an error, it will be detected in Makefile
context. Consequently, if the script is called to generate the kconfig
fragment, validation has already occured, and there should be no error.
So we change the error function to generate Makefile code, so that
errors are caught as explained above.

Lastly, when the value of BR2_EXTERNAL changes, we want to 'forget'
about the previous value of the BR2_EXTERNAL_MK variable, especially in
the case where BR2_EXTERNAL is now set to empty, so that we do not try
to include it later. That's why we first generate empty version of
BR2_EXTERNAL_MK, and then assign it the new value, if any.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
64e12a370c core: get rid of our dummy br2-external tree
Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no needed.

As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer needed.

Ditch both of those files, and:

  - only generate actual content in the Kconfig snippet when we actually
    do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
    empty);

  - add a variable that contains the path to the external.mk provided by
    the user, or empty if none, and include the path set in that variable
    (make can 'include' nothing without any problem! ;-) )

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Yann E. MORIN
0f4435e950 core: do not hard-code inclusion of br2-external in Kconfig
Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.

This will ultimately allow us to use more than one br2-external tree.

Offload the "User-provided options" menu to the generated Kconfig
snippet. We can also move the definition of the Kconfig-version of
BR2_EXTERNAL into this snippet.

We introduce an extra check that was not present in the previous code,
to check that we do have permission on that directory. Prevciously, it
was handled as a side effect of not being able to cd into there, but it
is cleaner to check it expressly.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-16 13:01:02 +02:00
Thomas Petazzoni
183d9b654c support/scripts/get-developers: add new script
This script, and its companion library, is more-or-less Buildroot's
equivalent to the kernel get_maintainer.pl script: it allows to get the
list of developers to whom a set of patches should be sent to.

To do so, it first relies on a text file, named DEVELOPERS, at the root
of the Buildroot source tree (added in a followup commit) to list the
developers and the files they are interested in. The DEVELOPERS file's
format is simple:

N:     Firstname Lastname <email>
F:     path/to/file
F:     path/to/another/file

This allows to associate developers with the files they are looking
after, be they related to a package, a defconfig, a filesystem image, a
package infrastructure, the documentation, or anything else.

When a directory is given, the tool assumes that the developer handles
all files and subdirectories in this directory. For example
"package/qt5/" can be used for the developers looking after all the Qt5
packages.

Conventional shell patterns can be used, so "package/python-*" can be
used for the developers who want to look after all packages matching
"python-*".

A few files are recognized specially:

 - .mk files are parsed, and if they contain $(eval
   $(<something>-package)), the developer is assumed to be looking after
   the corresponding package. This way, autobuilder failures for this
   package can be reported directly to this developer.

 - arch/Config.in.<arch> files are recognized as "the developer is
   looking after the <arch> architecture". In this case, get-developer
   parses the arch/Config.in.<arch> to get the list of possible BR2_ARCH
   values. This way, autobuilder failures for this package can be
   reported directly to this developer.

 - pkg/pkg-<infra>.mk are recognized as "the developer is looking after
   the <infra> package infrastructure. In this case, any patch that adds
   or touches a .mk file that uses this infrastructure will be sent to
   this developer.

Examples of usage:

$ ./support/scripts/get-developers 0001-ffmpeg-fix-bfin-build.patch
git send-email--to buildroot@buildroot.org --to "Luca Ceresoli <luca@lucaceresoli.net>" --to "Bernd Kuhls <bernd.kuhls@t-online.de>"

$ ./support/scripts/get-developers -p imx-lib
Arnout Vandecappelle <arnout@mind.be>
Gary Bisson <gary.bisson@boundarydevices.com>

$ ./support/scripts/get-developers -a bfin
Waldemar Brodkorb <wbx@openadk.org>

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-09-21 09:02:13 +02:00
Yann E. MORIN
d4db4277ff manual: get rid of the lists of packages
We currently have four lists of packages in the manual:
  - the non-virtual target packages,
  - the virtual target packages,
  - the host packages,
  - the deprecated features.

Those list take more than half of the manual. They do not serve much
purpose except to show off.

After the recent discussion on the list [0], remove them all.

We can now get rid of our biggish and complex generating script (and its
companion library kconfiglib).

[0] http://lists.busybox.net/pipermail/buildroot/2016-September/171199.html

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-09-19 19:32:59 +02:00
Thomas Petazzoni
34b1bdeba4 size-stats: fix bug when parsing file names with comma
The size-stats script fails when the usb_modeswitch_data is enabled,
because this package installs files that contain commas in their
name. However, the size-stats script also uses comma as a separator for
its CSV files, causing a "ValueError: too many values to unpack" in:

  pkg, fpath = l.split(",")

Fix this by splitting only the two fields that need to be split.

The bug was reported by Matthias <porto.rio@gmx.net>, who also suggested
a fix.

Fixes bug #9136.

Reported-by: Matthias <porto.rio@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-08-09 12:25:11 +02:00
Romain Naour
bd3c4d8c7f support: pkg-stats: add efl package in packages stats
Since efl update to 1.15 version, the efl package is a "real"
Buildroot package. It doesn't contain any subdirectories anymore.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-24 23:25:31 +02:00
Yegor Yefremov
28eae03f5e scanpypi: include LICENCE in the list of supported license files
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-07 11:48:52 +02:00
Yann E. MORIN
d5ae67b4f4 support/apply-patches: re-instate set -e
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.

Use "set -e" to exit immediately if a command return an error.

Be sure to ignore any expected error: when we check if a patch to be
applied has the same basename as an already applied patch, the grep
would fail when no such patch was already applied. We should not fail
in this case.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-06 14:47:30 +02:00
Thomas Petazzoni
75d8977ba8 Revert "apply-patches: catch unexpected failure"
This reverts commit 9cf1ad6cdb.

This breaks many packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-05 17:42:27 +02:00
Romain Naour
9cf1ad6cdb apply-patches: catch unexpected failure
As reported by Sébastien Szymanski [1], the apply-patches script
doesn't stop if a tar command can't extract an archive.

Use "set -e" to exit immediately if a command return an error.

[1] http://patchwork.ozlabs.org/patch/626196

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-05 17:36:57 +02:00
Francois Perrad
28a001fc7b scancpan: improve message when bad host perl version
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[Thomas:
 - add comment in scancpan about the version dependency, suggested by
   Yann E. Morin.
 - add comment in perl.mk about the need to sync any version change with
   scancpan, also suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 17:41:54 +02:00
Yegor Yefremov
2f4bd0f339 scanpypi: change hash file comments
sha256 checksum will be computed locally either by scanpypi at package
creation or by hand by package updates. Define this checksum as
'computed locally' so that one doesn't need to change this comment by
package updates. Also put comments for both md5 and sha256 in one line.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 16:57:10 +02:00
Yegor Yefremov
f49295cc42 scanpypi: fix help text handling
Make sure a help text is terminated with a full stop.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-04 00:35:16 +02:00
Yegor Yefremov
8a64ade2f3 scanpypi: rework runtime dependency handling
In most cases Python's package dependencies found in setup.py are
runtime dependencies and hence don't need to be mentioned in *.mk
file.

Also add '# runtime' tag to select statements in Config.in.

__create_mk_requirements() itself is left for future uses (cffi backend
handling etc.).

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-07-01 00:03:54 +02:00
Yann E. MORIN
1924159814 support/apply-patches: bail-out on duplicate patch basenames
Patches we save can come from various locations:
  - bundled with Buildroot
  - downloaded
  - from one or more global-patch-dir

It is possible that two patches lying into different locations have the
same basename, like so (first is bundled, second is from an hypothetical
global-patch-dir):
    package/foo/0001-fix-Makefile.patch
    /path/to/my/patches/foo/0001-fix-Makefile.patch

In that case, when running legal-info, we'd save only the second patch,
overwriting the first. That would be problematic, because:

  - either the second patch depends on the first, and thus would no longer
    apply (this is easy to detect, though),

  - or the second patch does not depend on the first, and the compliance
    delivery will not be complete (this is much harder to detect).

We fix that by checking that no two patches have the same same basename.
If we find that the basename of the patch to be applied collides with
that of a previously applied patch, we error out and report the duplicate.

The unfortunate side-effect is that existing setups will now break in
that situation, but that's a minor, corner-case issue that is easily
fixed.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: adjust coding style, fix minor typos in the commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 16:04:24 +02:00
Yann E. MORIN
a3165461bb core/apply-patches: store full path of applied patches
Currently, we only store the filename of the applied patches.

However, we are soon to want to install those patches in the legal-info
directory, so we'll have to know where those patches come from.

Instead of duplicating the logic to find the patches (bundled,
downloaded, from a global patch dir...), just store the full path to
each of those patches so we can retrieve them more easily later on.

Also always create the list-file, even if empty, so that we need not
test for its existence before reading it.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
[Tested only with patches in the Buildroot sources]
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: used $PWD instead of $(pwd), as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-24 15:58:38 +02:00
Denis THULIN
1008ac4406 scanpypi: new utility
A utility for creating python package from the python package index.
It fetches packages info from http://pypi.python.org and generates
corresponding packages files.

Signed-off-by: Denis THULIN <denis.thulin@openwide.fr>
Tested-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: minor tweaks.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-06-11 16:13:31 +02:00
Peter Korsgaard
577021e81b Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-06-01 17:55:16 +02:00
Yegor Yefremov
549bbba67f python/python3: globalize *.pyc files compilation
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.

To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.

The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.

The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.

A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
 - rework for python 3.5
 - integrate Samuel proposal that allows to detect compilation
   failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-26 22:07:35 +02:00
Thomas Petazzoni
8bf947e0ef support/scripts: fix graph-build-time help text
The graph-build-time help text currently looks like this:

usage: graph-build-time [-h] [--type GRAPH_TYPE] [--order GRAPH_ORDER]
                        [--alternate-colors] [--input OUTPUT] --output OUTPUT

Obviously, naming the parameter for --input as OUTPUT is not a very
good idea, so this commit fixes that to name it "INPUT", as expected.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-24 23:26:25 +02:00
Yann E. MORIN
88bad8f462 support/scripts: add helper to hardlink-or-copy
When preparing the legal-info, the source archives are copied in the
legal-info/ output directory. When the archives are big, it can take
quite a bit of time and unnecessarily uses disk space. When the
legal-info output directory is on the same filesystem as the BR2_DL_DIR,
we can easily reduce copy time and disk usage by just using hardlins
instead of copying. However, the BR2_DL_DIR may be on a different
filesystem, so we must fallback to copying in this case

Introduce a helper script that copies a source file into a destination
directory, by first attempting to hard-link, and falling back to a
plain copy in case the hardlink fails.

In case the destination already exists, it is forcibly removed first, to
avoid clobering any existing target file (and especially any hardlink to
it), since cp -f does not remove the destination file, but clobbers it.

In some situations, it will be necessary that the destination file is
named differently than the source, so if a third argument is specified,
it is treated as the basename of the destination file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-11 23:14:37 +02:00
Samuel Martin
bf8f0136e9 support/scripts/check-host-rpath: also check HOST_DIR/{bin, sbin}
At least syslinux is installing stuff in HOST_DIR/sbin.

Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-04-21 22:17:36 +02:00
Samuel Martin
ac031cfb97 support/scripts: fix graph-depends when run with python3
Make graph-depends script opening the output file in text mode since
only ascii characters will be written.

This change fixes the following error occuring when the default host
python interpreter is python3:

  make: Entering directory '/opt/buildroot'
  Getting targets
  Getting dependencies for ['toolchain-external', 'toolchain', 'busybox', ...]
  Getting dependencies for ['host-python3', 'host-pkgconf', 'host-gettext', ...]
  Getting dependencies for ['host-libxml2', 'host-swig', 'host-m4', ...]
  Getting version for ['toolchain-external', 'toolchain', 'busybox', ...]
  Traceback (most recent call last):
    File "/opt/buildroot/support/scripts/graph-depends", line 425, in <module>
      outfile.write("digraph G {\n")
  TypeError: a bytes-like object is required, not 'str'
  Makefile:807: recipe for target 'graph-depends' failed
  make[1]: *** [graph-depends] Error 1
  Makefile:84: recipe for target '_all' failed
  make: *** [_all] Error 2
  make: Leaving directory '/opt/buildroot'

While with python2, adding 'b' to the openning mode has no effect on
Linux (c.f. [2]), the above error is expected with python3 (c.f. [1]).

Therefore, just open the outfile in default (i.e. text) mode.

[1] https://docs.python.org/3/library/functions.html#open
[2] https://docs.python.org/2/library/functions.html#open

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-04-18 11:34:33 +02:00
Francois Perrad
746a91ab31 scancpan: use recommend & test flags only at first level
Currently, these flags are recursively propagated. This behavior is
not expected by users, because it can cause dependencies explosively.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-03-15 23:16:33 +01:00
Francois Perrad
d6cff4abdc scancpan: handle recommended dependencies as optional packages
Currently, without the flag -recommend, scancpan takes as dependency
only one which has the relationship "requires"; this mode works fine.
And, with the flag -recommend, scancpan takes all ones (ie. with
relationship "requires" or "recommends") in the same way; this mode
never works fine, because it is too simplistic.

With this commit, the "not required" dependencies are handled as
optional BR package or skipped if a cyclic dependency is detected.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-03-15 23:16:29 +01:00
Thomas Petazzoni
ee8e4c5c25 graph-depends: add support for excluding host packages
Just like the --stop-on and --exclude options allow to stop on or
exclude virtual packages from the list by passing the "virtual" magic
value, this commit extends the graph-depends logic to support a "host"
magic value for --stop-on and --exclude. This will allow to draw the
graph by stopping on host packages, or by excluding host packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: minor code beautification suggested by Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-08 21:29:06 +01:00
Thomas Petazzoni
7771bb93b2 graph-depends: fix handling of "virtual" in exclude_list
The condition to determine if a virtual package should be excluded
from the list due to "virtual" being passed in --exclude is under a
loop iterating over each entry of the exclude_list, but it doesn't use
the iterator of this list.

Indeed, the condition contains:

	"virtual" in exclude_list

which checks automatically if "virtual" was passed in the list. Due to
this, there is no need for this check to be within the "for p in
exclude_list" iteration. This commit fixes that by moving the check
outside of the loop.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-08 21:28:19 +01:00
Yann E. MORIN
90551bfac0 support/graph-depends: teach it to only check dependencies
Add an option to graph-depends to only do the dependency checks and not
generate the dot program.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:21:40 +01:00
Yann E. MORIN
f48c08f0f5 support/graph-depends: detect circular dependencies
Currently, if there is a circular dependency in the packages, the
graph-depends script just errors out with a Python RuntimeError which is
not caught, resulting in a very-long backtrace which does not provide
any hint as what the real issue is (even if "RuntimeError: maximum
recursion depth exceeded" is a pretty good hint at it).

We fix that by recursing the dependency chain of each package, until we
either end up with a package with no dependency, or with a package
already seen along the current dependency chain.

We need to introduce a new function, check_circular_deps(), because we
can't re-use the existing ones:

  - remove_mandatory_deps() does not iterate,

  - remove_transitive_deps() does iterate, but we do not call it for the
    top-level package if it is not 'all'

  - it does not make sense to use those functions anyway, as they were
    not designed to _check_ but to _act_ on the dependency chain.

Since we've had time-related issues in the past, we do not want to
introduce yet another time-hog, so here are timings with the circular
dependency check:

    $ time python -m cProfile -s cumtime support/scripts/graph-depends
    [...]
             28352654 function calls (20323050 primitive calls) in 87.292 seconds

       Ordered by: cumulative time

       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            1    0.012    0.012   87.292   87.292 graph-depends:24(<module>)
           21    0.000    0.000   73.685    3.509 subprocess.py:473(_eintr_retry_call)
            7    0.000    0.000   73.655   10.522 subprocess.py:768(communicate)
            7   73.653   10.522   73.653   10.522 {method 'read' of 'file' objects}
          5/1    0.027    0.005   43.488   43.488 graph-depends:164(get_all_depends)
            5    0.003    0.001   43.458    8.692 graph-depends:135(get_depends)
            1    0.001    0.001   25.712   25.712 graph-depends:98(get_version)
            1    0.001    0.001   13.457   13.457 graph-depends:337(remove_extra_deps)
         1717    1.672    0.001   13.050    0.008 graph-depends:290(remove_transitive_deps)
    9784086/2672326    5.079    0.000   11.363    0.000 graph-depends:274(is_dep)
    2883343/1980154    2.650    0.000    6.942    0.000 graph-depends:262(is_dep_uncached)
            1    0.000    0.000    4.529    4.529 graph-depends:121(get_targets)
      2883343    1.123    0.000    1.851    0.000 graph-depends:246(is_dep_cache_insert)
      9784086    1.783    0.000    1.783    0.000 graph-depends:255(is_dep_cache_lookup)
      2881580    0.728    0.000    0.728    0.000 {method 'update' of 'dict' objects}
            1    0.001    0.001    0.405    0.405 graph-depends:311(check_circular_deps)
    12264/1717    0.290    0.000    0.404    0.000 graph-depends:312(recurse)
    [...]
    real    1m27.371s
    user    1m15.075s
    sys     0m12.673s

The cumulative time spent in check_circular_deps is just below 0.5s,
which is largely less than 1% of the total run time.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:20:38 +01:00
Yann E. MORIN
a3f75bcc76 support/graph-depends: add option to specify output file
Currently, graph-depends outputs the dotfile program to stdout, and uses
stderr to trace the dependencies it is currently looking for.

Redirection was done because the output was directly piped into the dot
program to generate the final PDF/SVG/... dependency graph, but that
meant that an error in the graph-depends script was never caught
(because shell pipes only return the final command exit status, and an
empty dot program is perfectly valid so dot would not complain).

Add an option to tell graph-depends where to store the generated dot
program, and keep stdout as the default if not specified.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas: rename metavar from DOT_FILE to OUT_FILE for consistency with
the rest of the new option naming.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 23:15:30 +01:00
Thomas De Schampheleire
8c5afd120e support/scripts: add size-stats-compare script
Leverage the CSV files produces by size-stats (make graph-size) to allow
for a comparison of rootfs size between two different buildroot
compilations.

The script takes the file-size CSV files of two compilations as input, and
produces a textual report of the differences per package.
Using the -d/--detail flag, the report will show the file size changes
instead of package size changes.
The -t/--threshold option allows to ignore file size differences smaller
or equal than the given threshold (in bytes).

Example output is:

Size difference per package (bytes), threshold = 0
--------------------------------------------------------------------------------
       -8192         busybox
      228572   added dmalloc
      301584   added jq
--------------------------------------------------------------------------------
      521964         TOTAL

or with detailed view:

Size difference per file (bytes), threshold = 0
--------------------------------------------------------------------------------
       -8192         bin/busybox
       18152   added usr/bin/jq
       39252   added usr/bin/dmalloc
       46968   added usr/lib/libdmalloc.so
       47288   added usr/lib/libdmallocxx.so
       47316   added usr/lib/libdmallocth.so
       47748   added usr/lib/libdmallocthcxx.so
      283432   added usr/lib/libjq.so.1.0.4
--------------------------------------------------------------------------------
      521964         TOTAL

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-02-07 15:52:25 +01:00
Peter Korsgaard
53703ee67a support/scripts: drop ancient build-ext3-img script
It hasn't been updated since it was added in 2008, and nowadays things kind
of stuff should be handled with genimage.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-02-01 11:51:08 +01:00
Arnout Vandecappelle
c1a674f994 support/scripts/apply-patches.sh: fix whitespace
The apply-patches.sh script was using a mix of tabs and spaces, and
some three-space indentation. Normalize everything to four-space
indentation.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-13 22:09:08 +01:00
Thomas Petazzoni
a059055c52 support/scripts/size-stats: fix copy/paste error in ArgumentParser
A copy/paste error in the ArgumentParser() constructor call disclosed
the fact that the author of the script has shamefully based his work
on the existing graph-build-time script. This commit fixes this
mistake, therefore hiding in a better way how size-stats was
vampirized from graph-build-time.

Reported-by: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-01-13 16:35:19 +01:00
Peter Korsgaard
f62ce058be graph-depends: correct is_dep() comment
The uncached variant is called is_dep_uncached(), not is_dep_full().

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-12-29 23:46:10 +01:00
Thomas Petazzoni
5ed60cee1e graph-depends: optimize remove_transitive_deps()
For large configurations, the execution time of
remove_transitive_deps() becomes really high, due to the number of
nested loops + the is_dep() function being recursive.

For an allyespackageconfig, the remove_extra_deps() function takes 334
seconds to execute, and the overall time to generate the .dot file is
6 minutes and 39 seconds. Here is a timing of the different
graph-depends steps and the overall execution time:

  Getting dependencies:   42.5735 seconds
  Turn deps into a dict:   0.0023 seconds
  Remove extra deps:     334.1542 seconds
  Get version:            22.4919 seconds
  Generate .dot:           0.0197 seconds

  real	6m39.289s
  user	6m16.644s
  sys	0m8.792s

By adding a very simple cache for the results of is_dep(), we bring
down the execution time of the "Remove extra deps" step from 334
seconds to just 4 seconds, reducing the overall execution time to 1
minutes and 10 seconds:

  Getting dependencies:  42.9546 seconds
  Turn deps into a dict:  0.0025 seconds
  Remove extra deps:      4.9643 seconds
  Get version:           22.1865 seconds
  Generate .dot:          0.0207 seconds

  real	1m10.201s
  user	0m47.716s
  sys	0m7.948s

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Gustavo Zacarias <gustavo.zacarias@free-electrons.com>
[yann.morin.1998@free.fr:
  - rename is_dep() to is_dep_uncached(), keep existig code as-is
  - add is_dep() as a cached-version of is_dep_uncached()
  - use constructs more conform with 2to3
  - use exceptions (EAFP) rather than check-before-use (LBYL) to be more
    pythonist; that even decreases the duration yet a little bit more!
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2015-12-29 23:26:22 +01:00
Ryan Barnett
aab481c14e apply-patches: only use first field of line for series file
A series file for quilt has a valid syntax of:

  fixes/autoconf.diff -p1
  fixes/doc-html-local-css.diff -p1
  fixes/gnu-inline.diff -p1

However, with the current way that a series file is handled, it will
error out because the -p1 is tried as a file. This is because in the
for loop that iterates the files, we only look for comment lines. Then
each line is used within a bash for loop which uses spaces a
delimiter. In order to fix this, we should only use the string that
comes before a space in the series file.

Note that the format allows for any arbitrary depth to the -pN field.
But since we'll have only one package with -pN fields, and all will be
-p1, we for now always assume -p1. This will have to be fixed whenever
we get a package with other values.

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
[yann.morin.1998@free.fr: expand comment about the format of a series
file and how we interpret it]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
CC: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-12-12 14:34:47 +01:00
Yann E. MORIN
f42e262199 support/check-host-rpath: remove trailing '/' in host dir
Despite the comment saying so, the trailing '/' in the host directory is
not removed. Note however that it is properly removed from extracted
RPATH tags.

This is not visible when the host directory is our default $(O)/host
location, but breaks for user-supplied external host directory, when
the user leaves a trailing slash in the path.

Fix that.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-12-02 23:28:14 +01:00
Yann E. MORIN
5c0c385226 core: check host executables have appropriate RPATH
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.

We currently ensure that in two ways:
  - we add a RPATH tag that points to our host library directory,
  - we export LD_LIBRARY_PATH to point to that same directory.

With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.

However, it turns our that not all the host programs we build end up
with an RPATH tag:
  - some packages do not use our $(HOST_LDFLAGS)
  - some packages' build system are oblivious to those LDFLAGS

In this case, there are two situations:
  - the program is not linked to one of our host libraries: it in fact
    does not need an RPATH tag [0]
  - the program actually uses one of our host libraries: in that case it
    should have had an RPATH tag pointing to the host directory.

For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.

We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.

[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-18 22:47:10 +01:00
Arnout Vandecappelle
7caf46439c apply-patches.sh: use everything from the series file
When a series file exists, we should use every file mentioned in it,
not just the ones ending with .patch or .diff. Also, there's no need
to uncompress anything if it's mentioned in a series file (the tools
that manipulate series files don't support compressed patches).

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Doug Kehn <rdkehn@yahoo.com>
Tested-by: Doug Kehn <rdkehn@yahoo.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-17 09:24:53 +01:00
Cédric Marie
474d39a1ff Makefile: Remove 'quiet' variable
'quiet' variable is set and exported, but it is not used. We can safely
remove it.

This variable is inherited from the Makefile of the Linux kernel, and
is not used in Buildroot.

In support/scripts/mkmakefile, 'quiet' value is checked, but the test
is always true ('quiet' is never set to silent_), so the test can be
removed as well.

Signed-off-by: Cédric Marie <cedric.marie@openmailbox.org>
Reviewed-by: "James Knight" <james.d.knight@live.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-29 00:04:05 +01:00
Francois Perrad
e2aff3d7f5 scancpan: README as default license file
add this heuristic when no specific license file is found

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-10-26 22:56:05 +01:00
Yann E. MORIN
f0c4456530 support/mkusers: allow comments in users tables
The format of the users table files is non trivial, so it is sometimes
handy to add comments explaining the syntax (or simply the reason for
the user) inline in the files.

Ignore empty lines and comment lines prefixed with '#' similar to shell
or makedevs files.

Packages that defined no user (the vast majority) would cause an empty
line to be present in the internal users table, hence the reason we
skipped empty usernames. Now that we ignore empty lines, we no longer
need to check for empty usernames.

Reported-by: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-18 17:52:21 +02:00
Thomas Petazzoni
598c80be8f support/scripts: add size-stats script
This new script uses the data collected by the step_pkg_size
instrumentation hook to generate a pie chart of the size contribution
of each package to the target root filesystem, and two CSV files with
statistics about the package size and file size. To achieve this, it
looks at each file in $(TARGET_DIR), and using the
packages-file-list.txt information collected by the step_pkg_size
hook, it determines to which package the file belongs. It is therefore
able to give the size installed by each package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-17 16:12:59 +02:00
Francois Perrad
2e39ffcc6a pkg-perl: refactor perl infrastructure
the perl dependency of cpan module is no longer generated by scancpan,
but added at the infrastructure level

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-10-02 20:56:52 +01:00
Francois Perrad
ceb1214632 scancpan: fix detection of native module
When a module is native or depends of a native module, it must be
disabled for static builds via its Config.in

We detect native modules by looking at the filenames listed in the
MANIFEST. If there is a file which looks like it contains code that
much be compiled (e.g. .c, .h and so on...), then we exclude that
module (and its dependencies) from static builds.

That's what we tried to do so far, but failed when there was a
comment on the same line as the filename in the manifest, like so:
    foo-bar.c # Bla bla bla

Fix that by detecting either endof-line (as currently done) or
end-of-string.

For an example of failed build of perl-html-parser, see
http://autobuild.buildroot.net/results/128/128671dfa23d843698a63220c2fac1f44e1d5845/

[Thomas: use better commit log proposed by Yann E. Morin.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-09-20 15:14:10 +02:00
Francois Perrad
8498474ce3 scancpan: remove hack for Module-Build
with Perl 5.22, Module-Build is no longer a core module

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-09-06 22:48:49 +02:00
Francois Perrad
072986df1c perl: bump to version 5.22.0
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-09-06 22:48:42 +02:00
Yann E. MORIN
9d0d4c3cd2 docs/manual: fix generation of deprecated list
Since commit 5f117c3 (webkit: mark as deprecated), generation of the
manual has been broken.

This is because that commit added a deprecated dependency on a
prompt-less symbol, BR2_PACKAGE_WEBKIT_ARCH_SUPPORTS. However, the
generation script does not check that a symbol has a prompt before
it attempts to add it to the deprecated list. So, we end up with
traceback:

    Writing the virtual-packages list in:
            /home/ymorin/dev/buildroot/O/build/docs/manual/virtual-package-list.txt
    Traceback (most recent call last):
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 510, in <module>
        buildroot.print_list(list_name, dry_run=args.dry_run, output=output)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 466, in print_list
        item_label=item_label)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 126, in format_asciidoc_table
        enable_choice=enable_choice))
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 350, in _format_symbol_prompt_location
        return "| {0:<40} <| {1}\n".format(get_label_func(symbol),
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 458, in <lambda>
        get_label = lambda x: self._get_symbol_label(x, mark_depr)
      File "/home/ymorin/dev/buildroot/buildroot/support/scripts/gen-manual-lists.py", line 313, in _get_symbol_label
        label = symbol.get_prompts()[0]
    IndexError: list index out of range

However, we can not use the existing _is_deprecated filter function to
filter out symbols without prompts, because this function is also used
to add a '(deprecated)' tag in the man package list (not that it would
not work, but it does not seem /right/). Furthermore, it could also be
used (but is currently not) to build the list of virtual packages, which
do not have a prompt.

So, introduce a filter function, aptly named _is_deprecated_feature(),
to be used as the filter to find deprecated feature, and keep the
existing _is_deprecated() that can be used in any context to decide
whether a symbol is deprecated or not.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-08-02 19:18:51 +02:00
Francois Perrad
6e9d322a11 scancpan: adjust package name
underscore is not allowed in BR package name.
this problem was found with the Perl module DB_File
which must give the BR package perl-db-file.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-23 22:36:18 +02:00
Yann E. MORIN
6bae1ac575 core/out-of-tree: fix Makefile wrapper
Commit 971faf8 (Makefile: fix out-of-tree builds with multiple targets
with 'all') renamed the default target to '_all' to avoid name-clashing.

In doing so, I forgot to also fix the instance in the .PHONY rule.

Fix that now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-16 23:38:52 +02:00
Maxime Hadjinlian
f53d69614e graph-depends: Strip skeleton from dependency
skeleton being a mandatory dependency, we don't want all our packages to
have a link back to that node, the graph would be awful.

Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-14 18:10:58 +02:00
Guido Martínez
fd13247a22 scripts: mkmakefile: set umask before calling BR's makefile
Small optimization so we don't have another 'make' level (caused by the
umask fix) when running the generated makefile.

Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-07-13 17:19:56 +02:00
Ulf Magnusson
954c09ae62 support: remove outdated note re. Python 3 support
Kconfiglib now runs as either Python 2 or Python 3.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-12 21:19:52 +02:00
Ulf Magnusson
b4250efb2f support: get prompts via official Kconfiglib APIs
These weren't available when gen-manual-lists.py was first written.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:21 +02:00
Ulf Magnusson
aba6fa137a support: gen-manual-lists.py base directory simplification
Buildroot doesn't use $srctree from what I could tell.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:07 +02:00
Ulf Magnusson
00cc243170 support: update Kconfiglib to the latest version
Corresponds to a95f477 in https://github.com/ulfalizer/Kconfiglib.

Fixes:

 - Unset user values when loading a zero-byte .config. (5e54e2c)

 - Ignore indented .config assignments. (f8a7510)

 - Do not require $srctree to be set for non-kernel projects. (d56e9c1)

 - Allow digits in $-references to symbols. (ecacdd5)

 - Add Symbol.is_allnoconfig_y(). (deaa624)

 - Fix small output issue with Comments inside Choices.

Also adds Python 3 support and has a lot of internal cleanup and
optimization.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-11 22:10:01 +02:00
James Knight
8d972df185 scripts/mkusers: allow users with no password value set
The following allows a user definition to specify that a created user
entry should not have a password value set. Original implementation
allowed a user definition to provide a password value of "=" (no quotes)
to generate a crypt-encoded empty string value. In some cases, it may be
desired to have no value specified for a user's password. By using a
value "-" for a password, no value will be set in the shadow value.

An example when this can be used is when logging into a terminal.
Logging into a session with an encoded empty password will prompt a user
to enter a password since it does not know the password is empty. If the
password field blank, a login session will not prompt for a password.

Signed-off-by: James Knight <james.knight@rockwellcollins.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-06-09 23:13:41 +02:00
Thomas Petazzoni
8a58e0238e Makefile: rename TARGETS to PACKAGES
For clarity, this commit renames the TARGETS variable to the more
meaningful PACKAGES variable. Indeed, only packages (handled by one of
the package infrastructures) should be listed in this variable, and
not other random non-package targets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2015-04-14 10:16:51 +02:00
Masahiro Yamada
5b686a0674 Fix typos in comment blocks
[Thomas: fix issues noticed by Arnout:
  - Rewrap the linux/Config.in paragraph
  - Revert the "is a toolchain dependency" -> "has a toolchain
    dependency" change from pkg-generic.mk, as the original was
    correct.]

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-08 22:44:43 +02:00
Yann E. MORIN
5bcfaaaf4e support/graph-depends: allow excluding virtual packages
Like for --stop-on, make --exclude recognise the keyword 'virtual',
to stop on virtual packages (as explained in the help...).

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-01 20:02:54 +02:00
Yann E. MORIN
64b70f55cb support/graph-depends: add option to completely exclude a package
Similar to --stop-on, but also omits the package from the graph.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:55:31 +02:00
Yann E. MORIN
979267e8f7 support/graph-depends: accepts globs to stop on package
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:52:31 +02:00
Yann E. MORIN
5558a34e9a support/graph-depends: add option to stop on specific packages
Add a new option to graph-depends, that users can set to stop the graph
on a specific (set of) package(s).

This accepts any actual package name, or the 'virtual' keyword to stop
on virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Francois Perrad <fperrad@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-31 23:50:11 +02:00
Yann E. MORIN
96a8321a3c support/mkusers: enhance error messages
The current error messages are a bit terse, and do not provide all the
required information.

Expand them to provide more context.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Tom Elliott <tommygunsster@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-29 13:39:25 +02:00
Alexey Brodkin
0b32791f00 graph-depends: remove absent targets from TARGET_EXCEPTIONS
Most of targets listed in TARGET_EXCEPTIONS these days are long
gone, so why still keep them?

Most of those targets were removed in this commit:
http://git.buildroot.net/buildroot/commit/?id=02b88600312554bf166f6cfd71f7f2ede783096a

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@sunsite.dk>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-03-11 17:18:24 +01:00
Francois Perrad
ef0b091ca4 graph-depends: display virtual package with italic style
virtual packages are found by their version,
so we retrieve the version of all packages

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-03-08 22:17:11 +01:00
Fabio Porcedda
ff82572a7f pkg-stats: ignore linux-ext-fbtft.mk and doc-asciidoc.mk
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-02-15 23:28:55 +01:00
Fabio Porcedda
cd0c7c37c2 pkg-stats: add kconfig and rebar infrastructures
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-02-15 23:28:44 +01:00
Thomas Petazzoni
665e13c85e Rename BR2_PREFER_STATIC_LIB to BR2_STATIC_LIBS
Since a while, the semantic of BR2_PREFER_STATIC_LIB has been changed
from "prefer static libraries when possible" to "use only static
libraries". The former semantic didn't make much sense, since the user
had absolutely no control/idea of which package would use static
libraries, and which packages would not. Therefore, for quite some
time, we have been starting to enforce that BR2_PREFER_STATIC_LIB
should really build everything with static libraries.

As a consequence, this patch renames BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS, and adjust the Config.in option accordingly.

This also helps preparing the addition of other options to select
shared, shared+static or just static.

Note that we have verified that this commit can be reproduced by
simply doing a global rename of BR2_PREFER_STATIC_LIB to
BR2_STATIC_LIBS plus adding BR2_PREFER_STATIC_LIB to Config.in.legacy.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2014-12-11 22:48:13 +01:00
Francois Perrad
315d29e159 support/script/scancpan: remove duplicated dependency
Dependencies from metacpan comes as a list of modules which is
transformed in a list of distribution for BR.  Different modules could
be included in the same distribution, so duplication is possible.

This can for example be seen with the HTTP-Daemon module, which would
get two times the dependencies on HTTP-Message without this commit.

[Thomas: slightly extend commit log.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-09 21:15:09 +01:00
Francois Perrad
18033effad support/script/scancpan: fix corelist check
module could be removed of the core,
so check if the module is currently in the core,
but not if the module was once time included in the core.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-12-07 23:04:39 +01:00
Francois Perrad
6199f0f823 support/scripts/scancpan: generate hash file
retrieve MD5 and SHA256 from metacpan.org, and store them in the hash
file for each package.

[Thomas: remove the odd indentation of the filename for the md5 hash
lines in the hash file.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-11-22 19:43:00 +01:00
Bjørn Forsman
acc76a6ef9 Change /bin/bash shebangs into /usr/bin/env bash
Not all systems have /bin/bash (e.g. NixOS[1] doesn't). Buildroot
already uses /usr/bin/env shebangs for other interpreters (perl,
python), so why not bash?

This changes only the shebangs used by Buildroot itself; stuff installed
to the target system is left unchanged.

With this applied I can run Buildroot unmodified on NixOS.

[1]: http://nixos.org/

Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-25 01:55:37 +02:00
Fabio Porcedda
9a4e217b73 apply-patches.sh: don't print anything when "make -s" is used
The make "-s" option is used to enable the "Silent operation" so if that
option is used don't print anything as far as there isn't any error.

Add the "-s" option to "apply-patches.sh" to enable silent operation.

[Peter: use the existing QUIET variable]
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-25 01:42:01 +02:00
Gustavo Zacarias
2ff8f54e79 scripts/mkusers: allow the creation of groups alone
Let mkusers create groups alone, useful for supplementary permissions in
udev/systemd for example where users can be added to later at runtime.
Use a magic string "-" to signal that user creation should be skipped.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-22 21:39:12 +02:00
Thomas Petazzoni
256db411db pkg-stats: add statistics about hash files
As discussed during the Buildroot meeting, this commit extends the
pkg-stats script to include statistics about the number of packages
having vs. not having the hash file.

As of today, we have 104 packages with the hash file, and 1274
packages without.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-19 17:20:42 +02:00
Luca Ceresoli
9722381c0d scripts/graph-build-time: properly warn about missing modules
Currently the graph-build-time script prints a python exception if a
needed module cannot be imported. Catch the exception and tell the user
which packages are missing, as we do for other missing dependencies.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-12 17:23:06 +02:00
Luca Ceresoli
96183af9af scripts/graph-build-time: clarify backend selection
This instruction in the middle of 'import' lines looks very strange.

Also, it was not obvious to me what the 'Agg' backend is.

Both things are actually correct, but it took a while to find out why.
So clarify with a comment to save someone else's time.

[Peter: fix s/soe/some/ typo]
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Sascha Arthur <sascha.arthur@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-12 17:21:00 +02:00
Francois Perrad
1fd8d9e4dd support/script/scancpan: add -test option
Perl modules can have three different types of dependencies:

 - configure/build time dependency which becomes host dependency
 - runtime dependency which becomes target dependency
 - test time dependency which is useless in a cross-compiling context like BR

Before this patch, test time dependencies are handled like runtime
dependencies.

After this patch, test time dependencies are ignored by default. The
newly added -test option allows to add them anyway if needed.

[Thomas: reword commit log using Francois proposal.]

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-04 14:42:44 +02:00
Francois Perrad
f7ef4c2ed4 perl: bump to version 5.20.1
[Peter: squash 'add hash' patch into it]
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-02 21:03:31 +02:00
Thomas De Schampheleire
53223cbb4e check-kernel-headers: mktemp --tmpdir not available on RedHat RHEL5
mktemp --tmpdir is not available on older Redhat RHEL5 machines. The
alternative that has the same behavior is 'mktemp -t'.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-28 20:40:57 +02:00
Sascha Arthur
28ed63bae2 scripts/graph-build-time: remove X dependency
[Thomas: tweak commit title.]

Signed-off-by: Sascha Arthur <sascha.arthur@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-09-21 22:44:42 +02:00
Francois Perrad
56e7d0b886 support/script/scancpan: don't generate the trailing slash in <PKG>_SITE variable
The trailing slash was removed in all packages, so don't generate it.
See http://git.buildroot.net/buildroot/commit/?id=c7f4b964718bc5a3329bb730f59e24091ecbc7a2

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-08-30 09:01:32 +02:00
Yann E. MORIN
11c1076db9 toolchain: add option to copy the gconv libraries
The gconv libraries are used to translate between different character sets
('charsets', even 'csets' sometimes). Some packages need them to present
text to the user (eg. XBMC Gotham).

In (e)glibc they are implemented by the internal implemenation of iconv,
called gconv, and are provided as dlopen-able libraries.

Note that some gconv modules need extra libraries (shared by more than
one gconv module), so we must, when adding a subset of modules, scan the
installed modules in search of the missing libraries.

[Thomas: add general explanation in expunge-gconv-modules and fix
coding style.]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Cc: Eric Limpens <limpens@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-27 23:15:48 +02:00
Francois Perrad
827ba10035 support/script/scancpan: more license name ajustment
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-24 22:31:46 +02:00
Thomas De Schampheleire
60714bb2ed trivial: fix typo 'informations'
In English, unlike in French, almost all usages of the word 'information'
are uncountable, meaning that 'informations' is invalid.
This patch fixes this typo throughout the tree, except in CHANGES and
docs/news.html (historic text).

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-24 22:31:20 +02:00
Yann E. MORIN
1c4498082b support/gen-manual-lists.py: bail out early for legacy symbols
When searching for virtual package providers, there's no need to
handle legacy symbols at all, so just bail out early.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 18:03:26 +02:00
Yann E. MORIN
34363f0df1 support/gen-manual-lists.py: remove unneeded arg in private function
No need to pass as argument to a function, members of the class it's in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 18:02:07 +02:00
Francois Perrad
ed8ab0ce3d support/script/scanpcan: small fix
- typo in comment
- remove trailing space in _HOST_DEPENCENCIES when no dependency

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-19 11:38:31 +02:00
Francois Perrad
6d3962f0bd support/scripts/scancpan: automatically populate LICENSE_FILES
This commit improves the scancpan script to automatically populate the
LICENSE_FILES variable using informations available in the Perl
package MANIFEST file.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-18 19:45:20 +02:00
Francois Perrad
1c7ab72386 support/script/scancpan: add support for BR2_PREFER_STATIC_LIB
Perl extensions are loaded at runtime with dlopen(), so it does not
make sense to even build extensions that are written in C when
BR2_PREFER_STATIC_LIB is enabled. A Perl module written in C or with a
dependency on a module written in C is not available when doing a
static build.

Therefore, this commit adapts the scancpan script to automatically
generate a dependency on !BR2_PREFER_STATIC_LIB when the Perl module
would not work in a static-only configuration.

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-17 20:49:39 +02:00
Francois Perrad
6b8376d8f0 support/script/cpan: improve Makefile generation
- remove trailing space after perl when it's the only dependency
- license: substitution of perl name by BR name
- add a tabulation before source

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:45:25 +02:00
Vivien Didelot
7cf527426f graph-build-time: support python3
This patch is the result of 2to3.

Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:44:41 +02:00
Arnout Vandecappelle
cfb6bf5ee9 graph-depends: remove unnecessary redirect of stderr
It hides any error messages reported by make.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:44:00 +02:00
Arnout Vandecappelle
b82f66cf40 graph-depends: support calling from recursive make
The graph-depends script calls make. If the outer make was called
recursively, or if it was called with '-C <somedir>', then the
environment will contain "MAKEFLAGS=w --". Therefore, the recursive
make prints 'Entering' and 'Leaving' messages, which clobbers the
output for dot.

To avoid this, add "--no-print-directory" to the recursive make
arguments. Since we require GNU make 3.81, we can be sure that this
option is available.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-29 10:43:37 +02:00
Francois Perrad
0bf13117aa support/script/scancpan: populates Config.in with homepage
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-22 19:25:43 +02:00
Arnout Vandecappelle
63454b1c07 graph-depends.py: support python3
This patch is the result of 2to3.

In addition, universal_newlines=True is added to the Popen calls. In
python3, this makes sure that the output is decoded so that we get a
string instead of a buffer object.

Cc: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-22 19:05:18 +02:00
Yann E. MORIN
4cce4ac990 graphs/depends: do not draw transitive dependencies by default
The transitive dependencies make the graphs barely readable for large
configs, with a large number of packages.

So, just switch to not drawing the transitive dependencies by default.

By popular demand... ;-)

[Peter: reword]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc; Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-06-13 14:39:48 +02:00
Yann E. MORIN
3fbd9887b3 filesystems: also chown symlinks
Currently, the symlinks in the generated filesystems will have the
UID of the user running the build, because 'chown' does not change
the ownership of symlinks, by default.

Although the implications are limited, some may not want that UID
to leak in the generated filesystems.

So, use 'chown -h' so even symlinks get properly chowned.

Reported-by: Angelo Dureghello <angelo@barix.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-09 11:28:57 +02:00
Yann E. MORIN
4d922634da support/scripts: generate a list of virtual packages
Generate an asciidoc table that can be included in the manual, that
lists the existing virtual packages, the corresponding symbols, and
their providers (and sub-options thereof).

The core of this change is the addition of a new formatter for virtual
packages. This formatter is a bit tricky, as it has to catter for a
bunch of corner cases:
  - provider is not a package, but is sub-options of a package
  - such a sub-option may be itself 'select'-ed by one or more
    other sub-options
  - legacy packages should not be considered as a provider

Those cases are real:
  - sub-options of mesa3d provide EGL or GLES
  - selected sub-options of mesa3d provide GL
  - udev is a legacy package, but it provides udev

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:11:05 +02:00
Yann E. MORIN
fe696d7c65 support/scripts: introduce a symbol formatter to generate package lists
Currently, we can generate two different tables of packages:
  - a single-column table with the symbols' prompts,
  - a two-column table with the symbols' prompts and locations in the
    menuconfig.

For virtual packages, this is not enough, since we will have to display
more columns, with different content:
  - the virtual package name (but such symbols do not have a prompt)
  - the symbol name
  - the providers for the virtual package

So, instead of having a single function that knows how to generate any
table, introduce a formatter function that is passed as argument to,
and called by format_asciidoc_table(). Such formatter functions are
responsible for providing:
  - the layout of the table (number of columns, column arrangement),
  - the formatted header line,
  - a formatted line for a symbol.

What the formatter should ouput depends on its arguments:
  - if none are passed, the layout is returned,
  - if the header label is passed, it returns the formatted header line,
  - otherwise, it returns the formatted line for a symbol.

Two formatter functions are introduced in this changeset, to replace the
current 'sub_menu' feature:
  - _format_symbol_prompt() to display a one-column table with only the
    symbols' prompts,
  - _format_symbol_prompt_location() to display a two-column table with
    the symbols' prompts and locations.

This will help us to later introduce a new formatter to generate a table
for virtual packages.

[Thanks to Samuel for his pythonistic help!]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:10:32 +02:00
Yann E. MORIN
b2c75ecc5d support/scripts: who's responsible to decide what is a package
When generating the package lists, the responsibility to decide what is
actually a package symbol is currently split between the _is_package(),
the get_symbol_subset() and the format_asciidoc_table() functions.

The two latter functions check that an item is really a symbol, and that
is has a prompt.

While this is currently correct for real packages, this will no longer
be the case when we also generate a list of virtual packages, since they
do not have a prompt.

Move the responsibility to verify that a symbol is indeed a package symbol
to _is_package(), so it's all in one place, and makes it easier to change
for virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:09:57 +02:00
Yann E. MORIN
f93edd823f support/scripts: do not display virtual packages in generated lists
If a package has both a 'real' and a 'virtual' definition, consider it
is a virtual package and do not display it in the generated package list.

This is the case for jpeg and cryptodev, that are virtual packages, but
also real (but empty) packages used to provide a prompt to enable/disable
a choice to select an implementation. In this case, we do not want to
list the virtual packages, but only their implementations.

So, consider packages that are both real and virtual as virtual packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:09:38 +02:00
Yann E. MORIN
e49d4f0c37 support/scripts: prepare handling virtual packages in generated lists
Prepare to tell apart real packages from virtual packages.

Currently, the code implicitly recognises only real packages, and
discards virtual packages, because of the heuristic used to recognise
whether a symbol is a package:

  - for real package:
    - symbols  : BR2_PACKAGE_FOO
    - .mk files: foo.mk
  - for virtual packages:
    - symbols  : BR2_PACKAGE_HAS_FOO
    - .mk files: foo.mk

The current heuristic is to check for each symbol if a corresponding .mk
file exists, by stripping 'BR2_PACKAGE_' from the beginning of the symbol,
converting the result to lowercase, and checking if a .mk file exists.

So, as a side effect, it completely misses the virtual packages [*], which
is pretty nice since we get a list with only real packages that the user
can indeed select and see in the menuconfig.

[*] Except for 'cryptodev' and 'jpeg' which are both virtual packages and
normal packages. Except they are not normal packages, they are used to
display a choice of the implementation to use. This case will be fixed in
follow-up patches.

Since we'll soon need to also output the table of virtual packages, we
need to teach the _is_package() function to recognise them as well.

This patch is the first step into that direction: it introduces a new
function _is_real_package() that is just a wrapper to _is_package(), which
gains a new parameter, being the type of packages to filter on.

No behavioural change is made in this patch, it is just a preparatory
patch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-08 17:08:36 +02:00