Currently, when we need to build the full dependency graph, we call make
to show the list of packages (make show-targets), and then call it again
and again iteratively while it returns new packages.
Since calling make will parse the whole set of our Makefiles, this takes
quite a bit of time (~4s each here), and the total can get pretty long.
However, make being make, already builds the whole dependency tree
information, so we can just ask for it.
Add a new top-level rule 'show-dependency-tree' that displays the whole
set of dependencies for all packages. For each package, its name, type
and version is displayed, then all the direct, first-level dependencies
are dumped. We choose a format that is not unlike the dot-graph format,
because it is both easy to read as a human, and easy to parse as a
machine:
foo: target 1.2.3
foo -> bar host-meh
bar: target virtual
bar -> buz
buz: target 2.3.4
buz ->
host-meh: host virtual
host-meh -> host-bleark
host-bleark: host 3.4.5
host-bleark ->
rootfs-meh: host
rootfs-meh -> host-bleark
To be noted: rootfs are currently reported as if they were 'host'
packages, to stay aligned with how graph-depends currently treats them.
Ideally, graph-depends could be enhanced to recognise them separately,
but that is another story.
For just plain defconfig, which is about the smallest config we can have
with an internal toolchain, we already have a seven-fold improvement
(with the graph-depends rule modified to not run the pdf generation, to
be able to just compare the tree generation):
$ time make graph-depends
real 0m27.344s
$ time make show-dependency-tree
real 0m3.848s
>From defconfig, C++, wchar, locales, ssp, and allyespackageconfig,
tweaked for even more packages (qt5 not qt4, luajit to avoid multi
providers, etc...), the timings are (graph-depends still modified to
not generate the pdf):
$ time make graph-depends
real 1m56.459s
$ time make show-dependency-tree
real 0m5.748s
There. I don't think those numbers need any explanation whatsoever;
they do speak on their own. OK, for maths sake, the ratio is about
twenty-fold. So, "yeah", I guess... ;-)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Calling to the graph-depends script is very costly, as it calls back to
'make' a lot of time.
It turns out that we already have the list of recursive dependencies, so
we can just print that.
As for the recursive reverse dependencies, we use the same memoisation
technique to cut-down on the expansion cost, which would otherwise be on
the order of 𝑶(𝑛²) (with 𝑛 enabled packages).
>From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and
allyespackageconfig (tweaked to avoid multi providers, etc...), the
timings for X-show-recursive-rdepends are:
before after speedup #rdeps
libnss 0m22.932s 0m5.775s 3.97x 3
qt5base 0m41.176s 0m5.781s 7.12x 67
libjpeg 0m56.185s 0m5.749s 9.71x 228
libxml2 0m54.964s 0m5.795s 9.48x 271
freetype 0m46.754s 0m5.819s 8.07x 287
libpng 0m53.577s 0m5.760s 9.30x 303
sqlite 1m15.222s 0m5.807s 12.95x 801
libopenssl 1m25.471s 0m5.844s 14.63x 931
readline 1m13.805s 0m5.775s 12.78x 958
libzlib 1m11.807s 0m5.820s 12.34x 1039
toolchain 1m23.712s 0m6.080s 13.77x 2107
skeleton 1m27.839s 0m6.293s 13.96x 2111 (+1)
host-skeleton 1m27.405s 0m6.350s 13.76x 2172 (+2)
- speedup: ratio before/after
- #rdeps: number of recursive reverse dependencies, with the extra
dependencies returned with this patch, see below for the
reason.
So, for a low-level package with a lot of reverse dependencies, like
libzlibz, libopenssl or readline are, the timings are already very much
in favour of the change. This is less impressive with packages that
have few dependencies (libnss), but still much faster.
Also, remember that the config tested has as much packages enabled as
possible, so is in itself a degenerate case. With simpler and more
realistic configurations, the gains would probably be a bit lower than
reported above, but various tests still report good improvements
overall (note: coming up with a 'realistic' configuration is pretty
hard, as everyone and their dog have their notion of what is realistic
in their context, so nothing displayed here; timings are left as an
exercise for the interested parties to report aggravation in their
cases should they notice some regression).
Note that, more recursive reverse dependencies may be displayed now,
since we do not apply the exceptions applied in graph-depends. For
example, host-skeleton gains two new recursive reverse dependencies:
skeleton and toolchain, which are both exceptions in graph-depends.
As for direct (not reverse) dependencies: the gain is not as fantastic
as for reverse ones, but it is still noticeable, especially thanks to
a21212fb7c (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES);
just a few examples for %-show-recursive-depends:
before after speedup #deps
libzlib 0m46.864s 0m5.902s 7.94x 17
qt5base 0m57.590s 0m5.848s 9.85x 190
sqlite 0m46.601s 0m5.816s 8.01x 24
Basically, displaying recursive dependencies, direct or reverse, is
almost a constant now: it only slightly varies by about 10% depending
on the complexity of the dependency chain, with the parsing of the
Makefiles still accounting for the large majority of the time.
(PS. Thanks to Joseph for suggesting a list of interesting packages
to test, and thanks to Trent for his example of memoisation!)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Joseph Kogut <joseph.kogut@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When host-gzip is needed, it is a mandatory dependency of all packages.
As such, drawing the dependency lines toward host-gzip would uselessly
clutter the graph.
So, like for the skeleton, host-skeleton, and host-tar, we cut the
dependency chains toward host-gzip.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When host-tar is needed, it is a mandatory dependency of all packages.
As such, drawing the dependency lines toward host-tar would uselessly
clutter the graph.
So, like for the skeleton and host-skeleton, we cut the dependency chains
toward host-tar.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
host-skeleton is a dependency of almost all packages, except a very few.
As such, it clutters the dependency graph uselessly.
Do with it as we do for the skeleton: cut the dependency chains.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some times, multiple dependency graphs for a set of packages (mostly
the application-level packages for the project) are included in reports
(e.g. delivery notes). Repeating the mandatory dependencies on all
those graphs is useless and clutters the important dependencies.
When we had only two such mandatory dependencies (toolchain, skeleton),
it was manageable to list them as manual exclusions:
-x toolchain -x skeleton
But we now have quite a few such dependencies, and it becomes a bit more
cumbersome to manage, not counting the ones we may add in the future.
Add an option to exclude all those mandatory dependencies, to generate
neat graphs.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The current graph-depends implementation filters out a number of
"mandatory" dependencies that all packages have: dependency on
"toolchain" and dependency on "skeleton".
Despite this filtering, in full graph dependencies, "toolchain" and
"skeleton" are still shown, because they are target packages, and
therefore appear in the result of "make show-targets". Thanks to this,
they will be visible as dependencies of the "ALL" node, which is the
root of the dependency tree.
However, as we are going to introduce host-skeleton as a "mandatory
dependency" to be filtered out, this is no longer going to work.
This commit adjusts the remove_extra_deps() function to ensure that
when a mandatory dependency is removed, this dependency exists between
the root of the dependency tree and the mandatory dependency.
This issue was noticed by Yann E. Morin, and this commit provides a
different implementation than what Yann proposed in
https://patchwork.ozlabs.org/patch/910453/.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- list mandatory deps before removing them
- fix flake8 warnings
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The OpenJDK package supports a non-assembler variant of the hotspot
virtual machine. This allows archtectures which don't have hard-coded
support (which provides better performance) to still build a virtual
machine that can execute JAVA.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 6f28ce5322 (package/systemd: enable building of systemd-boot)
made the efi and gnu-efi optiona conditional upon whether systemd-boot
is built or not, but forgot to remove the non-conditional ones.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes the following check-package warning:
package/xenomai/xenomai.hash:3: empty line at end of file
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test is a simple "Hello, World" integration test of the OpenJDK
package.
It compiles the Java app on the host, then runs it on an emulated
AARCH64 target and verifies "Hello, World" is printed.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
OpenJDK is a free and open-source implementation of the Java Platform.
This package provides the option to build a client or a server JVM
interpreter.
The default option is the server option, as that is what the majority
of users use. This JVM interpreter loads more slowly, putting more
effort into JIT compilations to yield higher performance.
Unlike most autotools packages, OpenJDK is exceptionally different and
has many quirks, some of which are below:
- X11, alsa, and cups are required to build Java, even if it's a headless build.
See
http://hg.openjdk.java.net/jdk10/jdk10/raw-file/tip/common/doc/building.html#external-library-requirements
for more information.
- host-zip is needed for the zip executable.
- There is no autogen.sh file, instead, a user must call "./configure
autogen."
- OpenJDK ignores some variables unless passed via the environment.
These variables are: PATH, LD, CC, CXX, and CPP.
- OpenJDK defaults ld to the ld binary but passes -Xlinker and -z as
arguments during the linking process, which causes linking failures.
To fix this issue, ld is set to gcc.
- Make -jn is unsupported. Instead, one must use the "--with-jobs="
configure option, and use $(MAKE1).
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[Thomas:
- drop explanations about CC, LD, CXX, etc. be set to their "actual
binaries" instead of ccache: TARGET_CC/TARGET_LD/TARGET_CXX point
to the compiler wrapper, so the usage of ccache is hidden
- make sure at least one of the variants is enabled in Config.in
- drop the submenu for variant selection
- use system zlib instead of the bundled one. This works fine when
BUILD_SYSROOT_CFLAGS and BUILD_SYSROOT_LDFLAGS are passed
- fix minor nits in the Config.in comments]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
systemd-boot is the integration of gummiboot into systemd, when
gummiboot is no longer maintained [0].
Add an option to build systemd-boot as part of the systemd build.
Install the boot files, that can serve as a template for the user
to tweak for their system.
[0] https://cgit.freedesktop.org/gummiboot/commit/?id=55df1539c9d330732e88bd196afee386db6e4a1d
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
[yann.morin.1998@free.fr:
- add missing depends on i386 || x86_64
- add missing dependency to gnu-efi
- add missing boot files
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This fixes the problem with undeclared intptr_t type for builds not including
stdint.h, without breaking builds using -nostdinc.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Drop upstream patch (included in this release).
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Add COPYING to license files as README does not contain the full GPL
text
- Add hash for license files
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Also:
* License file has been renamed: LICENSE.txt -> LICENSE
* Update license file hash -> List of contributors are in a seperate
file now
* Configure option `--with-alsadatadir` has been renamed to `--with-alsaconfdir`
* Configure option `--enable-pcm-test` has been removed
* New optional configure option `ofono` (no build dependency, will
disable internal HFP in favor of oFonos HFP profile)
Signed-off-by: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Drop upstream patch (included in this release).
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This version drops python 2 support and requires cffi.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Starting X manually resulted in a symbol error:
| Xorg: symbol lookup error: /usr/lib/xorg/modules/drivers/fbdev_drv.so: undefined symbol: shadowUpdatePackedWeak
This bug was reported against the fbdev driver, which is fixed
upstream in 0.5.0:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900317
Signed-off-by: Michel Stempin <michel.stempin@wanadoo.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Fixes CVE-2018-4700: Linux session cookies used a predictable random
number seed: https://github.com/apple/cups/releases/tag/v2.2.10.
- Remove fifth patch (already in version)
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Switch site to github to get latest release
- Add hash for license file
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Update patches and send them upstream
- Update license hash (typos fixed and year updated)
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Go 1.12 was released on 2/25/2019: https://blog.golang.org/go1.12
Go 1.12.1 was released on 3/14/2019:
https://golang.org/doc/devel/release.html#go1.12.minor
Additional notes on how Go modules will evolve in 2019 are here:
https://blog.golang.org/modules2019
In Go 1.12, module support remains the same as in Go 1.11. GOPATH is scheduled
for deprecation in 1.13, however. The Buildroot Go package infrastructure should
therefore aim to migrate to a new, currently undefined, Go module friendly
compilation approach before Go 1.13 is released.
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Prelink-cross emulates a runtime linker for a given sysroot. This is
necessary to allow gobject-introspection to build it's typelib files
during cross-compiling.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: drop Config.in.host file and visible option.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add --enable-install-libiberty to HOST_BINUTILS_CONF_OPTS to allow
prelink-cross to build properly.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It is possible to generate one-line config for the package just by
normalize it to the form:
BR2_PACKAGE_${pkg_replaced-to_and_uppercase}
it simplifes a bit of testing package where no additional config options
are needed.
Signed-off-by: Vadim Kochan <vadim4j@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This library was previously a part of sunxi-mali.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The package sunxi-mali-mainline is available for recent kernels.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The old 3.4 Linux kernel used by this defconfig doesn't build with gcc
7.x.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Changes to build and runtime requirements:
* Python 3.4 or later is required to build the GNU C Library.
* On most architectures, GCC 5 or later is required to build the GNU C
Library. (On powerpc64le, GCC 6.2 or later is still required, as
before.)
While at it, remove the double "glibc-" prefix in the version.
https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since version 2.29, glibc requires python 3.4 or later to build the
GNU C Library [1].
We add a new check to verify the version of python3 interpreter
installed on the host. If no suitable python3 interpreter is found,
define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package
dependencies when needed.
[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Adam Duskett <aduskett@gmail.com>
Reviewed-by: Adam Duskett <aduskett@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: drop not so useful comment in the .mk file, as suggested by
Yann E. Morin.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
See https://nodejs.org/en/blog/release/v10.15.3/
The host tool "torque" is unfortunately not built by complying to our
LDFLAGS, so it is not built with the proper RPATH. We fix that using
patchelf, and install it to $(HOST_DIR) to make it available during
the build of the target nodejs.
Signed-off-by: Martin Bark <martin@barkynet.com>
[Thomas:
- add explanation in the commit log about torque
- install torque in $(HOST_DIR)/bin and not $(HOST_DIR)/usr/bin]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use --with-sqlite and --without-sqlite options
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
