Go to file
Yann E. MORIN 29aeb02aa0 infra/pkg-generic: use pure Makefile-based recursive dependencies
Calling to the graph-depends script is very costly, as it calls back to
'make' a lot of time.

It turns out that we already have the list of recursive dependencies, so
we can just print that.

As for the recursive reverse dependencies, we use the same memoisation
technique to cut-down on the expansion cost, which would otherwise be on
the order of 𝑶(𝑛²) (with 𝑛 enabled packages).

>From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and
allyespackageconfig (tweaked to avoid multi providers, etc...), the
timings for X-show-recursive-rdepends are:

                    before      after       speedup     #rdeps
    libnss          0m22.932s   0m5.775s     3.97x      3
    qt5base         0m41.176s   0m5.781s     7.12x      67
    libjpeg         0m56.185s   0m5.749s     9.71x      228
    libxml2         0m54.964s   0m5.795s     9.48x      271
    freetype        0m46.754s   0m5.819s     8.07x      287
    libpng          0m53.577s   0m5.760s     9.30x      303
    sqlite          1m15.222s   0m5.807s    12.95x      801
    libopenssl      1m25.471s   0m5.844s    14.63x      931
    readline        1m13.805s   0m5.775s    12.78x      958
    libzlib         1m11.807s   0m5.820s    12.34x      1039
    toolchain       1m23.712s   0m6.080s    13.77x      2107
    skeleton        1m27.839s   0m6.293s    13.96x      2111 (+1)
    host-skeleton   1m27.405s   0m6.350s    13.76x      2172 (+2)

  - speedup: ratio before/after
  - #rdeps: number of recursive reverse dependencies, with the extra
            dependencies returned with this patch, see below for the
            reason.

So, for a low-level package with a lot of reverse dependencies, like
libzlibz, libopenssl or readline are, the timings are already very much
in favour of the change. This is less impressive with packages that
have few dependencies (libnss), but still much faster.

Also, remember that the config tested has as much packages enabled as
possible, so is in itself a degenerate case. With simpler and more
realistic configurations, the gains would probably be a bit lower than
reported above, but various tests still report good improvements
overall (note: coming up with a 'realistic' configuration is pretty
hard, as everyone and their dog have their notion of what is realistic
in their context, so nothing displayed here; timings are left as an
exercise for the interested parties to report aggravation in their
cases should they notice some regression).

Note that, more recursive reverse dependencies may be displayed now,
since we do not apply the exceptions applied in graph-depends. For
example, host-skeleton gains two new recursive reverse dependencies:
skeleton and toolchain, which are both exceptions in graph-depends.

As for direct (not reverse) dependencies: the gain is not as fantastic
as for reverse ones, but it is still noticeable, especially thanks to
a21212fb7c (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES);
just a few examples for %-show-recursive-depends:

                    before      after       speedup     #deps
    libzlib         0m46.864s   0m5.902s     7.94x      17
    qt5base         0m57.590s   0m5.848s     9.85x      190
    sqlite          0m46.601s   0m5.816s     8.01x      24

Basically, displaying recursive dependencies, direct or reverse, is
almost a constant now: it only slightly varies by about 10% depending
on the complexity of the dependency chain, with the parsing of the
Makefiles still accounting for the large majority of the time.

(PS. Thanks to Joseph for suggesting a list of interesting packages
to test, and thanks to Trent for his example of memoisation!)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Joseph Kogut <joseph.kogut@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-17 15:02:01 +01:00
arch arch/mips: add (Marvell) Octeon III processor 2019-02-04 17:30:18 +01:00
board configs/olimex_a20_olinuxino_lime_legacy: remove defconfig 2019-03-15 23:13:41 +01:00
boot boot/optee-os: define a new variable OPTEE_OS_SDK 2019-03-05 21:51:23 +01:00
configs configs/olimex_a20_olinuxino_lime_legacy: remove defconfig 2019-03-15 23:13:41 +01:00
docs docs/manual: document new behaviour of 'make printvars' 2019-03-12 21:59:00 +01:00
fs fs/common.mk: make sure that static devices from packages are created 2018-12-04 21:53:14 +01:00
linux package/binutils: remove version 2.29 2019-03-14 22:18:42 +01:00
package infra/pkg-generic: use pure Makefile-based recursive dependencies 2019-03-17 15:02:01 +01:00
support support/graph-depends: also cut on host-gzip 2019-03-17 14:35:30 +01:00
system system: allow selecting merged /usr along with custom rootfs skeleton 2019-02-06 17:11:38 +01:00
toolchain toolchain: set the ssp gcc option in kconfig 2019-03-13 00:01:55 +01:00
utils utils/test-pkg: generate package config if it is not specified 2019-03-17 10:03:34 +01:00
.defconfig
.flake8
.gitignore
.gitlab-ci.yml support/testing/tests/package/test_openjdk: new test 2019-03-17 14:02:18 +01:00
.gitlab-ci.yml.in .gitlab-ci.yml: use "extends" keyword 2019-02-06 11:40:28 +01:00
CHANGES Update for 2019.02 2019-03-04 22:49:56 +01:00
Config.in toolchain: set the ssp gcc option in kconfig 2019-03-13 00:01:55 +01:00
Config.in.legacy package/libump: remove package 2019-03-15 23:18:23 +01:00
COPYING
DEVELOPERS support/testing/tests/package/test_openjdk: new test 2019-03-17 14:02:18 +01:00
Makefile Makefile: printvars: don't print anything when VARS is not set 2019-03-12 21:58:32 +01:00
Makefile.legacy
README

Buildroot is a simple, efficient and easy-to-use tool to generate embedded
Linux systems through cross-compilation.

The documentation can be found in docs/manual. You can generate a text
document with 'make manual-text' and read output/docs/manual/manual.text.
Online documentation can be found at http://buildroot.org/docs.html

To build and use the buildroot stuff, do the following:

1) run 'make menuconfig'
2) select the target architecture and the packages you wish to compile
3) run 'make'
4) wait while it compiles
5) find the kernel, bootloader, root filesystem, etc. in output/images

You do not need to be root to build or run buildroot.  Have fun!

Buildroot comes with a basic configuration for a number of boards. Run
'make list-defconfigs' to view the list of provided configurations.

Please feed suggestions, bug reports, insults, and bribes back to the
buildroot mailing list: buildroot@buildroot.org
You can also find us on #buildroot on Freenode IRC.

If you would like to contribute patches, please read
https://buildroot.org/manual.html#submitting-patches