e7dee76f9b
The top level Makefile in buildroot has a recursive rule which causes
the appearance of a hang as the number of directories in BR2_EXTERNAL
increases. When the number of directories in BR2_EXTERNAL is small, the
recursion occurs, but make detects the recursion and determines the
target does not have to be remade. This allows make to progress.
This is the failing rule:
define percent_defconfig
# Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig outputmakefile
@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(1)/configs/$$@ \
$$< --defconfig=$(1)/configs/$$@ $$(CONFIG_CONFIG_IN)
endef
$(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_defconfig,$(d))$(sep)))
The rule for %defconfig is created for each directory in BR2_EXTERNAL.
When the rule is matched, the stem is 'defconfig_name'. The second
prerequisite is expanded to $(1)/configs/defconfig_name_defconfig. The
rule, and all of the other rules defined by this macro, are invoked
again, but the stem is now $(1)/configs/defconfig_name_defconfig. The
second prerequisite is now expanded to
$(1)/configs/($1)/configs/defconfig_name_defconfig. This expansion
continues until make detects the infinite recursion.
With up to 5 br2-external trees, the time is very small, so that it is
not noticeable. But starting with 6 br2-external trees, the time is
insanely big (so much so that we did not even let it finish after it ran
for hours); see timings toward the end of the commit log.
We fix that by adding a single %_defconfig rule, which is now rsponsible
to find the actual defconfig file that triggered the rule, by iterating
on the reverse list of br2-external trees and then in main tree.
Of course, now, there is no way for make to warn that there is no such
defconfig, as it is no longer part of the prerequisites of the rule. So,
we delegate to the recipe the responsibility to check for that.
Timing (seconds) of `make pc_x86_64_bios_defconfig` with 1..1000
external trees, with make 4.2.1 (* with make 4.3), on a Core i7-7700HQ:
#trees Before After
1 0.312 0.319
2 0.319 0.323
3 0.325 0.327
4 0.353 0.339
5 0.993 0.349
6 1.26* 0.347
7 9.10* 0.362
8 85.93* 0.360
9 n/a 0.373
10 n/a 0.374
50 n/a 0.738
100 n/a 1.228
500 n/a 7.483
1000 n/a 16.076
How to reproduce:
#!/usr/bin/env bash
N="${1:-1000}"
for i in $(seq 1 1000); do
[ -d "br2-external/${i}/configs" ] && break
mkdir -p br2-external/${i}/configs
touch br2-external/${i}/{Config.in,external.mk}
echo "name: BR_TEST_${i}" >br2-external/${i}/external.desc
touch br2-external/${i}/configs/foo{,_${i}}_defconfig
done
time make \
BR2_EXTERNAL="$(
for i in $(seq 1 ${N}); do
printf '%s\n' "$(pwd)/br2-external/${i}"
done
)" \
foo_1_defconfig
Notes: the timings are very dependent on how much the CPU is otherwise
loaded, but having a multi-core CPU slightly loaded helps maintain a
high frequency on the siblings, and that can reduce the above timings
in half! Best to try on an otherwise-idle system.
Fixes: #14996
Reported-by: David Lawson <david.lawson1@tx.rr.com>
Signed-off-by: Nevo Hed <nhed+buildroot@starry.com>
[yann.morin.1998@free.fr:
- split long foreach
- drastically extend the commit log
- provide reproducer script and redo timings
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit
|
||
|---|---|---|
| arch | ||
| board | ||
| boot | ||
| configs | ||
| docs | ||
| fs | ||
| linux | ||
| package | ||
| support | ||
| system | ||
| toolchain | ||
| utils | ||
| .clang-format | ||
| .defconfig | ||
| .flake8 | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .shellcheckrc | ||
| CHANGES | ||
| Config.in | ||
| Config.in.legacy | ||
| COPYING | ||
| DEVELOPERS | ||
| Makefile | ||
| Makefile.legacy | ||
| README | ||
Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation. The documentation can be found in docs/manual. You can generate a text document with 'make manual-text' and read output/docs/manual/manual.text. Online documentation can be found at http://buildroot.org/docs.html To build and use the buildroot stuff, do the following: 1) run 'make menuconfig' 2) select the target architecture and the packages you wish to compile 3) run 'make' 4) wait while it compiles 5) find the kernel, bootloader, root filesystem, etc. in output/images You do not need to be root to build or run buildroot. Have fun! Buildroot comes with a basic configuration for a number of boards. Run 'make list-defconfigs' to view the list of provided configurations. Please feed suggestions, bug reports, insults, and bribes back to the buildroot mailing list: buildroot@buildroot.org You can also find us on #buildroot on OFTC IRC. If you would like to contribute patches, please read https://buildroot.org/manual.html#submitting-patches