Commit Graph

86 Commits

Author SHA1 Message Date
Yann E. MORIN
8b8f5e3366 utils/genrandconfig: fix matching for a few lines
The matching in genrandconfig is idiomatically done by matching whole
lines, i.e. with the terminating \n but a few places are missing that.

Those are only matching against '=y', a boolean symbol, so it is in
practice not causing any issue. Still, for consistency, fix those.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-06-10 21:54:21 +02:00
James Hilliard
929a491f40 utils/genrandconfig: drop glibc Linaro toolchain version check
It has been well over 10 years since glibc 2.14 was released; the last
Debian version that had an earlier glibc was Wheezy, which Freexian
stopped to maintain as an ELTS in June 2020, 4 years ago, while the
oldest still maintained Ubuntu has glibc 2.21. It is now safe to assume
glibc 2.14 on all major, relevant distributions nowadays.

The distutils module is no longer bundled with python 3.12 so this
eliminates the need to install additional python modules under python
3.12.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
[yann.morin.1998@free.fr: add Debian and Ubuntu references]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-06-03 22:27:37 +02:00
James Hilliard
584ebdea6e utils/genrandconfig: switch to async/await format
This requires python 3.5 or newer but is a bit cleaner than the
previous coroutine method.

This should also fix a python3.12 issue:
[Tue, 28 May 2024 13:09:05] INFO: generate the configuration
Traceback (most recent call last):
  File "/home/autobuild/autobuild/instance-0/buildroot/utils/genrandconfig", line 833, in <module>
    ret = asyncio.run(gen_config(args))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 89, in run
    raise ValueError("a coroutine was expected, got {!r}".format(coro))
ValueError: a coroutine was expected, got <generator object gen_config at 0xffff7bd822c0>
[Tue, 28 May 2024 13:09:06] WARN: failed to generate configuration

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-05-31 21:20:03 +02:00
Romain Naour
c6784ea0e1 utils/genrandconfig: remove Sourcery CodeBench Nios-II exclusion
BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII has been removed by [1].

[1] 7fa0159a50

Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-05-01 23:35:15 +02:00
Fabrice Fontaine
6ee61ef95b utils/genrandconfig: fix BR2_TOOLCHAIN_BARE_METAL_BUILDROOT_ARCH
Commit b7232c51dd added default setting
for bare-metal toolchain arch but unfortunately it used configlines.add
instead of configlines.append resulting in the following build failure:

/bin/sh: line 8: /home/autobuild/autobuild/instance-1/output-1/per-package/host-gcc-bare-metal/host/bin/-ar: No such file or directory

While at it, also append /n for consistency

Fixes: b7232c51dd
 - http://autobuild.buildroot.org/results/95ac565653ddb5c14ec71470c32a34ad10b048cb

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-01 09:13:34 +01:00
Thomas Petazzoni
b7232c51dd utils/genrandconfig: add default setting for bare-metal toolchain arch
We recently added support for building a bare-metal toolchain, and the
autobuilders are therefore trying to build such toolchains. However,
by default the BR2_TOOLCHAIN_BARE_METAL_BUILDROOT_ARCH option is
empty, causing the target tuple to be empty, and therefore
host-gcc-bare-metal tries to build a toolchain for the target
"x86_64-pc-linux-gnu", which isn't a bare-metal target, and fails
badly.

In order to properly test this bare-metal support, this commit adjusts
the genrandconfig script so that it injects a valid target
architecture tuple.

Fixes:

  http://autobuild.buildroot.net/results/6cb1514b19bfe056fb984d3538bdccdda5e174c1/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-07 10:23:48 +01:00
Fabrice Fontaine
283e6859f1 utils/genrandconfig: add ubi handling
Add a custom case to make sure that a random configuration with an empty
configuration file for ubi doesn't fail.

ubinize: error!: no sections found the ini-file "/home/buildroot/autobuild/instance-2/output-1/build/ubinize.cfg"

Fixes:
 - http://autobuild.buildroot.org/results/f678c17cc4df06fb2737467e769cd8f72a3ea420

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-08-12 22:06:21 +02:00
Yann E. MORIN
a845276757 utils/genradconfig: drop legacy lpc32xxcdl exclusion
Commit 503252d8b0 (boot/lpc32xxcdl: remove package) forgot to remove
the special handling in genrandconfig. Since the package no longer
exists, we don't need to special-case its symbol anymore, so drop it
now.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-08-10 19:45:19 +02:00
Bernd Kuhls
9b31b64382 utils/genrandconfig: increase default target ubifs image size
The new value is calculated based on:
ROOTFS_SIZE = 5G (see commit 3c98eb26db)
BR2_TARGET_ROOTFS_UBIFS_LEBSIZE = 0x1f800

5G / 0x1f800 = 41610

Fixes:
http://autobuild.buildroot.net/results/f72/f72918d63510b170e5da01bfa9c247cf9dcf507f/

Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-07-14 23:26:09 +02:00
Bernd Kuhls
af5ccec266 utils/genrandconfig: disable cramfs images
Fixes build error:

  warning: estimate of required size (upper bound) is 1374MB, but
  maximum image size is 272MB, we might die prematurely
  mkcramfs: filesystem too big

Fixes:
http://autobuild.buildroot.net/results/d47/d47f9b462707dffe1b6665f143701303b04e2adc/

Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-07-09 18:30:10 +02:00
Fabrice Fontaine
ad4982d18e utils/genrandconfig: add boot-wrapper-aarch64 handling
Fix the following build failure raised since the addition of the package
in commit 7689b72e00:

configure: error: Could not find DTB file: /home/autobuild/autobuild/instance-9/output-1/build/linux-6.1.9/arch/arm64/boot/dts/.dtb

Fixes:
 - http://autobuild.buildroot.org/results/44287ccc8cc9767704642919e6d928d1f57b436d

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-27 16:22:54 +01:00
Fabrice Fontaine
5e1a50e4e6 utils/genrandconfig: add lpc32xxcdl handling
Fix the following build failure raised since the addition of the package
in commit 20695936ad:

make[2]: *** /tmp/instance-0/output-1/build/lpc32xxcdl-2.11/csps/lpc32xx/bsps//source: No such file or directory.  Stop.

Fixes:
 - http://autobuild.buildroot.org/results/72c9a4080318b1b247ca3517c95c689dff1068d2

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-02-25 21:53:48 +01:00
Fabrice Fontaine
02b6af2663 utils/genrandconfig: add ti-k3-r5-loader handling
Fix the following build failure raised since the addition of the package
in commit 0189bcb47c:

make[2]: *** No rule to make target '_defconfig'.  Stop.

Fixes:
 - http://autobuild.buildroot.org/results/d7449b2b2f2349af672bfeee832b89a223a7d9cc

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-02-25 10:39:05 +01:00
James Hilliard
579b65b552 utils/genrandconfig: migrate to asyncio subprocess calls
Since genrandconfig no longer appears to support python2 we can
migrate the subprocess calls to use asyncio variants.

This has the advantage of allowing for runners like autobuild-run to
integrate directly into genrandconfig by calling the asyncio
gen_config using importlib instead of having to run genrandconfig as
a subprocess.

Using asyncio is advantageous here as it eliminates the requirement
for the runner to deal with blocking subprocess calls(by having to
use threading for example).

Also cleanup some unused functions/python2 compatibility shims.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2023-02-06 15:54:38 +01:00
Fabrice Fontaine
5c1f67428a utils/genrandconfig: add mxs-bootlets board handling
Add a custom case to make sure that a random configuration with an empty
board for mxs-bootlets doesn't fail. It reverts to
BR2_TARGET_MXS_BOOTLETS_STMP37xx in that case.

>>> mxs-bootlets 10.12.01 Building
BOARD= CROSS_COMPILE="/home/thomas/autobuild/instance-1/output-1/per-package/mxs-bootlets/host/bin/arm-buildroot-linux-uclibcgnueabi-" /usr/bin/make -j1 -C /home/thomas/autobuild/instance-1/output-1/build/mxs-bootlets-10.12.01 power_prep
/home/thomas/autobuild/instance-1/output-1/per-package/xinetd/host/bin/arm-buildroot-linux-uclibcgnueabi-gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -O2 -g0  -static -DNO_RPC  -I../../include   -c -o inet_aton.o inet_aton.c
make[1]: Entering directory '/home/thomas/autobuild/instance-1/buildroot'
make[1]: warning: -j1 forced in submake: resetting jobserver mode.
build power_prep
/usr/bin/make -C power_prep ARCH= BOARD=
make[2]: Entering directory '/home/thomas/autobuild/instance-1/output-1/build/mxs-bootlets-10.12.01/power_prep'
/usr/bin/make -C ./../mach-/hw
make[3]: Entering directory '/home/thomas/autobuild/instance-1/output-1/build/mxs-bootlets-10.12.01/power_prep'
make[3]: *** ../mach-/hw: No such file or directory.  Stop.

Fixes:
 - http://autobuild.buildroot.org/results/44a2efc64b9b8ff4541430d6b649e7a11a4e4873

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-12-05 08:14:00 +01:00
Thomas Petazzoni
75cb8a4902 utils/genrandconfig: don't build igh-ethercat drivers
igh-ethercat comes with a small number of patched Linux kernel network
drivers, which aim at replacing the ones available in upstream Linux
kernel. All those drivers are provided only for specific kernel
releases. For example:

r8169-2.6.24-ethercat.c
r8169-2.6.24-orig.c
r8169-2.6.27-ethercat.c
r8169-2.6.27-orig.c
r8169-2.6.28-ethercat.c
r8169-2.6.28-orig.c
r8169-2.6.29-ethercat.c
r8169-2.6.29-orig.c
r8169-2.6.31-ethercat.c
r8169-2.6.31-orig.c
r8169-2.6.32-ethercat.c
r8169-2.6.32-orig.c
r8169-2.6.33-ethercat.c
r8169-2.6.33-orig.c
r8169-2.6.35-ethercat.c
r8169-2.6.35-orig.c
r8169-2.6.36-ethercat.c
r8169-2.6.36-orig.c
r8169-2.6.37-ethercat.c
r8169-2.6.37-orig.c
r8169-3.10-ethercat.c
r8169-3.10-orig.c
r8169-3.12-ethercat.c
r8169-3.12-orig.c
r8169-3.14-ethercat.c
r8169-3.14-orig.c
r8169-3.16-ethercat.c
r8169-3.16-orig.c
r8169-3.2-ethercat.c
r8169-3.2-orig.c
r8169-3.4-ethercat.c
r8169-3.4-orig.c
r8169-3.6-ethercat.c
r8169-3.6-orig.c
r8169-3.8-ethercat.c
r8169-3.8-orig.c
r8169-4.4-ethercat.c
r8169-4.4-orig.c

Obviously, this doesn't play well with the random configuration
testing done by utils/genrandconfig. This commit avoids this issue by
making sure we never build any of those drivers as part of the
genrandconfig generated configurations.

Fixes:

  http://autobuild.buildroot.net/results/07b7475d780c067d99ee5618a5fd2bb024a5b4e7/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-21 22:20:43 +01:00
Thomas Petazzoni
c81da5e41c utils/genrandconfig: disallow configs with BR2_XTENSA_CUSTOM=y
When BR2_XTENSA_CUSTOM=y is used with the internal toolchain, an
overlay file is mandatory, which genrandconfig can't provide. So we
simply disallow such configurations.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-11-13 22:37:27 +01:00
Fabrice Fontaine
bada3ae53f utils/genrandconfig: handle a10disp
a10disp will raise the following build failure with a mainline kernel:

cp: cannot stat '/home/autobuild/autobuild/instance-0/output-1/build/linux-5.17.15/include/video/sunxi_disp_ioctl.h': No such file or directory

Fixes:
 - http://autobuild.buildroot.org/results/1f2607d6adece4d5dfe17fbdb032a2d228fc030f

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-10-31 13:20:03 +01:00
Thomas Petazzoni
e1550ef755 toolchain/toolchain-buildroot: introduce BR2_TOOLCHAIN_BUILDROOT_NONE
In the internal toolchain backend, we have a choice..endchoice block
to allow the user to select the C library, between glibc, uClibc and
musl.

However, there are situations were no C library at all is
supported. In this case, the choice does not appear, and does not
allow to see the Config.in comments that are within the
choice..endchoice block and that may explain why no C library is
available.

For example, on RISC-V 32-bit, the only C library supported is glibc,
and the minimum kernel header version required by glibc on this
architecture is 5.4.0. In a future commit, we are going to add this
dependency on glibc (to fix build issues on configurations that have
headers < 5.4.0). But since glibc is the only supported C library on
RISC-V 32-bit, it means that the choice..endchoice for the C library
contains no entry, preventing from seeing the Config.in comment.

To address this issue, this commit adds a "dummy"
BR2_TOOLCHAIN_BUILDROOT_NONE option that shows up in the
choice..endchoice only when no C library is available. Thanks to this,
the choice..endchoice is never empty, and the Config.in comments can
be seen.

If the user keeps BR2_TOOLCHAIN_BUILDROOT_NONE selected, then the
build will anyway abort early because package/Makefile.in has a check
to verify that a C library is selected, and aborts the build if not.

Some could say that the problem should be resolved by instead
preventing the selection of headers < 5.4.0 on RISC-V 32-bit, but that
is difficult to do as the user can choose a custom header version, or
simply specific that (s)he wants to use the headers of the kernel
being built. In those situations, it's difficult to prevent selecting
headers < 5.4.0.

Prevent random configurations from triggering a build failure in our
autobuilders, by excluding that symbol from accepted configuration.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: update genrandconfig]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-30 12:42:41 +01:00
Yann E. MORIN
fa538315dc utils/genrandconfig: fix flake8
Commit af494d92d3 (utils/genrandconfig: disable libopenssl without
atomics) intorduced a code-style issue that flake8 does not like:

    $ make check-flake8
    utils/genrandconfig:253:8: E713 test for membership should be 'not in'
    1     E713 test for membership should be 'not in'

Fixes: af494d92d3
    https://gitlab.com/buildroot.org/buildroot/-/jobs/3045260108

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-09-18 17:09:30 +02:00
Fabrice Fontaine
af494d92d3 utils/genrandconfig: disable libopenssl without atomics
libopenssl needs atomic or the build will fail (e.g. on sparcv8 without
libatomic):

${LDCMD:-/nvmedata/autobuild/instance-7/output-1/host/bin/sparc-buildroot-linux-uclibc-gcc}  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -O0 -g2  -g2  -L.   \
	-o apps/openssl apps/asn1pars.o apps/ca.o apps/ciphers.o apps/cms.o apps/crl.o apps/crl2p7.o apps/dgst.o apps/dhparam.o apps/dsa.o apps/dsaparam.o apps/ec.o apps/ecparam.o apps/enc.o apps/engine.o apps/errstr.o apps/gendsa.o apps/genpkey.o apps/genrsa.o apps/nseq.o apps/ocsp.o apps/openssl.o apps/passwd.o apps/pkcs12.o apps/pkcs7.o apps/pkcs8.o apps/pkey.o apps/pkeyparam.o apps/pkeyutl.o apps/prime.o apps/rand.o apps/rehash.o apps/req.o apps/rsa.o apps/rsautl.o apps/s_client.o apps/s_server.o apps/s_time.o apps/sess_id.o apps/smime.o apps/speed.o apps/spkac.o apps/srp.o apps/storeutl.o apps/ts.o apps/verify.o apps/version.o apps/x509.o \
	 apps/libapps.a -lssl -lcrypto -ldl
/nvmedata/autobuild/instance-7/output-1/host/lib/gcc/sparc-buildroot-linux-uclibc/10.3.0/../../../../sparc-buildroot-linux-uclibc/bin/ld: ./libssl.so: undefined reference to `__atomic_fetch_sub_4'

It should be noted that openssl3 has added OPENSSL_DEV_NO_ATOMICS but
"this is intended for internal development only, to check the
refcounting is properly coded.  It should never become a configuration
option, hence the name of the macro.":
503d4745a1

Atomics are not available in Buildroot if:
 - architecture is 32 bit and something other than ARM or xtensa, and
 - GCC < 4.8 or no threads or FLAT.

The nothreads case can theoretically happen in many different
situations, but in practice nobody disables threads. So the only
interesting case is the FLAT case. Since ARM and RISC-V 64 both have
atomics intrinsics, that leaves just m68k NOMMU as FLAT. So this is
truly a corner case.

The proper solution would be to patch GCC to also provide libatomic in
those cases.
- For nothreads, atomics are in fact not needed, so libatomic can simply
  be implemented as stubs.
- For FLAT, it's probably just a matter of having a match to uclinux in
  libatomic/configure.tgt.

Again, though, this happens only in such niche cases that it's not worth
working on it.

Fixes:
 - http://autobuild.buildroot.org/results/bce526d538f43a541fdfbc0c9b4a7cecebbbc539

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-18 15:36:36 +02:00
Fabrice Fontaine
485aa2d691 utils/genrandconfig: add mxs-bootlets custom handling
Add custom cases to make sure that a random configuration with an empty
git or tarball location for mxs-bootlets doesn't fail. It reverts to
BR2_TARGET_MXS_BOOTLETS_FREESCALE in that case.

Fixes:
 - http://autobuild.buildroot.org/results/dcbeb73d152c79c18b1fa3bef3b4fa07635d7b36
 - http://autobuild.buildroot.org/results/f61cf3fc58db9ded5ec42ebf4a9847584700698d

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-06 21:58:49 +02:00
Arnout Vandecappelle
1fc299ad6a utils/genrandconfig: remove spurious semicolon
From flake8:
utils/genrandconfig:429:21: E703 statement ends with a semicolon
1     E703 statement ends with a semicolon

Fixes: d3e029575c

Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-08-24 13:21:39 +02:00
Fabrice Fontaine
fee46b54e7 utils/genrandconfig: add aufs-util handling
Add a custom case to make sure that a random configuration with an empty
version for aufs-util doesn't fail.

Fixes:
 - http://autobuild.buildroot.org/results/e242cf66a02983bcf6e95b37f8e458bd18aee683

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-24 13:16:27 +02:00
Fabrice Fontaine
d3e029575c utils/genrandconfig: improve ARM trusted firmware fixup
Improve commit 541e794a95 by adding a
custom case to make sure that a random configuration with an empty
platform for arm-trusted-firmware doesn't fail:

make_helpers/plat_helpers.mk:15: *** "Error: Unknown platform. Please use PLAT=<platform name> to specify the platform".  Stop.

Fixes:
 - http://autobuild.buildroot.org/results/1b67220008223d1bcbe70b76d643f9d04362ba6b

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-23 23:55:49 +02:00
Yann E. MORIN
b6bfa3f744 utils/genrandconfig: dump traceback for unhandled exceptions
In case of an unexpected error, we currently only print the exception as
an str(). For example, the recent issue with the glibc version check
only reported:
    TypeError: cannot use a string pattern on a bytes-like object

That does not help in fixing the issue; the exception text is also not
usually very user-friendly either anyway.

We change the reporting to print the traceback, which in the glibc
version check mentioned above, the error is reported as:

    Traceback (most recent call last):
      File "./utils/genrandconfig", line 740, in <module>
        ret = gen_config(args)
      File "./utils/genrandconfig", line 676, in gen_config
        if not is_toolchain_usable(configfile, toolchainconfig):
      File "./utils/genrandconfig", line 186, in is_toolchain_usable
        if StrictVersion('2.14') > StrictVersion(glibc_version):
      File "/usr/lib/python3.8/distutils/version.py", line 40, in __init__
        self.parse(vstring)
      File "/usr/lib/python3.8/distutils/version.py", line 135, in parse
        match = self.version_re.match(vstring)
    TypeError: cannot use a string pattern on a bytes-like object

With this, the error is much easier to pinpoint (it's the last one that
is not in a system module).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-23 22:19:52 +02:00
Yann E. MORIN
12e4f7c5c4 utils/genrandconfig: fix checking host glibc version
Unless explicitly told otherwise, subprocess.check_output() returns
bytes objects [0].

When we try to check the C library version (to check the Linaro
toolchain is usable), genrandconfig currently fails with:
    TypeError: cannot use a string pattern on a bytes-like object

So, as suggested in the python documentation, decocde() the output of
subprocess.check_output() before we can use it.

[0] https://docs.python.org/3/library/subprocess.html#subprocess.check_output

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-23 22:19:45 +02:00
Fabrice Fontaine
2d3d130bc1 utils/genrandconfig: add opensbi custom handling
Add custom cases to make sure that a random configuration with an empty
git, tarball location or version for opensbi doesn't fail. It reverts to
BR2_TARGET_OPENSBI_LATEST_VERSION in that case.

Fixes:
 - http://autobuild.buildroot.org/results/b31882b1a7119b2b590cc6dfed44fe515a63a6b8
 - http://autobuild.buildroot.org/results/61866e8941741c9746dcb2c60b6db3b0e5b9a47b

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-20 11:31:56 +02:00
Fabrice Fontaine
2bf6b06308 utils/genrandconfig: add xenomai custom handling
Add custom cases to make sure that a random configuration with an empty
git, tarball location or version for xenomai doesn't fail. It reverts to
BR2_PACKAGE_XENOMAI_LATEST_VERSION in these cases.

Fixes:
 - http://autobuild.buildroot.org/results/19419759b4add0197b4e629d0b2216c2a07624b2

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-08-15 18:45:00 +02:00
Fabrice Fontaine
3ce97967e0 utils/genrandconfig: fix indentation
Commit 0f3115b2ca broke the indentation
(missing one space)

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-09 13:32:24 +02:00
Fabrice Fontaine
0f3115b2ca utils/genrandconfig: add refpolicy custom git handling
Add a custom case to make sure that a random configuration with an empty
git location for refpolicy doesn't fail. It reverts to
BR2_PACKAGE_REFPOLICY_UPSTREAM_VERSION in that case.

Fixes:
 - http://autobuild.buildroot.org/results/98da07a1f9b88e2e8a028a55c8cf76c6ebc28304

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-08 22:48:29 +02:00
Kory Maincent
d780b828fc utils/genrandconfig: add optee-os custom tarball handling
Add a custom case to make sure that a random configuration with an
empty tarball location for OP-TEE OS doesn't fail. It reverts to
BR2_TARGET_OPTEE_OS_LATEST in that case.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-07-06 23:36:34 +02:00
Thomas Petazzoni
793ee1011e Remove support for the NDS32 architecture
The support for this architecture has been removed from the upstream
Linux kernel, as of commit:

  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=aec499c75cf8e0b599be4d559e6922b613085f8f

Which states:

  The nds32 architecture, also known as AndeStar V3, is a custom
  32-bit RISC target designed by Andes Technologies. Support was added
  to the kernel in 2016 as the replacement RISC-V based V5 processors
  were already announced, and maintained by (current or former) Andes
  employees.

  As explained by Alan Kao, new customers are now all using RISC-V,
  and all known nds32 users are already on longterm stable kernels
  provided by Andes, with no development work going into mainline
  support any more.

There has also been little to no maintenance done in Buildroot for
this architecture in recent times, so let's follow the Linux kernel
community decision and drop support for this CPU architecture.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-06-19 18:30:45 +02:00
James Hilliard
541e794a95 utils/genrandconfig: add ARM trusted firmware fixup
Fixes:
boot/arm-trusted-firmware/arm-trusted-firmware.mk:200: *** No repository specified. Please check BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_URL.  Stop.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-18 21:16:01 +02:00
James Hilliard
fcca757443 utils/genrandconfig: add DTS cuimage fixup
If we don't have a DTS name or path we need to also disable cuimage
to disable DTS support as BR2_LINUX_KERNEL_CUIMAGE requires DTS
support.

Fixes:
linux/linux.mk:591: *** No kernel device tree source specified, check your BR2_LINUX_KERNEL_INTREE_DTS_NAME / BR2_LINUX_KERNEL_CUSTOM_DTS_PATH settings.  Stop.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-04 21:38:10 +02:00
James Hilliard
57926b025b utils/genrandconfig: only fixup host uboot tools source when empty
If the kconfig values being replaced are not empty we should not
try to fix them.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-03 22:22:04 +02:00
James Hilliard
fe740dcac8 utils/genrandconfig: use high quality random for KCONFIG_SEED
Internally genrandconfig will use gettimeofday when generating a
KCONFIG_SEED, since autobuild-run executes genrandconfig at the same
time for multiple autobuilder runners this could potentially result
in the same KCONFIG_SEED being generated for those test runs started
at the same time.

To ensure this doesn't happen set the KCONFIG_SEED using the urandom
entropy source.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-03 22:20:18 +02:00
James Hilliard
71c2a29e5e utils/genrandconfig: fix ext2 config variable
This should have been BR2_TARGET_ROOTFS_EXT2 as the
BR2_TARGET_ROOTFS_EXT2_GEN variable is an integer variable used
to indicate the ext2/ext3/ext4 variant.

Fixes:
 - http://autobuild.buildroot.net/results/5d7/5d7833212bd8a2b575945f848d8c91dabe7d3c30

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-02 23:40:23 +02:00
James Hilliard
3c98eb26db utils/genrandconfig: increase default target rootfs size to 5G
Since there isn't a way to reliably compute the required size lets
just set it to what should be a high enough value to not run out
of space.

Both ext2 and fsfs create sparse files, so this is not a waste of disk
space.

Fixes:
 - http://autobuild.buildroot.net/results/a3c68cdc515353c0d2650ee743aa3bd8ab99a418
 - http://autobuild.buildroot.net/results/ad426ff903d78a92de8f0208876e2196136a09a8

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-04-30 20:40:21 +02:00
Thomas Petazzoni
04ea3d38dd arch: drop support for SH2A
Until commit "arch/Config.in.sh: fixup MMU selection" in this series,
SH2A could either be used with BR2_USE_MMU disabled or BR2_USE_MMU
enabled.

The later made absolutely no sense, since SH2A does not have a MMU:
MMU support was introduced starting from SH3 according to
https://en.wikipedia.org/wiki/SuperH#SH-3

Also, since commit 22d5501e03 ("arch:
tidy up binary formats config"), which was merged in Buildroot
2015.05, the architecture tuple used when BR2_sh2a=y and BR2_USE_MMU
disabled is sh2a-buildroot-uclinux-uclibc, and this was already
unsupported back in the days of Buildroot 2015.08 and binutils 2.24,
causing the build to fail with:

*** BFD does not support target sh2a-buildroot-uclinux-uclibc.

just like it fails to build today with recent version of binutils.

So, this has been broken since 2015.08, and nobody complained. SH2A is
seldom used, so it's time to kill it.

It is worth mentioning that there had been an attempt at resurrecting
SH2 support around 2015 (see https://lwn.net/Articles/647636/) as part
of the J2 core. This effort led to the addition of FDPIC support for
SH2A in the musl C library (and therefore proper ELF binaries, with
shared libraries), but that was never supported in Buildroot. Now that
the J2 project is essentially dead, there is no reason to bother with
this.

Fixes:

  http://autobuild.buildroot.net/results/63d01d33ae30f86b63b9f42a9fea116f2f3e9005/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-04-24 10:38:52 +02:00
James Hilliard
aeee90ec10 utils/genrandconfig: add randconfig based --no-toolchains-csv option
Currently we only test a limited set of toolchains that are mostly
prebuilt, add a flag to allow using randconfig for randomizing
additional toolchain settings instead of randpackageconfig.

To avoid invalid configs we need to add additional config validation
filtering and fixups. Although the loop around 'make randconfig'
attempts does make sure we eventually end up with a valid configuration,
there is a bias towards e.g. disabling the kernel. It would be possible
to e.g. always force BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y, but
that removes the possibility for a toolchains-csv config to add a valid
custom defconfig. (Note that currently this isn't possible anyway since
'make randpackageconfig' is used if a toolchains-csv is provided, but
eventually we want to use 'make randconfig' also if a toolchains-csv is
provided.)

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-04-09 18:45:03 +02:00
Peter Korsgaard
6c00a319ac utils/genrandconfig: drop BR2_PACKAGE_PYTHON_PY_ONLY logic
The option was removed by commit 306839586b (package/python: drop target
package).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-02-12 21:39:43 +01:00
Thomas Petazzoni
0e64537917 utils/genrandconfig: reduce the maximum "size" of random configurations
genrandconfig is used by the Buildroot autobuilders to generate
semi-random configurations that we build test. As part of this, we use
"make randpackageconfig" to randomize the selection of packages,
together with a KCONFIG_PROBABILITY value, which indicates the
probabibility for each option to be enabled. This probability is
itself randomized, between 1% and 30% for every build.

However, with our increasing number of packages (over 2900), when we
use a 30% probability for options to be enabled, it means a *lot* of
options are enabled, causing very large configurations to be
tested. These configurations are not very realistic, and they take
ages to build on our autobuilders: we have builds that take 4, 5 or
even 7 hours to build.

In order to test a larger number of configurations and therefore a
larger variety of configurations, this commit reduces the maximum
probability to 20%.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-11-14 13:50:50 +01:00
Bernd Kuhls
8d469858ce utils/genrandconfig: test configurations with BR2_PACKAGE_PYTHON3_PY_ONLY
Python3 variant of
https://git.buildroot.net/buildroot-test/commit/?id=c0de21d9530af53eae5588d99d90c7e0cb87c543

to support Kodi 19 which depends on python3:
https://git.buildroot.net/buildroot/commit/?id=148e695e37561fe45d4726cb68f6454464d17797

Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-10-18 22:10:08 +02:00
Petr Vorel
35f15db30a support: utils: use python3 explicitly
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).

Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import

Tested with python3 -m py_compile.

[1] https://www.python.org/doc/sunset-python-2/

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-09-22 21:30:24 +02:00
Peter Korsgaard
8d07baab43 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-06-07 17:14:37 +02:00
Thomas De Schampheleire
7c9dfe4fd9 utils/genrandconfig: also test BR2_ENABLE_RUNTIME_DEBUG
... similar to BR2_ENABLE_DEBUG.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-01 22:40:12 +02:00
Matthew Weber
4e55bc8a4d utils/genrandconfig: drop hardening Config enables
Since 810ba387be, some form of these options are enable
by default. Specifically:

- Kept FORTIFY level 2 option as the default is now level 1.
- Removed all SSP options as the default now uses the best
  option based on toolchain support.
- Similar to SSP, for RELRO, the default now uses the best
  option based on toolchain support.
- Completely drop PIC PIE as it defaults =y

Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-24 15:29:26 +02:00
Matt Weber
3930fd2ddd genrandconfig: uboot-tools env/scr creation test files
Normally the kconfig stings would end up empty and cause a build
error.  This patch provides test files to allow testing the creation
of uboot environment and script bin files from user provided txt files.

Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2020-10-06 23:37:48 +02:00
Peter Korsgaard
06417e97e3 utils/genrandconfig: drop outdated python-nfc check
Commit 9ea528f84b (package/python-nfc: bump to version 0.13.5) changed the
python-nfc package to download from github, so the package no longer needs
bzr on the host.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-04 19:08:12 +01:00