Use the function 'parse_developers' function from getdeveloperlib that
collect the information about the developers and the files they
maintain. Then set the maintainer(s) to each package.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Remove the patch_count attribute and use a class property instead.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch changes the type of the latest_version variable to a dict.
This is for better readability/usability of the data. With this the json
output is more descriptive in later processing of the json output.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
During the CVE checking phase, we can still see a huge amount of
Python processes (actually 128) running on the host, even though
the CVE step is entirely ran in the main thread.
These are actually the worker processes spawned to check for the
packages URL statuses and the latest versions from release-monitoring.
This is because of an issue in Python's multiprocessing implementation:
https://bugs.python.org/issue34172
The problem was already there before the CVE matching step was
introduced, but because pkg-stat was terminating right after the
release-monitoring step, it went unnoticed.
Also, do not hold a reference to the multiprocessing pool from
the Package class, as this is not needed.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In Python 3, the functions from the subprocess module return bytes
(and no longer strings as in Python 2), which must be decoded for
further text operations.
Now, pkg-stats can be run in Python 3.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
It seems like throughout the series that the CVE pkg-stats support
went through, the support for ignoring CVEs in the per-package
<pkg>_IGNORE_CVES variable was forgotten.
Let's re-introduce this, which is now very simple thanks to the CVE
class, its .identifier() propertly and the .is_cve_ignored() method of
the Package class
Cc: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
With classpath removed, no packages select these symbols any more - So drop
them and their corresponding logic in dependencies.sh / genrandconfig.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
During the CVE checking phase, we can still see a huge amount of
Python processes (actually 128) running on the host, even though
the CVE step is entirely ran in the main thread.
These are actually the worker processes spawned to check for the
packages URL statuses and the latest versions from release-monitoring.
This is because of an issue in Python's multiprocessing implementation:
https://bugs.python.org/issue34172
The problem was already there before the CVE matching step was
introduced, but because pkg-stat was terminating right after the
release-monitoring step, it went unnoticed.
Also, do not hold a reference to the multiprocessing pool from
the Package class, as this is not needed.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In Python 3, the functions from the subprocess module return bytes
(and no longer strings as in Python 2), which must be decoded for
further text operations.
Now, pkg-stats can be run in Python 3.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
While investigating [1] one units failed due to missing kernel option
CONFIG_BINFMT_MISC needed by "proc-sys-fs-binfmt_misc.mount" service.
It's because the kernel support autofs4 but not MISC binaries.
Since the systemd test infra use the default defconfig (vexpress),
we need to provide a linux fragment to enable CONFIG_BINFMT_MISC.
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/454255917
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
[yann.morin.1998@free.fr:
- move the kernel config with the others in conf/
]
Tested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The NVD files that are used to build the list of CVEs affecting
Buildroot packages are quite large (a few hundreds MB of json),
and cause the pkg-stats scripts to have a huge memory footprint
(a few GB with Python 2.7).
However, because we only need to iterate on CVE items one by one,
we can process them in streaming (ie decoding one CVE at a time
from the JSON representation). Because the json module from the
python standard library does not support such a mode of operation,
we switch to the third-party package ijson, which is compatible
with both Python 2 and Python3.
To run the script with these modifications, one should install
the ijson python package. This can be done with pip:
`pip install ijson`. On Debian based distributions, this can
also be done with the apt package manager:
`apt install python-ijson`.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The NVD files that are used to build the list of CVEs affecting
Buildroot packages are quite large (a few hundreds MB of json),
and cause the pkg-stats scripts to have a huge memory footprint
(a few GB with Python 2.7).
However, because we only need to iterate on CVE items one by one,
we can process them in streaming (ie decoding one CVE at a time
from the JSON representation). Because the json module from the
python standard library does not support such a mode of operation,
we switch to the third-party package ijson, which is compatible
with both Python 2 and Python3.
To run the script with these modifications, one should install
the ijson python package. This can be done with pip:
`pip install ijson`. On Debian based distributions, this can
also be done with the apt package manager:
`apt install python-ijson`.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
It seems like throughout the series that the CVE pkg-stats support
went through, the support for ignoring CVEs in the per-package
<pkg>_IGNORE_CVES variable was forgotten.
Let's re-introduce this, which is now very simple thanks to the CVE
class, its .identifier() propertly and the .is_cve_ignored() method of
the Package class
Cc: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reorder imports using the isort utility to fix a warning from pylint3:
wrong-import-order: standard import "import multiprocessing" should be
placed before "import nose2"
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
According to PEP8 empty sequences should be checked as booleans.
Fixes the following PEP8 warning:
Do not use `len(SEQUENCE)` to determine if a sequence is empty
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit extends the pkg-stats script to grab information about the
CVEs affecting the Buildroot packages.
To do so, it downloads the NVD database from
https://nvd.nist.gov/vuln/data-feeds in JSON format, and processes the
JSON file to determine which of our packages is affected by which
CVE. The information is then displayed in both the HTML output and the
JSON output of pkg-stats.
To use this feature, you have to pass the new --nvd-path option,
pointing to a writable directory where pkg-stats will store the NVD
database. If the local database is less than 24 hours old, it will not
re-download it. If it is more than 24 hours old, it will re-download
only the files that have really been updated by upstream NVD.
Packages can use the newly introduced <pkg>_IGNORE_CVES variable to
tell pkg-stats that some CVEs should be ignored: it can be because a
patch we have is fixing the CVE, or because the CVE doesn't apply in
our case.
>From an implementation point of view:
- A new class CVE implement most of the required functionalities:
- Downloading the yearly NVD files
- Reading and extracting relevant data from these files
- Matching Packages against a CVE
- The statistics are extended with the total number of CVEs, and the
total number of packages that have at least one CVE pending.
- The HTML output is extended with these new details. There are no
changes to the code generating the JSON output because the existing
code is smart enough to automatically expose the new information.
This development is a collective effort with Titouan Christophe
<titouan.christophe@railnova.eu> and Thomas De Schampheleire
<thomas.de_schampheleire@nokia.com>.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since [1], the GLX support is enabled by BR2_PACKAGE_MESA3D_OPENGL_GLX
symbol.
Since [2], only one swrast provider can be built.
Keep BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/400391349
[1] 5cb821d563
[2] 09a0a28507
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Most, but not all our C code follows the Linux kernel code style (as
documented in Documentation/process/coding-style.rst). Adjust the few
places doing differently:
- Braces:
..but the preferred way, as shown to us by the prophets Kernighan
and Ritchie, is to put the opening brace last on the line
- Spaces after keywords:
Use a space after (most) keywords
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When Buildroot is released, it knows up to a certain kernel header
version, and no later. However, it is possible that an external
toolchain will be used, that uses headers newer than the latest version
Buildroot knows about.
This may also happen when testing a development, an rc-class, or a newly
released kernel, either in an external toolchain, or with an internal
toolchain with custom headers (same-as-kernel, custom version, custom
git, custom tarball).
In the current state, Buildroot would refuse to use such toolchains,
because the test is for strict equality.
We'd like to make that situation possible, but we also want the user not
to be lenient at the same time, and select the right headers version
when it is known.
So, we add a new Kconfig blind option that the latest kernel headers
version selects. This options is then used to decide whether we do a
strict or loose check of the kernel headers.
Suggested-by: Aaron Sierra <asierra@xes-inc.com>
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- only do a loose check for the latest version
- expand commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This also adds the new tests to the gitlab CI configuration.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Validates an archive can be installed and removed
- Builds an archives that uses postinst and prerm scripts
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The README file saved by legal-info does not mention the host package
variant of the saved material. Add them.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The test-case for python-gitdb2 consists solely in verifying that the
module can indeed be imported.
However, flake8 errors out on unused imports. Furthermore, it also
errors about wildcard imports, as it can detect unused symbols.
Commit d8c86be9cd (support/testing: fix python-gitdb2 test) tried to
address this issue, by explicitly squelching the two errors, F401 and
F403.
While that works on recent distros, the image used by our docker
pipeline is laggign behind and the flake8 there only handles at most a
single error in the noqa list.
Do as is done with the other python samples, and just blindly ignore
all errors.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
In commit aee39cbf27 (arch/riscv: set the default float ABI based on
ISA extensions), the default ABI changed, so the config fragments used
by the autobuilders were adapated accordingly, in commit f89871e810
(support/config-fragments: fix br-riscv{32,64} toolchain fragments).
But now, we need to revert again, because the newer toolchains are now
using the default ABI again.
We do not really do a revert, though, because the original change was
right, and a revert would mean it was not.
Fixes:
http://autobuild.buildroot.org/results/b59/b593267fb9fc9a002b977e049b2a5389dbaded30/ (riscv32)
http://autobuild.buildroot.org/results/b42/b42a4b22b29f47d5c85be119b310f1dfb61112a1/ (riscv64)
... and so many others on various packages...
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Mark Corbin <mark.corbin@embecosm.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Simple bump of the toolchain components. For nios2, the toolchain now
has SSP support as well.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- fix version it commit title
- mention SSP for nios2
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
To generate a reproducible archive from a svn repository mainly the same
aproach is done like for the archives from a git repository.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[yann.morin.1998@free.fr: get the date of the revision]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This adds a test case for python-avro, with a script that
performs a simple deserialization.
Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cthe test-case for python-gitdb2 consists solely in verifying that the
module can indeed be imported.
However, flake8 errors out on unused imports. Furthermore, it also
errors about wildcard imports, as it can detect unused symbols.
Squelch those errors.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Now that we have the EXTRACTOR_PKG_DEPENDENCY.* variables available,
we can use them to implement extractor-system-dependency: if for a
given archive type, the corresponding EXTRACTOR_PKG_DEPENDENCY.<type>
variable is empty, then it means we need the corresponding extractor
tool to be provided by the system.
Following this, EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS is no
longer used, so we can drop it from support/dependencies/.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Add a simple test case that imports the module.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module and instantiates a new
SlidingWindowMapManager class.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This reverts commit 6f35d96756.
Repeat after me: on the master branch you will not work. On the master
branch you will not work.
This definitely shouldn't have been pushed. Sorry about that.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Running "git fetch origin ${cset}:${cset}" to create a local ref
${cset} from the remote ref ${cset} causes Git to issue a warning like
the below, when the version is a full commit hash:
===
warning: refname '49eb4ecb1ef9879ebc6789a1bdb536ab2b1d9871' is ambiguous.
Git normally never creates a ref that ends with 40 hex characters
because it will be ignored when you just specify 40-hex. These refs
may be created by mistake. For example,
git switch -c $br $(git rev-parse ...)
where "$br" is somehow empty and a 40-hex ref is created. Please
examine these refs and maybe delete them. Turn this message off by
running "git config advice.objectNameWarning false"
===
This warning is very confusing for users, and is caused by the fact
that Git doesn't like our local ref name to look like a commit hash.
So, this commit proposes to fix the issue by having the local ref
named buildroot-${cset}, i.e
buildroot-${version-specified-by-the-package}.
The generated tarballs are exactly identical, nothing changes, it is
really just internally the local ref we are using to checkout the
correct version that is different. And it avoids the confusing
warning.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit c4e6d5c8be ("core: implement
per-package SDK and target") had a mistake on the regexp that is used
to match $(PER_PACKAGE_DIR)/<something>/, and due to this, the regexp
was never matched.
The + sign in [^/]+ which was suggested by Yann E. Morin during the
review of the per-package patch series (instead of [^/]*) needs to be
escaped to be taken into account correctly. Without this, the regexp
doesn't match, and the replacement is not done, causing:
(1) For the libtool fixup in pkg-generic.mk, the lack of replacement
causes libtool .la files to not be tweaked as expected, which it
turn causes build failures reported by the autobuilder.
(2) For the fix-rpath, the RPATH of host binaries in the SDK were not
correct.
Interestingly, we have the same regexp in
support/scripts/check-host-rpath, but here the + sign does not need to
be escaped.
Fixes:
http://autobuild.buildroot.net/results/d4d996f3923699e266afd40cc7180de0f7257d99/ (libsvg-cairo)
http://autobuild.buildroot.net/results/56330f86872f67a2ce328e09b4c7b12aa835a432/ (bind)
http://autobuild.buildroot.net/results/9e0fc42d2c9f856b92954b08019b83ce668ef289/ (ibrcommon)
and probably a number of other similar issues
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test comprises of four simple steps:
1: Start a new simple project called testsite.
2: Run ./manage.py migrate on the new testsite.
3: Run ./manage.py runserver 0.0.0.0:1234 & sleep 30
- The sleep 30 is necessary as it may take several seconds for
the django server to fully start.
4: Run netstat to ensure the server opened port 1234.
Signed-off-by: Adam Duskett <aduskett@greenlots.com>
[Thomas: use self.assertRunOk() when appropriate]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fix these warnings:
E122 continuation line missing indentation or outdented
E127 continuation line over-indented for visual indent
E265 block comment should start with '# '
E302 expected 2 blank lines, found 1
F401 'pexpect' imported but unused
Fixes:
- https://gitlab.com/buildroot.org/buildroot/-/jobs/360824861
Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The test starts a simple container with an iperf3 server.
The container is using the tini init system, with a shared rootfs.
An iperf3 client is started from the host to check that the container
is really up and running.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This method asserts that the given command ran successfully.
The goal is for it to be used by the different tests when needed.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This is required by wpewebkit and webkitgtk.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Python 2.7 will not be maintained past 2020.
Many scripts on the tree are used during the build and should keep
Python 2 compatibility for a while.
This is not the case for the runtime test infra. It's meant to be run in
modern distros only, so it can safely switch to support Python 3 only.
An advantage of this approach is to have less scenarios to test in.
Otherwise every change to the test infra or runtime tests would need to
be tested against both versions of the interpreter, increasing the
effort of the developers, to ensure the compatibility to Python 2 was
not broken.
In order to accomplish the change to Python 3:
- change the shebang for run-tests;
- use Python 3 urllib as a drop-in replacement for Python 2 urllib2;
- when writing the downloaded binary files, explicitly open the output
file as binary;
- when subprocess is used to retrieve the text output from commands,
explicitly ask for text output. For this, use 'universal_newlines'
because 'text' was added only on Python 3.7;
- when pexpect is used to retrieve the text output from qemu or git,
explicitly ask for text output using 'encoding';
- the code using csv currently follows the example in the documentation
for the Python 2 module, change it to follow the example in the
documentation for the Python 3 module;
- fix the relative import for test_git.py to be Python 3 compliant.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The test infra will soon be converted to Python 3 only.
So add the interpreter and also the Python 3 variant of modules nose2
and pexpect to the docker image used to run runtime tests.
Keep the Python 2 variant of those modules to allow a gradual
transition.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
genimage makes a full copy of the given rootpath to ${GENIMAGE_TMP}/root
so passing TARGET_DIR would be a waste of time and disk space. We don't
rely on genimage to build the rootfs image, just to insert a pre-built
one in the disk image.
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, we only require a gcc 4.4 version, which now is pretty old
(released in April 2009). This requirement is not even tested nowadays,
with our oldest autobuilder having a 4.7 version only.
And even then, 4.7 is still old enough that it prevents us from
upgrading some packages. For example cmake 3.10+ requires C++11
constructs that were only added in gcc 4.8 (when C++11 support was
finally completed in gcc).
So, update our requirements for gcc to at least 4.8.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We no longer have anything that needs it during the build, so we don't
require it anymore.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Back a few years ago, when we were starting to think about top-level
parallel build, we were not sure how to deal with packages that
installed the same files, so we wanted to catch the situation to assess
how prevalent that was, before we decided what to do and how to address
it.
However, the trend nowadays is that packages will install in a
per-package target/ (and staging/ and host/), and the final directories
will be assembled in a reproducible (alphabetical) order, so if two
packages install the same file, the last one will win (as is currently
the case).
Besides, check-uniq-files reports loads of spurious errors when packages
get reinstalled (e.g. during development).
Finally, check-uniq-files is the only script called during the build,
that is written in python.
So, get rid of check-uniq-files.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes https://gitlab.com/buildroot.org/buildroot/-/jobs/332656041
The recent bump of a number of python packages broke docker-compose, as
docker-compose specifies both minimum and maximum versions for (most of) its
dependencies:
Dependencies of docker-compse 1.20.1 (! = unmet):
cached-property: < 2 (currently 1.51)
docopt: < 0.7 (currently 0.6.2)
! pyyaml: < 4.0, patched to < 4.3 (currently 5.1.2)
requests: < 2.19, patched to < 3 (currently 2.22.0)
! texttable: < 0.10 (currently 1.6.2)
websocket-client: < 1.0 (currently 0.56.0)
! docker: < 4.0 (currently 4.1.0)
dockerpty: < 0.5 (currently 0.4.1)
six: < 2 (currently 1.12.0)
jsonschema: < 3 (currently 2.5.1)
enum34: < 2 (currently 1.1.6)
backports.ssl-match-hostname: >= 3.5 (currently 3.7.0.1)
ipaddress: >= 1.0.16 (currently 1.0.23)
To fix this, bump docker-compose to the most recent release (1.24.1). This
is unfortunately not enough, as our docker, pyyaml, requests and texttable
packages are too new, so add 3 patches from upstream to relax the version
checks of dependencies. Notice that patch 0003 is from
https://github.com/docker/compose/pull/6623 and has not been merged yet.
Discussions around the problem of these maximum versions of the dependencies
and the fact that all downstream users have to patch it is ongoing here:
https://github.com/docker/compose/issues/6756
docker-compose 1.24.1 added a requirement for ssh support in python-docker in:
7b82b2e8c7
So add a dependency for python-paramiko and update the toolchain dependency
for C++ (from python-paramiko -> python-cryptography) and adjust the
toolchain configuration of the runtime test to match.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
There is no need for double grep, so choose a better regexp. Use &&
instead of ; between commands so the sequence of commands fail faster.
Break the last sequence of commands in 2 calls run() so the proper
return code can be tested for each.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When selected, host-ccache is a dependency of almost all packages.
As such, it clutters the dependency graph uselessly.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The POSIX specification defines a 'trap <action> EXIT' mechanism that is
useful to perform clean-up actions in shell scripts. A trap has two main
advantages over hand-crafted clean-up mechanisms:
- It runs even if the process is terminated by a SIGTERM.
- It runs even if the script stops due to a pipeline failure (set -e).
Now we can make the script to stop immediately if a compilation error
occurs, instead of letting it try to run an unexisting program.
This change may appear to be overkill but Buildroot is an open source
project and each piece of code is a potential learning tool for other
developments. We must strive to provide good examples.
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Acked-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some installations mount /tmp with the 'noexec' option, which prevents
running the program generated there to check the kernel headers.
Avoid the problem by generating the program under $(BUILD_DIR), passed
as the first argument to check-kernel-headers.sh.
We could globally export a TMPDIR environment variable with some path
under $(BUILD_DIR) but such solution would be too intrusive, depriving
the user from the freedom to set TMPDIR at his will (or needs).
Fixes: https://bugs.busybox.net/show_bug.cgi?id=12241
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since commit aee39cbf27 ("arch/riscv:
set the default float ABI based on ISA extensions"), RISC-V 32/64 use
the lp32d/lp64d ABIs by default. But our pre-built external toolchains
were built with the LP32/LP64 ABI.
Building with lp32d/lp64d gcc flags, but a toolchain built with the
LP32/LP64 ABI causes a number of failures such as:
/home/mark/buildroot-test/instance-1/output/host/riscv64-buildroot-linux-gnu/sysroot/usr/include/gnu/stubs.h:11:11: fatal error: gnu/stubs-lp64d.h: No such file or directory
or:
/home/mark/buildroot-test/instance-1/output/host/opt/ext-toolchain/bin/../lib/gcc/riscv64-buildroot-linux-gnu/7.4.0/../../../../riscv64-buildroot-linux-gnu/bin/ld: /tmp/cc2BTtFE.o: can't link hard-float modules with soft-float modules
/home/mark/buildroot-test/instance-1/output/host/opt/ext-toolchain/bin/../lib/gcc/riscv64-buildroot-linux-gnu/7.4.0/../../../../riscv64-buildroot-linux-gnu/bin/ld: failed to merge target specific data of file /tmp/cc2BTtFE.o
So let's fix our config fragments to reflect the ABIs those toolchains
were built with.
Fixes:
http://autobuild.buildroot.net/results/a3959b0613cf561059483abc580b144be4817d1a/ (libsepol)
http://autobuild.buildroot.net/results/3db50d8a0a913413b2198d6c301419136d2d22a7/ (attr)
http://autobuild.buildroot.net/results/7780fada05b8440ae3e97618615624a6a2dac03f/ (libusb)
and many others
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since release 18.12 binaries-marvell repository provides
common firmware supporting both A7K and A8K SoC families.
This commit bumps package version to 18.12 and removes
platform specific binary selections from Config.in.
Single firmware image suitable for both A7K and A8K
platforms is now specified in mk file explicitely.
Legacy handling is not needed, as configs which did have
the option set will continue to work without change.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
[Arnout:
- Remove BINARIES_MARVELL_IMAGE entirely;
- Add remark about legacy handling;
- Remove the deprecated option from the defconfigs and test that use
it.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The JSON::PP Perl module is used at build time by the webkitgtk and
wpewebkit packages.
Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As suggested by Baruch Siach, using "git rev-parse HEAD" is a lot
simpler than playing around with "git log" to just retrieve the commit
id corresponding to the current HEAD.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
pkg-stats extracts the Buildroot commit id from which the package
information was collected. However, when doing so, it always assumes
we're using the master branch, by running "git log master".
But in fact, pkg-stats can be run from any branch/tag, so it makes a
lot more sense to use "git log HEAD".
Cc: victor.huesca@bootlin.com
Cc: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Now that we can order packages from biggest to smallest, it makes sense
to assign the most aggressive colours to the biggest packages.
As such, reorder the current colours so that we have, in order:
- red-ish
- orange-ish
- yellow-ish
- purple-ish
- eggplant-ish (is that even a colour? :-] )
- some-indeterminate-blue-ish
- dark-green-ish
- light-green-ish
For the previous, smallest-first ordering, it does not matter much what
the ordering is: the actual colours are still somewhat-unpredictably
assigned to packages, depending on the cut-off limit...
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, the packages are sorted smallest first, and biggest last
(with unknown and others second-to-last and last, resp.).
Add an option to invert the ordering (but keeping unknown and others at
their current positions).
This has the nice side effect that we can now control the colours
assigned to the biggest package(s), as the colours are cycled from the
first to the last. Currently, the biggest packages gets a redish colour,
which is appropriate, but the second gets a greenish one, which is not
as appropriate (but changing that can come later).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When dealing with embedded devices, storage is more often than not some
kind of flash device, on which the memory is usually counted as powers
of 1024 instead of powers of 1000. As such, people may prefer reports
using IEC prefixes [0] instead of the SI prefixes.
Add an option to that effect.
We use argparse's ability to use custom actions [1] [2], to provide a
set of options that act on a boolean, but has a single help entry and
internally ensures consistency of the settings. We could have been using
the more conventional store_true/store_false actions instead, but that
would have meant either two help entries, one for each set of options,
and/or some logic after parse_args() to check the validity of the
settings.
[0] https://en.wikipedia.org/wiki/Binary_prefix
[1] https://docs.python.org/2/library/argparse.html#action
[2] https://docs.python.org/2/library/argparse.html#argparse.Action
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, we group packages that contribute less then 1%, into the
"Other" category.
However, in some cases, there can be a lot of very comparatively small
packages, and they may not exceed this limit, and so only the "Others"
category would be displayed, which is not nice.
Conversely, if there are a lot of packages, most of which only so
slightly exceeding this limit, then we get all of them in the graph,
which is not nice either.
Add a way for the developers to pass a different cut-off limit. As for
the dependency graph which has BR2_GRAPH_DEPS_OPTS, add the environment
variable BR2_GRAPH_SIZE_OPTS to carry those extra option (in preparation
for more to come, later).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Arnout:
- remove empty base class definition from Config;
- use parser.error instead of ValueError for invalid argument.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, we forcibly report sizes in multiple of Kilobytes. In some
big configurations, the sizes of the system as a whole, as well as that
of individual packages, may exceed megabytes, and when some artistic
assets get used, even the gigabyte may get exceed.
These big sizes are not easy to read when expressed in kilobytes.
Additionally, some very small packages might have sizes below the
kilobyte (and when we can specify the cut-off grouping size, they may
get reported), and thus the size displayed for those would be 0 kB.
Add a helper function that can format a floating-point size into a
string with all the appropriate formatting:
- there are at least 3 meaningfull digits visible, i.e. we display
"3.14" or "10.4" instead of just "3" or "10", but for big number we
don't care about too many precision either, so we report "100" or
"1000", not "100.42" or "1000.27";
- the proper SI prefix is appended, if needed.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently, the "unknown" category may be reported anywhere, so it does
not really stand out when there are a lot of packages in the graph.
Move it towards the end, but right before the "other" category, so that
it is a bit more visible. Like for Others, don't report it if its size
is zero.
Also, make it title case (i.e. "Unknown" instead of "unknown").
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
It is nicer overall to have a main() function, like all our other
scripts tend to have too.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
There are three E501 warnings returned by flake8, when run locally,
because we enforce a local 80-char limit, but that are not reported by
the gitlab-ci jobs because only a 132-char limit is required there.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add support to test that the root passowrd is working as expected.
- Buildtime test: Check the hash present in the generated '/etc/shadow'.
- Runtime test: Build an armv7 image and try to login with a password.
Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently many test cases call subprocess.check_output on their own.
Factor out that code to an infra method so the call get standardized.
This will be handful when switching the test infra to use Python 3.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The python-cbor sample script to be run on the target dumps a binary
variable to the target stdout. This is done for debug purposes-only, in
the case the test on test infra fails.
This non-utf-8 is currently silently ignored by the Emulator class /
pexpect.spawn from the test infra because the infra uses Python 2, that
in turn do not differentiate between string and byte data.
Make the code Python3-friendly (Python 3 in the host) by doing the right
thing and encoding the data before printing it.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Do the same as other fs tests and minimally check the ubi image before
booting.
The call to 'file' was already there, but the output wasn't tested for
some unknown reason. Add the assert for the output of the command.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Newer versions of perl-io-socket-ssl require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
perl tests.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139402
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Newer versions of lua-http require entropy.
Switch to use armv5 builtin kernel that already provides entropy for all
lua tests.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139374https://gitlab.com/buildroot.org/buildroot/-/jobs/269139376
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since version 19.3.3, python-autobahn assumes that when CPython is used
msgpack will be used as well [1]. But it still allows the user to
override this behavior by setting an environment variable [2] to use
umsgpack.
Make the test to explicitly use umsgpack since it is part of the minimal
config (python-crossbar selects python-u-msgpack).
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139430
[1] ea019b8042
[2] fe70ceebe0
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Newer versions of python-treq display a warning at runtime when
service_identity is not installed:
"Without the service_identity module, Twisted can perform only
rudimentary TLS client hostname verification."
This warning message confuses the test that looks for another string in
stdout. Make the test ignore other messages while still expecting
"Connection refused".
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/269139449https://gitlab.com/buildroot.org/buildroot/-/jobs/269139450
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since "2467822c85 package/checksec: bump to version 2.1.0" the hardening
tests fail because upstream slightly changed the way the script is
called.
According to README.md: "- All options now require `--$option=$value`
instead of `--$option $value`"
Instead of just replacing '--output json' with '--output=json' take into
account that upstream also changed the usage example to show --format
instead of --output. Both options do exactly the same, but following the
usage example seems to be more future-proof.
Upstream also improved the json output. Now when a file is passed as
parameter, the json has the file name as the main key, instead of the
string "file". Adjust the test cases accordingly.
Fixes:
tests.core.test_hardening.TestFortifyConserv
tests.core.test_hardening.TestFortifyNone
tests.core.test_hardening.TestRelro
tests.core.test_hardening.TestRelroPartial
tests.core.test_hardening.TestSspNone
tests.core.test_hardening.TestSspStrong
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Prior to b3ba26150d
("toolchain/toolchain-external/toolchain-external-custom: be more
flexible on gcc version"), the default gcc version selected by
Buildroot for custom external toolchain was affected by the
BR2_ARCH_NEEDS_GCC_AT_LEAST_xyz definitions.
Since BR2_riscv selects BR2_ARCH_NEEDS_GCC_AT_LEAST_7, gcc 7.x was the
default gcc version assumed to be used in a custom RISC-V external
toolchain, so our config snippets for RISC-V toolchains were correct.
With b3ba26150d applied, the default gcc
version assumed for custom external toolchains is the latest one
(currently gcc 9.x), while our RISC-V toolchains use gcc 7.x. So we
now need to explicitly give the gcc version used by our RISC-V
toolchains, otherwise the build fails with:
Incorrect selection of gcc version: expected 9.x, got 7.4.0
Fixes:
http://autobuild.buildroot.net/results/b872befe1adec2633b9cbcc49bc0eb7619f606c2/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Similar to toolchains and jpeg, we now offer a way for br2-external
trees to provide their openssl implementation, which gets included in
the openssl choice.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Similar to toolchains, we now offer a way for br2-external trees to
provide their libjpeg implementation, which gets included in the jpeg
choice.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since we have a choice for the pre-configured pre-built toolchains,
there is no possbility for a br2-external to provide its own. The
only solution so far for defconfigs in br2-external trees is to use
BR2_TOOLCHAIN_EXTERNAL_CUSTOM and define all the bits by itself...
This is not so convemient, so offer a way for br2-external trees to
provide such pre-configured toolchains.
To allow for this, we now scan each br2-external tree and look for a
specific file, provides.toolchains.in. We generate a kconfig file that
sources each such file, and that generated file is sourced from within
the toolchain choice, thus making the toolchains from a br2-external
tree possible and available in the same location as the ones known to
Buildroot:
Toolchain --->
Toolchain type (External toolchain) --->
Toolchain --->
(X) Arm ARM 2019.03
( ) Linaro ARM 2018.05
( ) Custom toolchain
*** Toolchains from my-br2-ext-tree: ***
( ) My custom ARM toolchain
*** Toolchains from another-br2-ext-tree: ***
( ) Another custom ARM toolchain
( ) A third custom ARM toolchain
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, the kconfig part contains two things: the kconfig option
with the paths to br2-external trees, and the kconfig menus for the
br2-external trees.
When we want to include more kconfig files from the br2-external tree
(e.g. to get definitions for pre-built toolchains), we will need to
have the paths defined earlier, so they can be used from the br2-external
tree to include files earlier than the existing menus.
Split the generated kconfig file in two: one to define the paths, which
gets included early in our main Config.in, and one to actually define
the existing menus, which still gets included at the same place they
currently are.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We currently redirect the output of each helper function. This was nice
as long as we were generating single .mk and .in fragments.
But we are soon to need more .in fragments.
So, do the redirection inside the .in helpers.
We do not (currently) need to generate more than one .mk fragment, but
for consistency, do the redirection in the .mk helper too.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When we introduced support for multiple br2-external trees, we
introduced two files, one on the Makefile side, needed very early,
and one on the kconfig side, needed later in the configuration
process. We naturally introduced a two-step generation, as it looked
like the simplest and most obvious way.
But now, we are on the verge of generating more files on the kconfig
side, and it does not make sense to add even more steps to generate
them.
And even better yet, we can generate both the Makefile-side and
kconfig-side files at the same time, in fact.
Make it so.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We do not usually provide help for our internal scripts. Besides, such
help has a tendency to bitrot pretty quickly anyway.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit b14b02698 (core/br2-external: restore compatibility with old
distros) switched to using 'eval' to emulate associative arrays, for
those distros too old to have bash-4+.
In so doing, it forgot to declare the new local variables in the
respective helper functions.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
