The .affects() method of the CVE class in support/scripts/cve.py can
return 3 values: CVE_AFFECTS, CVE_DOESNT_AFFECT and CVE_UNKNOWN.
We of course properly account for CVEs where .affects() return
CVE_AFFECTS, but the ones for which CVE_UNKNOWN is returned are
currently ignored, and therefore treated as if they did not affect the
package.
However CVE_UNKNOWN in fact indicates that the v_start/v_end fields of
the CPE entry could not be parsed by
distutils.version.LooseVersion(). Instead of ignoring such cases, this
commit adds support for the concept of "unsure CVEs", which will be
listed next to CVEs known to affect the package, so that we are aware
of them and can investigate the version issue.
Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In a follow-up commit, we are about to bump python-cryptography to a
new version, which has the interesting charateristic of using Rust
code. This means python-cryptography will now only be available on
platforms supported by Rust, which for now excludes uclibc-based
configurations (none of the Rust Tier1/Tier2 platforms use uClibc,
there is some uClibc support in Tier3 platforms but they have not been
added to Buildroot for now).
So in preparation for this bump, we switch the few test cases of
Python packages that directly or indirectly use python-cryptography to
use a glibc toolchain. Another impacted test case is the
docker-compose test case, but it already uses a glibc toolchain;
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In most pure Rust packages, the Cargo.toml manifest is at the root
directory, which is why we could call "cargo vendor" without
specifying the path of the manifest.
However, other packages, such as python-cryptography, which have parts
implemented in Rust, have their Cargo.toml located in a specific
subdirectory.
This commit extends the cargo-post-process download script to
understand a BR_CARGO_MANIFEST_PATH environment variable, which allows
a package to pass the location of the Cargo.toml file. If not passed,
"Cargo.toml" is used, preserving the existing behavior for other
packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This breaks the post_process_unpack() function in
support/download/helpers, which had a sequence of pipe, with "head"
that can abort early and cause the pipe to fail.
Fixes intermitent:
make[1]: *** [package/pkg-generic.mk:190: /builds/tpetazzoni/buildroot/test-output/TestDockerCompose/build/containerd-1.5.8/.stamp_downloaded] Error 141
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In order to be package agnostic, the install phase is now using cargo
instead of install. TARGET_CONFIGURE_OPTS is now also set when running
cargo in order to support cross compiling C code within cargo.
This commit also adds support/download/cargo-post-process to perform
the vendoring on Cargo packages.
The <pkg>_LICENSE variable of cargo packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
[Thomas: add support for host-cargo-package and vendoring]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This commit introduces the download post-process script
support/download/go-post-process, and hooks it into the Go package
infrastructure.
The -modcacherw flag is added to ensure that the Go cache is
read/write, and can be deleted properly upon "make clean".
The <pkg>_LICENSE variable of golang packages is expanded with ",
vendored dependencies licenses probably not listed" as currently for
all packages, the licenses of the vendored dependencies are not taken
into account.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
For now, the download post-process logic uses mk_tar_gz, which repacks
a tarball compressed with gzip. So we can only accept as input a
tarball also compressed with gzip. To enforce that, this commit
changes post_process_unpack() to use tar xzf. This makes sure that if
a tarball compressed with something else than gzip gets used, it will
bail out and we will notice.
Support for other compression schemes can be added later on.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The commit [1] added a sed command used to retreive a pattern
to keep only defconfigs whose name start with the pattern.
"<foo>-defconfigs-<pattern>"
The sed command doesn't work as expected if <foo> contains a
single hyphen [2]:
"qemu-6.2.0-defconfigs-qemu"
Update the sed command to ignore completely the part before
"-defconfigs-".
[1] 65d2f04c01
[2] http://lists.busybox.net/pipermail/buildroot/2022-January/632507.html
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The way that python-pybind can be used is fairly complicated, so a
runtime test for it is convenient. In addition, this test validates that
the headers actually work at runtime.
Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
[Arnout:
- Retain python3 only.
- python-pybind is a target package, not host.
- Select python-pybind instead of depend.
- Simplify python-pybind-example package.
- Check in python-pybind-example build if pybind11.get_include()
produces output.
- Don't use python3 -m pybind11 --includes: it includes the main python
includes, which are for the host, not for the target.
- Use TestPythonPackageBase instead of open-coding something imported
with host python.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The original patch for commit cff428fe31 ("download/git: support Git
LFS") included a call to "git lfs install" but this was a problem as it
could modify ~/.gitconfig outside the dl/ tree. When this was
updated it was thought that the modification to gitconfig was
unnecessary because the LFS fetch and checkout steps are performed
manually.
Unfortunately, this is not correct and the LFS checkout fails with:
Cannot checkout LFS objects, Git LFS is not installed.
Add the call to "git lfs install", with the --local option so that only
the repository's .git/config is modified and not the user's global
~/.gitconfig.
This is also required for submodules as the parent repository's config
is not inherited.
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: John Keeping <john@metanate.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Create a test to check Mender client at runtime.
The aim of this test is:
- to check the correct execution of simple Mender commands,
in a minimal environment;
- to validate there is no missing dependencies for runtime.
This test is not a board integration test for Mender,
including well-configured bootloader, partitioning, ...
Check:
- the daemon is started;
- the current 'artifact name' (name of the image or update) of the active
partition is read, without error.
For that, we need to fake (see the 'overlay' directory):
- some bootloader environment variables;
- the name of an update.
Signed-off-by: Mikael Bourhis-Cloarec <mikael.bourhis@smile.fr>
[Romain: remove single hyphen command (Mender 3.0.0)]
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
download post process scripts will often need to unpack the source
code tarball, do some operation, and then repack it. In order to help
with this, post-process-helpers provide an unpack() function and a
repack() function.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
In order to support package managers such as Cargo (Rust) or Go, we
want to run some custom logic after the main download, but before
packing the tarball and checking the hash.
To implement this, this commit introduces a concept of download
post-processing: if -p <something> is passed to the dl-wrapper, then
support/download/<something>-post-process will be called.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- double-quote variable expansion when calling post-process script
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, relocate-sdk.sh must be run _after_ relocating the SDK. There
are cases where it is useful to already prepare the SDK _before_
relocating. For example, it allows to prepare a tarball that the user
has to extract to a specific, pre-defined location and nothing more than
that, which is simpler for the user than requiring the script to be run.
In addition, it hides the build directory that was used by the SDK
builder (somewhat).
Add an optional argument to relocate-sdk.sh that gives the target
directory.
Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Git Large File Storage replaces large files with text pointers in the
Git repository while storing the contents on a remote server. If a
repository is using this extension, then git-lfs must be used to
checkout the large files before the source archive is generated.
Signed-off-by: John Keeping <john@metanate.com>
[vfazio:
- add git-lfs to DL_TOOLS_DEPENDENCIES
- fixup for 5a0d681394
("infra/pkg-download: make the DOWNLOAD macro fully parameterised")
]
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[Arnout:
- don't "git lfs install";
- recurse into submodules.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
support/testing/tests/download/sshd.py:50:28: E261 at least two spaces before inline comment
1 E261 at least two spaces before inline comment
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add download test infrastructure which starts an OpenSSH server using
the sshd binary installed on the Buildroot host. This server can then be
used to test the expected usage of the SCP and SFTP download methods.
The test creates new SSH keys for the server and client, so that the
server can be run as a non-root user.
A new test module has been added called `tests.download.sshd` which
contains helper methods to create the SSH keys and a class called
`OpenSSHDaemon` which handles the sshd server component.
The tests download example packages in the br2-external project `ssh`.
They check the following conditions for both SCP and SFTP download
methods:
- Correct hash.
- Incorrect hash.
- No hash file.
The SSH download test infrastructure is based on test_git.py.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
[Arnout:
- remove spurious end-of-line backslash;
- remove unnecessary executable bit;
- skip test instead of failing if sshd, ssh-keygen, scp or sftp are not
found;
- decode the output of subprocess;
- use subprocess.check_output instead of subprocess.get_output;
- use subprocess.check_call instead of manually checking return code;
- don't set always-overridden SSHD_PORT_NUMBER in .mk file;
- explicitly set sshd options on commandline instead of relying on host
/etc/sshd/sshd_config;
- let sshd listen only on localhost;
- user internal sftp server;
- disable BACKUP_SITE, no network is supposed to be accessed;
- remove the -bad and -nohash versions;
- rename {sftp,scp}-good to plain {sftp,scp};
- move the sftp and scp packages into a single "ssh" external.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add Secure File Transfer Program (SFTP) support using a simple wrapper.
SFTP is a common protocol used to transfer files securely between
enterprises, but it is not currently supported in Buildroot because all
of the packages are usually available via HTTP, git or some other
download method.
SFTP is similar to FTP but it preforms all operations over an encrypted
SSH transport using a specific protocol. This is unlike ftps, which is
traditional FTP over an SSL/TLS connection.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Michael Drake <michael.drake@codethink.co.uk>
[Arnout:
- update documentation with sftp everywhere scp is mentioned;
- rename "verbose" variable to "quiet";
- print the sftp command, similar to wget and scp helpers.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Install the openssh-server package into the test container. This
package, as well as its dependency openssh-client, is required to test
SCP and SFTP download methods on the localhost, as if these tools were
already installed on the user's machine.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
It wasn't immediately obvious to me what the two Buildroot base test
classes were for, so add docstrings to explain the differences between
BRConfigTest and BRTest.
Signed-off-by: Thomas Preston <thomas.preston@codethink.co.uk>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This package was initially requested by José Pekkarinen, so he is
assigned as the maintainer for it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
commit b3c66481e1 replaced RISC-V LP64
bootlin toolchains by RISC-V LP64D. The config symbols
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE and
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_STABLE were marked as legacy.
Those changes were not reflected in the autobuild toolchain configs in
support/config-fragments/autobuild/bootlin-riscv64-{glibc,musl}.config
When testing a package with the command:
./utils/test-pkg --all --package somepackage
bootlin-riscv64-{glibc,musl} toolchain are always skipped. The build
logfile contains:
[...]
Value requested for BR2_TOOLCHAIN_EXTERNAL_BOOTLIN not in final .config
Requested value: BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
Actual value:
Value requested for BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE not in final .config
Requested value: BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE=y
Actual value: # BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_RISCV64_GLIBC_BLEEDING_EDGE is not set
This commit update the autobuild config fragments for RISC-V 64bit
toolchains so they can be used by test-pkg.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
-E flag instructs patch to remove empty files. However, in some cases
empty files are essential. If they are missing, build could be broken
or other bad things can happen.
Note that empty files are still removed when their headers are properly
formattedo: timestamp set to 1970-01-01T00:00:00Z, destination set to
/dev/null.
Signed-off-by: Andrey Nechypurenko <andreynech@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, we have two functions that build a comma-separated list
of items; one is double-quoting the items, while the other is
single-quoting them. Their naming is not very consistent.
Besides, in a followup change, we will need to build a comma-separated
list of items that are already double-quoted.
Introduce a macro that does just build a comma-separated list, and
use that in the two other macros; rename the existing macro so the
naming is consistent.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Following the releases of 2021.11 Bootlin toolchains, this commit
represents the result of re-running the gen-bootlin-toolchains script.
The only part that isn't auto-generated are the contents of
Config.in.legacy, which account for the replacement of the RISC-V LP64
toolchain by RISC-V LP64D toolchains.
The complete set of runtime test cases was verified on Gitlab CI:
https://gitlab.com/tpetazzoni/buildroot/-/pipelines/437767674
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
toolchains.bootlin.com no longer provides a LP64 RISC-V 64-bit
toolchain, but a more useful LP64D RISC-V 64-bit toolchain. Of course,
the old tarballs remain available, but no new versions of the LP64
toolchain will be produced.
This commit reflects this change in the gen-bootlin-toolchains script.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Already supported:
- Pushing a branch called "<foo>-defconfigs" tests all defconfigs.
- Pushing a branch called "<foo>-defconfig-<defconfig-name>" will
test one particular defconfig
This commit adds support for:
- Pushing a branch called "<foo>-defconfigs-<pattern>" which will
test all defconfigs whose name start with the pattern. For example
"<foo>-defconfigs-qemu_" will test all Qemu defconfigs
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
support/testing/tests/package/test_php_lua.py:35:1: E302 expected 2 blank lines, found 1
Add the missing line before class definition.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The php-pam package provides a PHP PAM (Pluggable Authentication
Modules) integration.
https://pecl.php.net/package/PAM
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The php-lua package provides a PHP extension that embeds the lua
interpreter and offers an OO-API to lua variables and functions.
https://pecl.php.net/package/lua
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Two patches are present and were retrieved from the following
upstream pull request in order to support PHP8:
https://github.com/laruence/php-lua/pull/47
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
APCu is an in-memory key-value store for PHP.
Keys are of type string and values can be any PHP variables.
APCu only supports userland caching of variables
https://pecl.php.net/package/APCU
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
It is possible that some users of buildroot have put it in a repository
and call into it from another Makefile such as:
.DEFAULT:
$(MAKE) O=$(abspath $(O)) -C buildroot $(@)
This technique works well except that Make tells us that it changes into
the buildroot directory:
make[1]: Entering directory 'buildroot'
Because this line doesn't have an equals within it, python raises a
ValueError exception within pkg-stats.
This patch has python tell the invoked make not to print directories
Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Like for the github helper, add some tests to test the download of
Gitlab's generated tarball.
[1] f83826c90d
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Back in 2013, a github download helper has been introduced to cope with
changes in github download-URL's [1][2].
Since then a testing infrastructure has been introduced in Buildroot
but no tests has been added to check if the github download helper is
still working.
It was reported recently [3] that the github helper doesn't work anymore
using tags. Buildroot is not the only project having the issue, see
Github feedback discussions [4].
Add tests for direct archive download (archives uploaded by maintainers),
download from a git tag and git hash using the github helper.
Make sure that Buildroot doesn't use BR2_BACKUP_SITE
(http://sources.buildroot.net).
[1] https://bugs.busybox.net/show_bug.cgi?id=6302
[2] c7c7d0697c
[3] https://bugs.busybox.net/show_bug.cgi?id=14396
[4] https://github.com/github/feedback/discussions/8149
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When calling 'printvars', the 'suitable-host-package' macro is printed
(a macro is just a variable like the others, after all, just with some
parameters). Because it is printed as a variable, it is missing its
parameters, but it still tries to evaluate the $(shell) construct.
This causes spurious warning:
make[1]: support/dependencies/check-host-.sh: Command not found
Only try and call the script if there is actually a tool to check for.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Now that our pipelines are using the Docker image from the Gitlab
registry, there is no longer any reason to push the image to the
Docker hub.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
On a properly setup machine, it is totally useless to use sudo to run
docker; it is very bad practice. Instead, users really should add
themselves to the docker group.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The heuristic to extract the various variables of interest is pretty
crude: we filter on variables ending with certain suffixes (like
'%_VERSION' to get the version strings).
However, in doing so, we may dump variables that are not actual package
versions (especially with br2-external trees), and those may contain one
or more equal sign. And anyway, an actual package version string may
very well contain an equal sign too.
But the current situation is that the output of 'printvars' is split on
all equal signs, which will not fit in the 2-tuple we assign the result,
thus causing an exception.
Fix that by limiting to a single split.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As explained by Jörg [1], iteration with pairs() does not result in the
same order since luajit 2.1.
From [2]
"Table iteration with pairs() does not result in the same order?
The order of table iteration is explicitly undefined by the Lua
language standard. Different Lua implementations or versions may use
different orders for otherwise identical tables. Different ways of
constructing a table may result in different orders, too. Due to
improved VM security, LuaJIT 2.1 may even use a different order on
separate VM invocations or when string keys are newly interned.
If your program relies on a deterministic order, it has a bug.
Rewrite it, so it doesn't rely on the key order.
Or sort the table keys, if you must."
Note: The "luvi -v" return 255 even on success.
[1] http://lists.busybox.net/pipermail/buildroot/2021-November/627938.html
[2] https://luajit.org/faq.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the commit replacing moonjit by luajit [1] luvi doesn't work without
rng support enabled.
Switch to armv5 to use virtio-rng-pci on the qemu command line [2].
[1] 9450b53c8e
[2] https://git.buildroot.net/buildroot/tree/support/testing/infra/emulator.py?h=2021.08.1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently when a tag is added to the Buildroot git tree, the gitlab-ci
create a pipeline with several hundred of jobs (~750) to build all
defconfigs and execute the Buildroot testsuite.
However, there is only a limited number of gitlab-ci runner (9 runners)
and some jobs reach the timeout limit (24h) while waiting for a runner
[1]. Indeed, the Buildroot project doesn't use the Gitlab's shared
runners.
In addition to the pipeline created when a new tag is added to the
git repository, two pipelines are created each weeks to execute the
Buildroot testsuite (on monday [2]) and build all defconfigs (on
Thursday [3]).
At some point there are too many jobs waiting in gitlab due board
defconfigs builds. Indded a board defconfig requires a lot of time
(~30min) compared to other jobs in order to build a toolchain and a
kernel linux along with a basic rootfs. There is currently 262
defconfigs.
This is even worse when several pipelines are trigged at the same
time (new git tag and scheduled pipeline trigger).
In order to reduce the number of long jobs, don't build board
defconfigs with pipelines trigged on tag, keeping only the runtime
tests and the Qemu's defconfigs.
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/1758966541
[2] https://gitlab.com/buildroot.org/buildroot/-/pipelines/404035190
[3] https://gitlab.com/buildroot.org/buildroot/-/pipelines/401685550
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d815599e37)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fixes:
support/testing/tests/package/sample_python_unittest_xml_reporting.py:4:1: E302 expected 2 blank lines, found 1
support/testing/tests/package/sample_python_unittest_xml_reporting.py:8:1: E305 expected 2 blank lines after class or function definition, found 1
1 E302 expected 2 blank lines, found 1
1 E305 expected 2 blank lines after class or function definition, found 1
Signed-off-by: Pierre-Jean Texier <texier.pj2@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The test_jffs2 test fail for the same reason as test_ubi test with qemu >= 2.9
due to a qemu 2.8 bug. See commit d8447c38f5.
Divide the erase block size by two.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1687590514
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Based on initial work from Nicolas Carrier
<nicolas.carrier@orolia.com>, with the following additions:
- Updated to a newer version
- Added proper license file handling
- Added runtime test case
- Restricted to Python 3.x
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The php-pecl-dbus package provides a PHP extension for interaction
with D-Bus busses.
https://github.com/derickr/pecl-dbus
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The current Gitlab CI mechanism allows to trigger all tests in a CI
pipeline by pushing a branch named <something>-runtime-tests, or to
trigger a single test in a CI pipeline by pushing a branch name
<something>-tests.<name of test>.
However, there are cases where it is useful to run a suite of tests,
for example to run all tests in tests.init.test_busybox.
This commit makes that possible by extending the current semantic of
<something>-tests.<name of test> to not expect a complete test name,
but instead to accept all tests that starts with the given pattern.
This allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox.TestInitSystemBusyboxRo
like it was the case before. But it now also allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox
to run all Busybox tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a rudimentary test inspired from the "Using boto3" section in
the package README ([1]).
Note that it doesn't try to do anything with the instanciated
resource, as this would require a network connection when the test
runs.
[1]: https://github.com/boto/boto3
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a rudimentary test inspired from the "Using botocore" section in
the package README ([1]).
Note that it doesn't try to use the instantiated client, as this would
require a network connection when the test runs.
[1]: https://github.com/boto/botocore
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
On some developers machines, the default timeout (5 seconds) is not
enough for the test to succeed.
Increase it to 20 seconds, to let more time for the rsa keys to be
generated.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
A simple test that runs nmap twice to create the files scanme-1.xml and
scanme2.xml, then runs pyndiff on both files.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The pkg-stats script queries release-monitoring.org to find the latest
upstream versions of our packages. However, up until recently,
release-monitoring.org had no notion of stable
vs. development/release-candidate versions, so for some packages the
"latest" version was in fact a development/release-candidate version
that we didn't want to package in Buildroot.
However, in recent time, release-monitoring.org has gained support for
differentiating stable vs. development releases of upstream
projects. See for example
https://release-monitoring.org/project/10024/ for the glib library,
which has a number of versions marked "Pre-release".
The JSON blurb returned by release-monitoring.org has 3 relevant
fields:
- "version", which we are using currently, which is a string
containing the reference of the latest version, including
pre-release.
- "versions", which is an array of strings listing all versions,
pre-release or not.
- "stable_versions", which is an array of string listing only
non-pre-release versions. It is ordered newest first to oldest
last.
So, this commit changes from using 'version' to using
'stable_versions[0]'.
As an example, before this change, pkg-stats reports that nfs-utils
needs to be bumped to 2.5.5rc3, while after this patch, it reports
that nfs-utils is already at 2.5.4, and that this is the latest stable
version (modulo an issue where Buildroot has 2.5.4 and
release-monitoring.org has 2-5-4, this will be addressed separately).
Note that part of this change was already done in commit f7b0e0860, but
it was incomplete.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The pkg-stats scripts tries to match packages against
release-monitoring.org in two ways:
- First by using the "Buildroot" distribution registered on
release-monitoring.org, in which we have added a lot of mappings
between Buildroot package names and release-monitoring.org package
names. If there is a match using this distribution, the package
status is RM_API_STATUS_FOUND_BY_DISTRO, which means that the
resulting HTML has a "found by distro" statement.
- Then, if the first solution didn't work, by using the pattern
matching, as done in the check_package_get_latest_version_by_guess()
function.
However, there is a bug in this later case: it sets the package status
to RM_API_STATUS_FOUND_BY_DISTRO as well, while it should have been
RM_API_STATUS_FOUND_BY_PATTERN. Due to this bug, in the resulting HTML
file from a pkg-stats run, all packages are marked as "found by
distro" even the ones that are "found by guess".
This commit fixes that by setting the correct package status.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
For example with libpng: 1.6.37 instead of 1.7.0beta89
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: coalesce into a single line]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
A recent update of flake8 in CI introduced a new check E741. It
basically checks that variables are at least 3 characters long. Up to
now, however, we have used shorter names in some places - all of them
turn out to be "l" for a line of text.
Replace all those "l" variables with "line".
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/scripts/boot-qemu-image.py:47:21: E741 ambiguous variable name 'l'
support/scripts/check-dotconfig.py:20:38: E741 ambiguous variable name 'l'
support/scripts/size-stats:76:13: E741 ambiguous variable name 'l'
support/testing/tests/core/test_bad_arch.py:17:32: E741 ambiguous variable name 'l'
support/testing/tests/package/test_python_treq.py:10:30: E741 ambiguous variable name 'l'
support/testing/tests/toolchain/test_external.py:30:42: E741 ambiguous variable name 'l'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The dbus-next package uses the Python type annotation for dbus types. This is
not compatible with the python typing assumption that flake8 makes.
Exclude F821 from this line.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/testing/tests/package/sample_python_dbus_next.py:17:36: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:48: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:56: F821 undefined name 's'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the ubi/ubifs test has been introduced, it's not possible to
boot the same ubi image twice [1]:
"TODO: if you boot Qemu twice on the same UBI image, it fails to
attach the image the second time, with "ubi0 error:
ubi_read_volume_table: the layout volume was not found"."
For some reason, the kernel corrupt the ubi image if the ubifs
rootfs is mounted with write access. Use a custom config file
to mount the rootfs readonly (vol_type=static). Doing so requires
to add the flash size (vol_size=64MiB).
At least it allows to boot several times the same ubi image.
[1] bf4a6490e4
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The current ubi/ubifs test (test_ubi.py) rely on a Qemu bug present in
2.8.0 that was fixed in Qemu 2.9.0 [1]. The ubi/ubifs settings is
updated to run with Qemu >= 2.9.0 using the new multiple chip handling.
If needed, the old behavior can be enabled using the pflash01 property
"old-multiple-chip-handling" [2].
The issue was not detected until now since we are sill using an old
qemu (2.8 from Debian stretch) for testing in gitlab (using the
Buildroot Docker image used by gitlab-ci.yml).
First the logical eraseblock size (LEB) must be updated to the value
0x3ff80 reported by the kernel when using qemu >= 2.9.0.
UBIFS (ubi0:0): Mounting in unauthenticated mode
UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: LEB size mismatch: 524160 in superblock, 262016 real
UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: bad superblock, error 1
But the system is still failing to boot:
UBIFS error (ubi0:0 pid 1): ubifs_scan: garbage
UBIFS error (ubi0:0 pid 1): ubifs_recover_master_node: failed to recover master node
ubifs is reading garbage since Qemu >= 2.9.0 report a sector
length per device divided by the number of devices (see commit [1]).
The kernel detect two flash devices (dmesg):
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
Divide the physical eraseblock (PEB) size by two.
Tested with qemu 2.9.0, 5.1.0.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1543100932
[1] https://git.qemu.org/?p=qemu.git;a=commitdiff;h=feb0b1aa11f14ee71660aba46b46387d1f923c9e
[2] http://lists.busybox.net/pipermail/buildroot/2021-September/622069.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Adding the Image format on the Qemu command line avoid this warning:
"WARNING: Image format was not specified for 'output/TestUbi/images/rootfs.ubi' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions."
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit e6ee07f41a (package/python-flask-expects-json: new package)
added a non-functional test case that, as noticed by Edgar, fails with:
AssertionError: '%{http_code}' != '200'
That's because the % sign is self-escaped, à-la C, in the first part
of the command, probably to avoid its being %-formatted. But only the
second part of the command is %-formatted, so we do not need to
self-escape % in the first part.
Additionally, since eb3ee3078a (support/testing/infra/emulator.py:
prevent the commands from wrapping), we no longer need to play tricks
with commands that are too long to fit on the first line of the shell
prompt.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The lua-augeas package provides a Lua binding for augeas
https://github.com/ncopa/lua-augeas
Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
dtbocfg, which stands for Device Tree Blob Overlay Configuration
File System, was developed to serve as a userspace API of Device
Tree Overlay.
https://github.com/ikwzm/dtbocfg
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 516b837002)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The list of tests is as follows:
TestGdbHostOnlyDefault: build just minimal host-gdb, default version
TestGdbHostOnlyAllFeatures: build host-gdb, default version, with all
features enabled (TUI, Python, simulator)
TestGdbserverOnly: build just target gdbserver, default version
TestGdbFullTarget: build just target gdb, default version
TestGdbHostOnly9x: build minimal host-gdb, 9.x version
TestGdbHostGdbserver9x: build minimal host-gdb 9.x + gdbserver
TestGdbHostGdbTarget9x: build minimal host-gdb 9.x + full gdb
TestGdbHostOnly11x: build minimal host-gdb, 11.x version
TestGdbHostGdbserver11x: build minimal host-gdb 11.x + gdbserver
TestGdbHostGdbTarget11x: build minimal host-gdb 11.x + gdb
TestGdbArc: build minimal host-gdb + gdb + gdbserver, for the special
ARC architecture version
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Traditional VT-10x terminals (and their emulators) [0] have a "magic
margins" feature that enables the last character position to be updated
without scrolling the screen: whenever a character is printed on the
last column, the cursor stays over the character, instead of moving to
the next line.
The Busybox shell, ash, attempts to defeat this feature by printing
CR,LF right after echoing a character to the last column.[1] This
doesn't play well with emulator.py. The run() method of the Emulator
class captures the output of the emulated system and assumes the first
line it reads is the echo of the command, and all subsequent lines are
the command's output. If the line made by the command + shell prompt is
longer than 80 characters, then it is echoed as two or more lines, and
all but the first one are mistaken for the command's output.
We fix this by telling the emulated system that we are using an
ultra-wide terminal with 29999 columns. Larger values would be ignored
and replaced by the default, namely 80 columns.[2]
[0] https://vt100.net/docs/vt100-ug/chapter3.html - DECAWM
[1] https://git.busybox.net/busybox/tree/libbb/lineedit.c?h=1_34_0#n412
[2] https://git.busybox.net/busybox/tree/libbb/xfuncs.c?h=1_34_0#n258
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Co-authored-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Compiling on Ubuntu 20.04 generates this:
./util.c: In function ‘file_write_dep’
./util.c:54:18: warning: ‘..config.tmp’ directive writing 12 bytes into a region of size between 1 and 4097 [-Wformat-overflow=]
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~
./util.c:54:2: note: ‘sprintf’ output between 13 and 4109 bytes into a destination of size 4097
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
and similar warnings on confdata.c, lines 778, 989, 995, 1000, 1007,
1040, 1046 and 1054. Avoid the warnings by enlarging the destination
buffer of fprintf().
Normally, we want changes to kconfig to be reflected by patches in
support/kconfig/patches. This makes it easier to resync with upstream
kconfig. However, in this case, everything that is changed here is
already changed completely (and differently) upstream, so there is no
added value in keeping the patch.
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
Add dedicated Grub2 builtin config for the EFI compatible cases.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.
However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:
Traceback (most recent call last):
[...]
emulator.login()
File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.
We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.
[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- don't change encoding, use codec_errors
- rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When Grub2 is build it is configured only for one boot set-up, BIOS Legacy,
EFI 32 bit or EFI 64 bit. It can not deal with several boot set-up on the
same image.
This patch allows to build Grub2 for different configurations simultaneously.
To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI
BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more
reasonable to just extend the grub2 package into 3 packages.
We can no longer use autotools-package as a consequence of this multi-build, and
we have to resort to generic-package and a partial duplication of
the autotools-infra. Grub2 was already using custom option like --prefix or
--exec-prefix so this won't add much more weirdness.
We use a GRUB2_TUPLES list to describe all the configurations selected.
For each boot case described in the GRUB2_TUPLES list, it configures and
builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.
We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the
BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- keep sub-options properly indented
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
It's requirement has been removed in previous commit.
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
`which' has been discontinued after 2.21 release in 2015 due this (git
repository is empty [1]) and version shipped in Debian produces warning
[2]:
/usr/bin/which: this version of `which' is deprecated; use `command -v' in scripts instead.
`command is POSIX [3] and supported on all common shells (bash, zsh,
dash, busybox sh, mksh).
Patch tested on dash as the default shell.
[1] https://git.savannah.gnu.org/cgit/which.git
[2] 3a8dd10b45
[3] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS, as this commit is based on initial work
from Nicolas.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds a new package called python-flask-expects-json, which
also to validate the JSON blurbs submitted to a Flask web
application. A runtime test is added as well, making sure that the
package minimally works with an example Flask application.
The files added by this commit are associated both to Nicolas Carrier
and myself in the DEVELOPERS file, as Nicolas is also interested in
this package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit adds a test for python-flask package. As we are about to
add python-flask-expects-json together with a test, it made sense to
also add a test for python-flask itself.
As far as the DEVELOPERS file is concerned, the test files are added
both to the existing maintainer of package/python-flask, as well as to
myself.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit bumps the python-smmap2 and python-gitdb2 packages in
lockstep, as the new version of gitdb2 requires a newer version of
smmap2, but the current version of gitdb2 cannot work with the newer
version of smmap2 (sigh).
Also, upstream the projects have been renamed: gitdb2 is now named
gitdb on PyPi (see https://pypi.org/project/gitdb2/) and smmap2 is now
named smmap (https://pypi.org/project/smmap2/). However, to avoid
needless churn, we don't rename the Buildroot packages, but that
rename is visible in the name of the tarballs being downloaded.
Also, since version 4.0.0, smmap supports only Python 3.x, so we add a
dependency on Python 3.x and drop the test case of gitdb2 on Python
2.x.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Starting with Qemu 6.1.0, gcc 7.5 is needed to build.
Since we build host-qemu package for qemu defconfig, we have to
upgrade to (at least) Debian buster that provide gcc 8 as host compiler.
While testing this upgrate, the test_edk2 failed since it actually
requires Qemu >= 4.1.0 to support arm SBSA reference machine [1].
Debian Buster only provide Qemu 3.1.
Finally, upgrade to Debian bullseye but it requires some linux
kernel version bump in several defconfigs since host gcc is based
on gcc-10 [2].
[1] https://git.qemu.org/?p=qemu.git;a=commit;h=64580903c2b3aee08d74d64e6248a313b246cb69
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38
Signed-off-by: Romain Naour <romain.naour@gmail.com>
[Arnout: don't install python]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).
Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import
Tested with python3 -m py_compile.
[1] https://www.python.org/doc/sunset-python-2/
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
To avoid spending some time to build the x86_64 toolchain (~20min),
switch to corei7 cpu (Nahalem) and use the prebuilt Bootlin toolchain.
We have to use the "stable" Bootlin toolchain to use the same kernel version
for the toolchain kernel headers and the running kernel.
With the "bleeding-edge" toolchain we have the "kernel too old" issue
(running kernel 4.19 vs kernel headers 5.4)
Runtime tested locally.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Switch from the Buildroot internal toolchain for armv5 to
the prebuilt Bootlin external toolchain.
The test doesn't require to build a toolchain, there was
no prebuilt glibc toolchain recent enough at the time this
test has been introduced.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This test already use builtin kernel provided by the testsuite infra:
self.emulator.boot(arch="armv7",
kernel="builtin",
options=["-initrd", img])
But a second kernel is build from the its defconfig. This second kernel
is not used by the test.
The TestRust (using BR2_PACKAGE_HOST_RUST=y) is really long to build,
save some cpu time by removing the kernel build.
This unused kernel (based on 4.11.3 release) doesn't even build with
host gcc >= 10.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.19.79 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
The issue was fixed in 4.19.114 [2]
Bump to the latest 4.19.x version.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=621f2ded601546119fabccd1651b1ae29d26cd38
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 5.5.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
But we can't just update this test to the next linux kernel LTS 5.10.y since
the minimum gcc version has been updated to gcc 4.9 since 5.8 kernel [2]
and the Sourcery CodeBench ARM 2014.05 is used (gcc 4.8 based).
Enable arm cortex A9 and VFP support to switch to the ARM arm prebuilt
toolchain (the Bootlin toolchain could be used).
While at it use the prebuilt buildin kernel for the vexpress target
recently updated to 5.10.7.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1564202094
[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6ec4476ac82512f09c94aff5972654b70f3772b2
[3] 3cf2782906
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This patch adds runtime testing of the OCI archive created by the
sloci scripting. It launches a containerd instance, imports, and
runs the OCI container.
The existing QEMU AARCH64 kernel config was extended to enable common
options used by a container runtime (cgroup and overlayfs).
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
[Arnout: adapt file name which is arm64 now; add to DEVELOPERS]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
