Some upcoming test cases can use one or more br2-external trees as
fixtures that provide packages used only in runtime tests.
Add support for br2-external into the BRTest class. Any test case can
then provide a list of paths for being used as br2-external trees
during the build of the image to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Thomas: use named argument for make_extra_opts.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Make the builder able to call 'VAR1=1 make VAR2=2 target'.
Allow sending extra parameters to be added to the end of make command
line. Uses for these purposes:
- to configure a br2-external, using the 'BR2_EXTERNAL="dir" variable.
- to specify a make target, such as 'foo-source.'
Allow adding variables to the environment when calling make.
These added variables allow a user to override default values from BuildRoot,
such as 'BR2_DL_DIR="dl"'.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Daniel J. Leach <dleach@belcan.com>
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Having a docstring in the test_run() method clutters the "run-tests
-l" output:
test_run (tests.package.test_python_crossbar.TestPythonPy3Crossbar)
Test a python package. ... ok
[...]
test_run (tests.package.test_python_pexpect.TestPythonPy2Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_pexpect.TestPythonPy3Pexpect)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy2Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_twisted.TestPythonPy3Twisted)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy2Pynacl)
Test a python package. ... ok
test_run (tests.package.test_python_pynacl.TestPythonPy3Pynacl)
Test a python package. ... ok
So let's simply drop this docstring that is not particularly useful.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. The target has no https
server, so a connection from in the target to localhost must not
succeed.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by calling 'ls' and
checking the output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that minimally uses the module.
Add haveged to the target to generate enough entropy so pynacl ->
libsodium don't hang waiting for /dev/random.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage. Call 'login' and try
wrong user/password, expecting the 'Login incorrect' message.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that creates a hash for a password and verifies
it against an incorrect and a correct password.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by creating a class with
two constants.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case to check the basic usage by checking the
corresponding representation of a 12-bit decimal number in hex, binary
and integer.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a minimal script to check the basic usage by creating and using a
small state machine.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a minimal script to check the basic usage creating a class with 2
attributes.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since commit "2927f412be support/testing: standardize defconfig
fragments style" all other test cases use the same style for defconfig
fragments:
- start after a backslash;
- be declared as a multi-line string literal;
- be indented one level more than the variable that contains it.
Do the same here for consistency.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test invokes "crossbar version" command, that checks all
dependencies found in setup.py files and prints some system related
information.
Add haveged to the target to generate enough entropy so crossbar ->
pynacl -> libsodium don't hang waiting for /dev/random.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
[Ricardo: move test script to a separate file, remove Python 2 variant,
add haveged to target to add entropy and avoid hanging]
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test scripts to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Get the base defconfig fragment from the immediate parent class and not
directly from TestPythonBase because it is the correct way of doing
this. This way the base class TestPythonTwisted could even be placed in
a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move the test script to be run on the target from inline in the test
case to a separate file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Test cases for python packages are very similar among each other: run a
simple script in the target that minimally tests the package.
So create a new helper class named TestPythonPackageBase that holds all
the logic to run a script on the target.
TestPythonPackageBase adds in build time one or more sample scripts to
be run on the target. The test case for the python package must
explicitly list them in the "sample_scripts" property. The test case
then automatically logins to the target, checks the scripts are really
in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt
to make the logfile more friendly, since someone analysing a failure can
easily check the expected script was executed) and then calls the python
interpreter passing the sample script as parameter.
An optional property "timeout" exists for the case the sample script
needs more time to run than the default timeout from the test infra
(currently 5 seconds).
A simple test case for a package that only supports Python 2 will look
like this:
|from tests.package.test_python import TestPythonPackageBase
|
|
|class TestPythonPy2<Package>(TestPythonPackageBase):
| __test__ = True
| config = TestPythonPackageBase.config + \
| """
| BR2_PACKAGE_PYTHON=y
| BR2_PACKAGE_PYTHON_<PACKAGE>=y
| """
| sample_scripts = ["tests/package/sample_python_<package>.py"]
| timeout = 15
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.
However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.
Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.
Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:
File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring
This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.
The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.
The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.
So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.
Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216
Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.
[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This commit fixes the following flake8 warnings:
support/testing/tests/fs/test_f2fs.py:6:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:12:1: E302 expected 2 blank lines, found 1
support/testing/tests/fs/test_f2fs.py:38:23: E225 missing whitespace around operator
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Resolves:
support/testing/tests/core/test_hardening.py:25:42: E231 missing whitespace after ','
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Catch the commonly used options of SSP, Relro, and fortify.
Using the package targets of busybox and lighttpd. This
can easily be expanded to a larger list.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. We update U-Boot and ATF to
what is used in the current version of
solidrun_macchiatobin_mainline_defconfig, for which the problem no
longer exists.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/107860312
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test case uses a too old U-Boot version, which is affected by the
infamous libfdt header conflict issue. Let's update to U-Boot 2017.11,
which is used by our current bananapi_m64_defconfig that was the
inspiration for this test case.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/107860310
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Use a minimal script to listen to a port and check using netstat.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
[Thomas: increase the delay after starting the Twisted server, as 5
seconds was not enough for Python 3.x configurations.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module and asserts a version
string for a fake package is generated.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module to use with twisted in
Python 2 and with asyncio in Python 3.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a simple test case that imports the module.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
While integrating proxy support in builder.py, a log flush
was left in the code. This commit cleans/removes that code.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Allow builder.py to inherit the system proxy settings from
the env if they are present.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fix three issues with code style in our test infra:
- 'print' is now a function,
- exceptions need to be caught-assigned with the 'as' keyword,
- old-style "%s"%() formatting is deprecated.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: drop indices in format strings.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When the run-time tests to build rust and rust-bin packages are run via Docker,
the $USER environment variable is not set, which makes cargo fail when
initializing the test project.
So add it to make cargo happy.
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The version of the ARM Trusted Firmware from Marvell was a Git branch,
not a Git commit, leading to unreproducible results. So let's use a
Git commit instead, which is the latest available from the branch that
was previously used.
More specifically, this branch has recently seen a fix that is needed
for ATF to build properly with recent gcc versions:
c96ec59f8b
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test case currently fails to build with:
./build/juno/release/bl1/context_mgmt.o: In function `cm_prepare_el3_exit':
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): undefined reference to `cm_set_next_context'
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): relocation truncated to fit: R_AARCH64_JUMP26 against undefined symbol `cm_set_next_context'
This issue has been fixed upstream in commit
10c252c14b7f446c0b49ef1aafbd5d37804577dd, available since v1.3. So
while we bump, let's bump to the latest version of ATF, v1.5.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/64360659
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Check:
- the daemon is started;
- a message is collected;
- the daemon does not issue a warning message on startup.
When the .conf file version does not match the package version a warning
message shows up on serial on every boot. This message is generated by
syslog-ng before it is running, so it is not logged to
/var/log/messages. So in order to test the message is generated, restart
the server. It makes the message appears on /var/log/messages (since the
server is already running) where its existence can be easily tested
using grep.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some test cases don't use a full build as setup, so split the build()
method into configure() and build().
It allows a test case to perform configuration at the setup stage and
the build inside the test itself.
Call this new method just before build in the BRTest base class, to keep
the current behavior for existing test cases.
This change will be needed when adding a common class to test the git
download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Use an empty environment when calling make, but import PATH so the
buildroot tree under test can find binaries from the host machine.
Since environment variables are now ignored, move the handling of
BR2_DL_DIR to the defconfig to keep the current precedence of -d:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(DIR)
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(DIR)
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.
However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.
We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.
This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Recent systemd bump has broken DBus dameon and DBus applications can no
longer find the daemon. So we want to catch those kind of failures
early.
We also want to check that the system as a whole is stable: no unit
should be failed.
Finally, ensure that we can read the jounrnal, even when we are doing our
tricks on read-only systems.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
in turn breaks the job in GitLab.
TestRustBase is not a test case by itself, so it can't have a method
with the name starting with "test_" otherwise nose2 assumes it is a test
case.
Move the test_run method from the base class to the derived classes.
While at it, update .gitlab-ci.yml with the new test cases.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035
Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To test the support for the Rust language, the following tests are added:
- building Rust compiler and Cargo from source.
- installing a pre-built Rust compiler and building Cargo from source.
For each test, a Rust test program is built and installed in the root file
system of a ARM vexpress QEMU system. The test is declared OK if the program can
be run properly from the test system.
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fix these warnings:
E122 continuation line missing indentation or outdented
E302 expected 2 blank lines, found 1
F401 'os' imported but unused
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, only post-build and post-image scripts were tested, each with
their own test-script.
The two test-scripts only differ in the name of the log file they
create, and it is based on the name of the script, so it is easy to
share the script.
This allows us to easily re-use it for testing post-fakeroot scripts.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
It needs mkzftree from zisofs-tools, so we add a dependency to it, and
we call that one explicitly (to avoid using the one from the host in
PATH).
It also needs the the uncompressed kernel image, but because it is
already in target/ so it gets compressed by mkzftree. We have two
options:
- compress everything but the kernel image,
- compress everything, kernel included, and recopy it later.
We choose the latter, because it is the simplest solution. So, we always
define the kernel-copy hook, but only register it when needed.
Finally, it needs a kernel with support for transparent
(de)compression, so we update the existing test config.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: really set the transparent compression option, take into
account the renaming of the option.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Marvell U-Boot needs DTC to build, so the test case fails to build
on systems where dtc isn't available. We add
BR2_TARGET_UBOOT_NEEDS_DTC=y to make sure that we build our own DTC.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/44126707
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
These new tests only do build tests, but allow to quickly verify that
the ATF/U-Boot combinations for vexpress, Allwinner and Marvell
platforms all continue to build properly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Linux 4.0 kernel doesn't build with gcc 6.x, which is used since
the toolchain update in commit
193dfffa83 ("support/testing: use more
recent toolchains"). So let's update to Linux 4.11 instead (like the
existing Qemu x86 defconfig does), and update the kernel configuration
file accordingly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
With the hard disk crash of autobuild.b.o, we lost old toolchains, so
use the latest toolchains, which have been restored.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the download directory, when specified with the -d option, is
only used to store the files downloaded by the testing infra, not those
downloaded by Buildroot.
So, we end up with this situation:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(TOP_DIR)/dl
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(BR2_DL_DIR)
This is not very consistent.
We change the behaviour so that the value of -d always takes precedence,
and is used by Buildroot as well, giving this new behaviour:
BR2_DL_DIR | -d DIR | test downloads | BR downloads
------------+----------+------------------+--------------
unset | unset | [error] | [error]
unset | set | in $(DIR) | in $(DIR)
set | unset | in $(BR2_DL_DIR) | in $(BR2_DL_DIR)
set | set | in $(DIR) | in $(DIR)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fix the remaining code style warnings from flake8:
- properly indent continuation lines;
- use proper code to test a parameter is not None.
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fix the trivial warnings from flake8:
- remove modules imported but unused;
- use 2 lines before class or module level method;
- remove blank line at end of file.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Change all defconfig fragments to take advantage of
"cf3cd4388a support/tests: allow properly indented config fragment".
Make each defconfig fragment:
- start after a backslash;
- be declared as a multi-line string literal;
- be indented one level more than the variable that contains it.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Move the strip out of leading spaces in defconfig fragments from the
BRTest class to the Builder class. It actually postpones the strip out,
consequentially allowing test cases to post-process the defconfig in
their own __init__ before calling the __init__ method from BRTest.
Moving this code to the Builder class also allows any new test class
that inherits from BRTest to reuse the same code even if the new class
overrides the setUp method.
At same time, prepend a newline to the jlevel handling otherwise it
would stop working for defconfig fragments that don't end in a newline.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
grub is no longer maintained: it is stuck at version 0.97 with huge
patches that have no opportunity to be applied upstream, as upstream
has even renamed it grub-legacy.
Besides, it no longer builds correctly with recent binutils versions,
and even the huge patches we could grab from Debian do not help the
slightest.
Since upstream really considers it dead, and there are at least two
alternatives (grub2 and syslinux), just remove grub.
Add a legacy entry.
Remove the test cases as well.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
When we list the available tests, we use test_dir, which is set from
the path of the script. However, when we run the tests, we use the
hard-coded path.
Ditto to find the config file.
For consistency, always use test_dir.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a parameter to run-tests to act as a multiplier for all timeouts of
emulator.
It can be used to avoid sporadic failures on slow host machines as well
in elastic runners on the cloud.
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: rename multiplier to timeout_multiplier everywhere]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
We add the 3 following combinations:
- basic systemd, read-only, network w/ ifupdown
- basic systemd, read-only, network w/ networkd
- full systemd, read-only, network w/ networkd
The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Arnout: regenerate .gitlab-ci.yml]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The "builtin" kernel does not boot a systemd-based system, so
we resort to building the same one as currently used by our
qemu_arm_vexpress_defconfig.
We test the 8 following combinations:
- busybox, read-only, without network
- busybox, read-only, with network
- busybox, read-write, without network
- busybox, read-write, with network
- basic systemd, read-write, network w/ ifupdown
- basic systemd, read-write, network w/ networkd
- full systemd, read-write, network w/ networkd
- no init system, read-only, without network
The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: update .gitlab-ci.yml]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The default audio backend for qemu is configured at compile time. It
generates annoying warning messages to qemu's stderr when running our
tests, like these:
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Explicitly set the audio backend to "none" at runtime to remove those
messages from our logs. There is no command line argument for this, so
use an environment variable when starting qemu.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As suggested by Arnout in [1].
While at it, simplify the logic by always appending the BR2_JLEVEL and
defaulting to 0 (the value copied from Config.in is used for 5 years now
and is very unlikely to change).
[1] http://patchwork.ozlabs.org/patch/790525/
Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit cf3cd4388a the -j option is
silently ignored.
The configuration lines are processed using '\n'.join().
This function adds intervening occurrences of the separator, but the
resulting string does not end at a separator.
>>> "n".join(["a","b"])
'anb'
It results in a defconfig that does not end in a newline.
When BR2_JLEVEL is added by -j logic to the defconfig it ends up
concatenated to the last line of the defconfig.
BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=yBR2_JLEVEL=7
The resulting .config has the default BR2_JLEVEL=0.
Instead of just workaround this problem by adding a newline before
BR2_JLEVEL when -j is used, make the defconfig to end in a newline since
it is a more future-proof solution.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The defconfig is composed on-the-fly by test infra + tests.
Dump it to the logfile before running 'make olddefconfig' so it can
easily analysed when debugging.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Depending on Python implementation used for testing, time it takes to
perform a given test can vary pretty significantly. To accout for that
allow individual test functions to specify different timeout value.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In order to be able to leverage the same test code for testing
different python interpreters (or wrappers around CPython) allow child
classes of TestPythonBase to override the name of the executable used
to run tests.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
