Commit Graph

414 Commits

Author SHA1 Message Date
Andrey Smirnov
ded5fa3059 testing/tests/package/test_python: do not hardcode interpreter name
In order to be able to leverage the same test code for testing
different python interpreters (or wrappers around CPython) allow child
classes of TestPythonBase to override the name of the executable used
to run tests.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 22:53:27 +02:00
Andrey Smirnov
97278ebddd testing/tests/package/test_python: add TestPython3
Add Python3 version of TestPython2 to make sure both versions of
Python get unit-tested.

Modify the code of libc_time_test() to support that change (convert
the code to use Python3-style "print").

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
[Thomas: update .gitlab-ci.yml.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 22:52:42 +02:00
Andrey Smirnov
4c85d3e459 testing/tests/package/test_python: refactor TestPythonBase
Convert TestPythonBase to a true base class that only provides code
implementing various tests without defining tests themselves in a
"discoverable" form.

To retain correct testing functionality, add TestPython2 derived class
that uses code from TestPythonBase to define actual runnable test.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: fix typo in commit log, update .gitlab-ci.yml, both pointed
by Ricardo.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 22:38:21 +02:00
Yann E. MORIN
3268968ccc support/tests: fix ext4 runtime test
The current test fails because of a legacy option, renamed during the
recent ext overhaul.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sébastien Szymanski <sebastien.szymanski@armadeus.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-14 19:37:14 +02:00
Arnout Vandecappelle
8367d388d4 support/testing: remove references to host/usr
The tools are now installed in host/bin instead of host/usr/bin.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 16:53:56 +02:00
Thomas Petazzoni
8bbc37e933 support/testing: add tests for musl and uclibc toolchains
These tests simply build a system with musl and uclibc toolchains, and
boot them under qemu. It allows to minimally validate that our support
for musl/uclibc external toolchains is working. We already had some
tests covering glibc toolchains, so we can now easily test that all
three C libraries are supported.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
This commit is part of the series, as I've written/used those tests to
validate that things are still working correctly with all of glibc,
uclibc and musl toolchains.
2017-07-05 12:20:05 +02:00
Luca Ceresoli
ad79bb5085 support/testing: rename variables for readability
The 'lines' variable is overwritten with its own fields. Thus it
contains a line first, and then a list of fields -- it never contains
'lines'.

Use two different variables named 'line' and 'fields' to make the code
more readable.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:54:43 +02:00
Luca Ceresoli
cfecdfadc1 support/testing: fs/ext: add a negative test for the "extent" feature
We only have a positive test for it, in ext4. Let's have a negative
one as well.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:48:08 +02:00
Luca Ceresoli
52258901ec support/testing: rename check_broken_links to has_broken_links
has_broken_links makes it self-explanatory that this is a predicate
function, and that the return value tells whether there _are_ broken
links, not the opposite.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:32 +02:00
Thomas Petazzoni
fb71372d5f support/testing: add toolchain tests
This commit adds an initial toolchain test case, testing the ARM
CodeSourcery toolchain, just checking that the proper sysroot is used,
and that a minimal Linux system boots fine under Qemu.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
8db88df8c9 support/testing: add package tests
This commit adds some basic tests for two Buildroot packages: python and
dropbear. These tests are by no mean meant to be exhaustive, but mainly
to serve as initial examples for other tests.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
bf4a6490e4 support/testing: add fs tests
This commit adds a number of test cases for various filesystem formats:
ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them
except yaffs2 are runtime tested. The iso9660 set of test cases is
particularly rich, testing the proper operation of the iso9660 support
with all of grub, grub2 and isolinux.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
96e21b617d support/testing: add core tests
This commit adds a few Buildroot "core" tests, testing functionalities
such as:

 - post-build and post-image scripts
 - root filesystem overlays
 - timezone support

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
a732fb222b support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.

This core infrastructure commit includes the following aspects:

 - A base test class, called BRTest, defined in
   support/testing/infra/basetest.py. This base test class inherited
   from the Python provided unittest.TestCase, and must be subclassed by
   all Buildroot test cases.

   Its main purpose is to provide the Python unittest setUp() and
   tearDown() methods. In our case, setUp() takes care of building the
   Buildroot system described in the test case, and instantiate the
   Emulator object in case runtime testing is needed. The tearDown()
   method simply cleans things up (stop the emulator, remove the output
   directory).

 - A Builder class, defined in support/testing/infra/builder.py, simply
   responsible for building the Buildroot system in each test case.

 - An Emulator class, defined in support/testing/infra/emulator.py,
   responsible for running the generated system under Qemu, allowing
   each test case to run arbitrary commands inside the emulated system.

 - A run-tests script, which is the entry point to start the tests.

Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00