Commit Graph

13 Commits

Author SHA1 Message Date
Luca Ceresoli
ad79bb5085 support/testing: rename variables for readability
The 'lines' variable is overwritten with its own fields. Thus it
contains a line first, and then a list of fields -- it never contains
'lines'.

Use two different variables named 'line' and 'fields' to make the code
more readable.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:54:43 +02:00
Luca Ceresoli
cfecdfadc1 support/testing: fs/ext: add a negative test for the "extent" feature
We only have a positive test for it, in ext4. Let's have a negative
one as well.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-28 14:48:08 +02:00
Luca Ceresoli
d332f2c521 support/testing: simplify logging by keeping the log file open
We currently call infra.smart_open() to open log files each time we
need to write to them.

Opening the file once in the constructor of Builder and Emulator and
writing to it whenever needed is simpler and slightly more efficient.

Remove smart_open and instead create a new open_log_file() function
which just opens the logfile. Also let it compute the filename, in
order to simplify even further the Builder and Emulator code.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:59:58 +02:00
Luca Ceresoli
fa3c5cad44 support/testing/run-tests: help: put the one-letter form before the long form
This is what the manpages usually do, and what Python does with the
automatically-added -h/--help parameter:

Before the change:
  $ ./support/testing/run-tests
    [...]
  optional arguments:
    -h, --help            show this help message and exit
    --list, -l            list of available test cases
    --all, -a             execute all test cases

After the change:
  $ ./support/testing/run-tests
    [...]
  optional arguments:
    -h, --help            show this help message and exit
    -l, --list            list of available test cases
    -a, --all             execute all test cases

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:49 +02:00
Luca Ceresoli
ee72dbe576 support/testing: document get_elf_prog_interpreter
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:38 +02:00
Luca Ceresoli
52258901ec support/testing: rename check_broken_links to has_broken_links
has_broken_links makes it self-explanatory that this is a predicate
function, and that the return value tells whether there _are_ broken
links, not the opposite.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:56:32 +02:00
Luca Ceresoli
bab4a2fd37 support/testing: remove unused variable
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-10 23:42:27 +02:00
Luca Ceresoli
08151c6aec support/testing: show messages in a more readable format
Currently messages from run-tests are quite unpleasant:

[br-tests/TestPostScripts/2017-05-09 15:51:57] Building
[br-tests/TestPostScripts/2017-05-09 15:52:23] Building done
[br-tests/TestPostScripts/2017-05-09 15:52:23] Cleaning up
.[br-tests/TestNoTimezone/2017-05-09 15:52:23] Starting
[br-tests/TestNoTimezone/2017-05-09 15:52:23] Building
[br-tests/TestNoTimezone/2017-05-09 15:53:17] Building done
[br-tests/TestNoTimezone/2017-05-09 15:53:22] Cleaning up
.[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Starting
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:53:22] Building
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:33] Building done
[br-tests/TestGlibcNonDefaultLimitedTimezone/2017-05-09 15:54:37] Cleaning up
[...]

Change them in a more readable way by removing the date and using a
columnar style:

15:12:22 TestPostScripts                          Starting
15:12:25 TestPostScripts                          Building
15:12:48 TestPostScripts                          Building done
15:12:48 TestPostScripts                          Cleaning up
.15:12:48 TestNoTimezone                           Starting
15:12:54 TestNoTimezone                           Building
15:13:44 TestNoTimezone                           Building done
15:13:49 TestNoTimezone                           Cleaning up
.15:13:49 TestGlibcNonDefaultLimitedTimezone       Starting
15:14:00 TestGlibcNonDefaultLimitedTimezone       Building
15:14:56 TestGlibcNonDefaultLimitedTimezone       Building done
15:15:01 TestGlibcNonDefaultLimitedTimezone       Cleaning up
[...]

Note the '.' and other characters presented by nose2 are still
printed. They are not affected by this change.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-10 23:42:13 +02:00
Thomas Petazzoni
fb71372d5f support/testing: add toolchain tests
This commit adds an initial toolchain test case, testing the ARM
CodeSourcery toolchain, just checking that the proper sysroot is used,
and that a minimal Linux system boots fine under Qemu.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
8db88df8c9 support/testing: add package tests
This commit adds some basic tests for two Buildroot packages: python and
dropbear. These tests are by no mean meant to be exhaustive, but mainly
to serve as initial examples for other tests.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
bf4a6490e4 support/testing: add fs tests
This commit adds a number of test cases for various filesystem formats:
ext2/3/4, iso9660, jffs2, squashfs, ubi/ubifs and yaffs2. All of them
except yaffs2 are runtime tested. The iso9660 set of test cases is
particularly rich, testing the proper operation of the iso9660 support
with all of grub, grub2 and isolinux.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
96e21b617d support/testing: add core tests
This commit adds a few Buildroot "core" tests, testing functionalities
such as:

 - post-build and post-image scripts
 - root filesystem overlays
 - timezone support

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00
Thomas Petazzoni
a732fb222b support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.

This core infrastructure commit includes the following aspects:

 - A base test class, called BRTest, defined in
   support/testing/infra/basetest.py. This base test class inherited
   from the Python provided unittest.TestCase, and must be subclassed by
   all Buildroot test cases.

   Its main purpose is to provide the Python unittest setUp() and
   tearDown() methods. In our case, setUp() takes care of building the
   Buildroot system described in the test case, and instantiate the
   Emulator object in case runtime testing is needed. The tearDown()
   method simply cleans things up (stop the emulator, remove the output
   directory).

 - A Builder class, defined in support/testing/infra/builder.py, simply
   responsible for building the Buildroot system in each test case.

 - An Emulator class, defined in support/testing/infra/emulator.py,
   responsible for running the generated system under Qemu, allowing
   each test case to run arbitrary commands inside the emulated system.

 - A run-tests script, which is the entry point to start the tests.

Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00