Commit Graph

9 Commits

Author SHA1 Message Date
Ricardo Martincoski
9bb306bd64 support/testing: large timeout for login prompt
When running multiple instances of emulator in parallel, the login
prompt can take some time to appear.

Use a large timeout when waiting for the prompt to avoid random
failures.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
abe64676d5 support/testing: fix code style in emulator
Remove unused import.
Use 2 empty lines before a class.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
4a622fe4b6 support/testing: use pexpect.expect directly
When using pexpect there is no need for a helper function. Just use
expect() directly everywhere.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
77b4b948fc support/testing: use pexpect.sendline directly
When using pexpect there is no need for a helper function. Just use
sendline() directly everywhere.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
e664c5f71c support/testing: let pexpect write stdout to log
When the parameter logfile is passed to spawn(), pexpect sends both
stdin and stdout to the logfile and it creates a double echo effect.

One way to avoid the double echo in the logfile would be to disable the
echo on the terminal just after login ("stty -echo"), but double echo of
user and password would remain.

Instead of that, send only the stdout to the logfile using the
logfile_read property.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
8ebc227806 support/testing: use qemu stdio in emulator
Instead of redirecting qemu serial to telnet, redirect it to stdio.

It allows to run testcases in parallel without random failing caused by
two emulators trying to use the same telnet port (1234).

'qemu -serial stdio' returns some extra <CR> characters, so remove them
from the log.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Ricardo Martincoski
eb0fab80f6 support/testing: use pexpect in emulator
Replace subprocess + telnetlib with pexpect.

Use the telnet installed on the host machine instead of telnetlib, while
the serial from qemu is not yet redirected to stdio.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 16:05:14 +02:00
Luca Ceresoli
d332f2c521 support/testing: simplify logging by keeping the log file open
We currently call infra.smart_open() to open log files each time we
need to write to them.

Opening the file once in the constructor of Builder and Emulator and
writing to it whenever needed is simpler and slightly more efficient.

Remove smart_open and instead create a new open_log_file() function
which just opens the logfile. Also let it compute the filename, in
order to simplify even further the Builder and Emulator code.

Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-17 21:59:58 +02:00
Thomas Petazzoni
a732fb222b support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.

This core infrastructure commit includes the following aspects:

 - A base test class, called BRTest, defined in
   support/testing/infra/basetest.py. This base test class inherited
   from the Python provided unittest.TestCase, and must be subclassed by
   all Buildroot test cases.

   Its main purpose is to provide the Python unittest setUp() and
   tearDown() methods. In our case, setUp() takes care of building the
   Buildroot system described in the test case, and instantiate the
   Emulator object in case runtime testing is needed. The tearDown()
   method simply cleans things up (stop the emulator, remove the output
   directory).

 - A Builder class, defined in support/testing/infra/builder.py, simply
   responsible for building the Buildroot system in each test case.

 - An Emulator class, defined in support/testing/infra/emulator.py,
   responsible for running the generated system under Qemu, allowing
   each test case to run arbitrary commands inside the emulated system.

 - A run-tests script, which is the entry point to start the tests.

Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00