2017-06-29 04:45:41 +02:00
|
|
|
import pexpect
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
|
|
|
import infra
|
2017-06-29 04:45:46 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
|
|
|
class Emulator(object):
|
|
|
|
|
|
|
|
def __init__(self, builddir, downloaddir, logtofile):
|
|
|
|
self.qemu = None
|
|
|
|
self.downloaddir = downloaddir
|
2017-05-10 23:33:46 +02:00
|
|
|
self.logfile = infra.open_log_file(builddir, "run", logtofile)
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
|
|
|
# Start Qemu to boot the system
|
|
|
|
#
|
|
|
|
# arch: Qemu architecture to use
|
|
|
|
#
|
|
|
|
# kernel: path to the kernel image, or the special string
|
|
|
|
# 'builtin'. 'builtin' means a pre-built kernel image will be
|
|
|
|
# downloaded from ARTEFACTS_URL and suitable options are
|
|
|
|
# automatically passed to qemu and added to the kernel cmdline. So
|
|
|
|
# far only armv5, armv7 and i386 builtin kernels are available.
|
|
|
|
# If None, then no kernel is used, and we assume a bootable device
|
|
|
|
# will be specified.
|
|
|
|
#
|
|
|
|
# kernel_cmdline: array of kernel arguments to pass to Qemu -append option
|
|
|
|
#
|
|
|
|
# options: array of command line options to pass to Qemu
|
|
|
|
#
|
|
|
|
def boot(self, arch, kernel=None, kernel_cmdline=None, options=None):
|
|
|
|
if arch in ["armv7", "armv5"]:
|
|
|
|
qemu_arch = "arm"
|
|
|
|
else:
|
|
|
|
qemu_arch = arch
|
|
|
|
|
|
|
|
qemu_cmd = ["qemu-system-{}".format(qemu_arch),
|
2017-06-29 04:45:42 +02:00
|
|
|
"-serial", "stdio",
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
"-display", "none"]
|
|
|
|
|
|
|
|
if options:
|
|
|
|
qemu_cmd += options
|
|
|
|
|
|
|
|
if kernel_cmdline is None:
|
|
|
|
kernel_cmdline = []
|
|
|
|
|
|
|
|
if kernel:
|
|
|
|
if kernel == "builtin":
|
|
|
|
if arch in ["armv7", "armv5"]:
|
|
|
|
kernel_cmdline.append("console=ttyAMA0")
|
|
|
|
|
|
|
|
if arch == "armv7":
|
|
|
|
kernel = infra.download(self.downloaddir,
|
|
|
|
"kernel-vexpress")
|
|
|
|
dtb = infra.download(self.downloaddir,
|
|
|
|
"vexpress-v2p-ca9.dtb")
|
|
|
|
qemu_cmd += ["-dtb", dtb]
|
|
|
|
qemu_cmd += ["-M", "vexpress-a9"]
|
|
|
|
elif arch == "armv5":
|
|
|
|
kernel = infra.download(self.downloaddir,
|
|
|
|
"kernel-versatile")
|
|
|
|
qemu_cmd += ["-M", "versatilepb"]
|
|
|
|
|
|
|
|
qemu_cmd += ["-kernel", kernel]
|
|
|
|
|
|
|
|
if kernel_cmdline:
|
|
|
|
qemu_cmd += ["-append", " ".join(kernel_cmdline)]
|
|
|
|
|
2017-05-10 23:33:46 +02:00
|
|
|
self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
|
2017-06-29 04:45:45 +02:00
|
|
|
self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:], timeout=5)
|
2017-06-29 04:45:43 +02:00
|
|
|
# We want only stdout into the log to avoid double echo
|
|
|
|
self.qemu.logfile_read = self.logfile
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
|
|
|
# Wait for the login prompt to appear, and then login as root with
|
|
|
|
# the provided password, or no password if not specified.
|
|
|
|
def login(self, password=None):
|
2017-06-29 04:45:49 +02:00
|
|
|
# The login prompt can take some time to appear when running multiple
|
|
|
|
# instances in parallel, so set the timeout to a large value
|
2017-06-29 04:45:45 +02:00
|
|
|
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
|
2017-06-29 04:45:49 +02:00
|
|
|
timeout=60)
|
2017-06-29 04:45:45 +02:00
|
|
|
if index != 0:
|
2017-05-10 23:33:46 +02:00
|
|
|
self.logfile.write("==> System does not boot")
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
raise SystemError("System does not boot")
|
|
|
|
|
2017-06-29 04:45:44 +02:00
|
|
|
self.qemu.sendline("root")
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
if password:
|
2017-06-29 04:45:45 +02:00
|
|
|
self.qemu.expect("Password:")
|
2017-06-29 04:45:44 +02:00
|
|
|
self.qemu.sendline(password)
|
2017-06-29 04:45:45 +02:00
|
|
|
index = self.qemu.expect(["# ", pexpect.TIMEOUT])
|
|
|
|
if index != 0:
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
raise SystemError("Cannot login")
|
|
|
|
self.run("dmesg -n 1")
|
|
|
|
|
|
|
|
# Run the given 'cmd' on the target
|
|
|
|
# return a tuple (output, exit_code)
|
|
|
|
def run(self, cmd):
|
2017-06-29 04:45:44 +02:00
|
|
|
self.qemu.sendline(cmd)
|
2017-06-29 04:45:45 +02:00
|
|
|
self.qemu.expect("# ")
|
|
|
|
# Remove double carriage return from qemu stdout so str.splitlines()
|
|
|
|
# works as expected.
|
|
|
|
output = self.qemu.before.replace("\r\r", "\r").splitlines()[1:]
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
2017-06-29 04:45:44 +02:00
|
|
|
self.qemu.sendline("echo $?")
|
2017-06-29 04:45:45 +02:00
|
|
|
self.qemu.expect("# ")
|
|
|
|
exit_code = self.qemu.before.splitlines()[2]
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
exit_code = int(exit_code)
|
|
|
|
|
|
|
|
return output, exit_code
|
|
|
|
|
|
|
|
def stop(self):
|
|
|
|
if self.qemu is None:
|
|
|
|
return
|
2017-06-29 04:45:41 +02:00
|
|
|
self.qemu.terminate(force=True)
|