kumquat-buildroot/support/testing/infra/__init__.py

98 lines
2.8 KiB
Python
Raw Normal View History

support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
import os
import re
import sys
import tempfile
import subprocess
from urllib2 import urlopen, HTTPError, URLError
ARTIFACTS_URL = "http://autobuild.buildroot.net/artefacts/"
def open_log_file(builddir, stage, logtofile=True):
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
"""
Open a file for logging and return its handler.
If logtofile is True, returns sys.stdout. Otherwise opens a file
with a suitable name in the build directory.
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
"""
if logtofile:
fhandle = open("{}-{}.log".format(builddir, stage), 'a+')
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
else:
fhandle = sys.stdout
return fhandle
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
def filepath(relpath):
return os.path.join(os.getcwd(), "support/testing", relpath)
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
def download(dldir, filename):
finalpath = os.path.join(dldir, filename)
if os.path.exists(finalpath):
return finalpath
if not os.path.exists(dldir):
os.makedirs(dldir)
tmpfile = tempfile.mktemp(dir=dldir)
print "Downloading to {}".format(tmpfile)
try:
url_fh = urlopen(os.path.join(ARTIFACTS_URL, filename))
with open(tmpfile, "w+") as tmpfile_fh:
tmpfile_fh.write(url_fh.read())
except (HTTPError, URLError), err:
os.unlink(tmpfile)
raise err
print "Renaming from %s to %s" % (tmpfile, finalpath)
os.rename(tmpfile, finalpath)
return finalpath
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
def get_elf_arch_tag(builddir, prefix, fpath, tag):
"""
Runs the cross readelf on 'fpath', then extracts the value of tag 'tag'.
Example:
>>> get_elf_arch_tag('output', 'arm-none-linux-gnueabi-',
'bin/busybox', 'Tag_CPU_arch')
v5TEJ
>>>
"""
cmd = ["host/bin/{}-readelf".format(prefix),
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
"-A", os.path.join("target", fpath)]
out = subprocess.check_output(cmd, cwd=builddir, env={"LANG": "C"})
regexp = re.compile("^ {}: (.*)$".format(tag))
for line in out.splitlines():
m = regexp.match(line)
if not m:
continue
return m.group(1)
return None
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
def get_file_arch(builddir, prefix, fpath):
return get_elf_arch_tag(builddir, prefix, fpath, "Tag_CPU_arch")
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
def get_elf_prog_interpreter(builddir, prefix, fpath):
"""
Runs the cross readelf on 'fpath' to extract the program interpreter
name and returns it.
Example:
>>> get_elf_prog_interpreter('br-tests/TestExternalToolchainLinaroArm',
'arm-linux-gnueabihf',
'bin/busybox')
/lib/ld-linux-armhf.so.3
>>>
"""
cmd = ["host/bin/{}-readelf".format(prefix),
support/testing: core testing infrastructure This commit adds the core of a new testing infrastructure that allows to perform runtime testing of Buildroot generated systems. This infrastructure uses the Python unittest logic as its foundation. This core infrastructure commit includes the following aspects: - A base test class, called BRTest, defined in support/testing/infra/basetest.py. This base test class inherited from the Python provided unittest.TestCase, and must be subclassed by all Buildroot test cases. Its main purpose is to provide the Python unittest setUp() and tearDown() methods. In our case, setUp() takes care of building the Buildroot system described in the test case, and instantiate the Emulator object in case runtime testing is needed. The tearDown() method simply cleans things up (stop the emulator, remove the output directory). - A Builder class, defined in support/testing/infra/builder.py, simply responsible for building the Buildroot system in each test case. - An Emulator class, defined in support/testing/infra/emulator.py, responsible for running the generated system under Qemu, allowing each test case to run arbitrary commands inside the emulated system. - A run-tests script, which is the entry point to start the tests. Even though I wrote the original version of this small infrastructure, a huge amount of rework and improvement has been done by Maxime Hadjinlian, and squashed into this patch. So many thanks to Maxime for cleaning up and improving my Python code! Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
"-l", os.path.join("target", fpath)]
out = subprocess.check_output(cmd, cwd=builddir, env={"LANG": "C"})
regexp = re.compile("^ *\[Requesting program interpreter: (.*)\]$")
for line in out.splitlines():
m = regexp.match(line)
if not m:
continue
return m.group(1)
return None