support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
import os
|
|
|
|
import re
|
|
|
|
import sys
|
|
|
|
import tempfile
|
|
|
|
import subprocess
|
|
|
|
from urllib2 import urlopen, HTTPError, URLError
|
|
|
|
|
|
|
|
ARTIFACTS_URL = "http://autobuild.buildroot.net/artefacts/"
|
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
2017-05-10 23:33:46 +02:00
|
|
|
def open_log_file(builddir, stage, logtofile=True):
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
"""
|
2017-05-10 23:33:46 +02:00
|
|
|
Open a file for logging and return its handler.
|
|
|
|
If logtofile is True, returns sys.stdout. Otherwise opens a file
|
|
|
|
with a suitable name in the build directory.
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
"""
|
2017-05-10 23:33:46 +02:00
|
|
|
if logtofile:
|
|
|
|
fhandle = open("{}-{}.log".format(builddir, stage), 'a+')
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
else:
|
|
|
|
fhandle = sys.stdout
|
2017-05-10 23:33:46 +02:00
|
|
|
return fhandle
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
def filepath(relpath):
|
|
|
|
return os.path.join(os.getcwd(), "support/testing", relpath)
|
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
def download(dldir, filename):
|
|
|
|
finalpath = os.path.join(dldir, filename)
|
|
|
|
if os.path.exists(finalpath):
|
|
|
|
return finalpath
|
|
|
|
|
|
|
|
if not os.path.exists(dldir):
|
|
|
|
os.makedirs(dldir)
|
|
|
|
|
|
|
|
tmpfile = tempfile.mktemp(dir=dldir)
|
|
|
|
print "Downloading to {}".format(tmpfile)
|
|
|
|
|
|
|
|
try:
|
|
|
|
url_fh = urlopen(os.path.join(ARTIFACTS_URL, filename))
|
|
|
|
with open(tmpfile, "w+") as tmpfile_fh:
|
|
|
|
tmpfile_fh.write(url_fh.read())
|
|
|
|
except (HTTPError, URLError), err:
|
|
|
|
os.unlink(tmpfile)
|
|
|
|
raise err
|
|
|
|
|
|
|
|
print "Renaming from %s to %s" % (tmpfile, finalpath)
|
|
|
|
os.rename(tmpfile, finalpath)
|
|
|
|
return finalpath
|
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
def get_elf_arch_tag(builddir, prefix, fpath, tag):
|
|
|
|
"""
|
|
|
|
Runs the cross readelf on 'fpath', then extracts the value of tag 'tag'.
|
|
|
|
Example:
|
|
|
|
>>> get_elf_arch_tag('output', 'arm-none-linux-gnueabi-',
|
|
|
|
'bin/busybox', 'Tag_CPU_arch')
|
|
|
|
v5TEJ
|
|
|
|
>>>
|
|
|
|
"""
|
2017-07-10 01:21:19 +02:00
|
|
|
cmd = ["host/bin/{}-readelf".format(prefix),
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
"-A", os.path.join("target", fpath)]
|
|
|
|
out = subprocess.check_output(cmd, cwd=builddir, env={"LANG": "C"})
|
|
|
|
regexp = re.compile("^ {}: (.*)$".format(tag))
|
|
|
|
for line in out.splitlines():
|
|
|
|
m = regexp.match(line)
|
|
|
|
if not m:
|
|
|
|
continue
|
|
|
|
return m.group(1)
|
|
|
|
return None
|
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
def get_file_arch(builddir, prefix, fpath):
|
|
|
|
return get_elf_arch_tag(builddir, prefix, fpath, "Tag_CPU_arch")
|
|
|
|
|
2017-10-05 23:42:09 +02:00
|
|
|
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
def get_elf_prog_interpreter(builddir, prefix, fpath):
|
2017-05-16 22:45:30 +02:00
|
|
|
"""
|
|
|
|
Runs the cross readelf on 'fpath' to extract the program interpreter
|
|
|
|
name and returns it.
|
|
|
|
Example:
|
|
|
|
>>> get_elf_prog_interpreter('br-tests/TestExternalToolchainLinaroArm',
|
|
|
|
'arm-linux-gnueabihf',
|
|
|
|
'bin/busybox')
|
|
|
|
/lib/ld-linux-armhf.so.3
|
|
|
|
>>>
|
|
|
|
"""
|
2017-07-10 01:21:19 +02:00
|
|
|
cmd = ["host/bin/{}-readelf".format(prefix),
|
support/testing: core testing infrastructure
This commit adds the core of a new testing infrastructure that allows to
perform runtime testing of Buildroot generated systems. This
infrastructure uses the Python unittest logic as its foundation.
This core infrastructure commit includes the following aspects:
- A base test class, called BRTest, defined in
support/testing/infra/basetest.py. This base test class inherited
from the Python provided unittest.TestCase, and must be subclassed by
all Buildroot test cases.
Its main purpose is to provide the Python unittest setUp() and
tearDown() methods. In our case, setUp() takes care of building the
Buildroot system described in the test case, and instantiate the
Emulator object in case runtime testing is needed. The tearDown()
method simply cleans things up (stop the emulator, remove the output
directory).
- A Builder class, defined in support/testing/infra/builder.py, simply
responsible for building the Buildroot system in each test case.
- An Emulator class, defined in support/testing/infra/emulator.py,
responsible for running the generated system under Qemu, allowing
each test case to run arbitrary commands inside the emulated system.
- A run-tests script, which is the entry point to start the tests.
Even though I wrote the original version of this small infrastructure, a
huge amount of rework and improvement has been done by Maxime
Hadjinlian, and squashed into this patch. So many thanks to Maxime for
cleaning up and improving my Python code!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-03-20 21:36:50 +01:00
|
|
|
"-l", os.path.join("target", fpath)]
|
|
|
|
out = subprocess.check_output(cmd, cwd=builddir, env={"LANG": "C"})
|
|
|
|
regexp = re.compile("^ *\[Requesting program interpreter: (.*)\]$")
|
|
|
|
for line in out.splitlines():
|
|
|
|
m = regexp.match(line)
|
|
|
|
if not m:
|
|
|
|
continue
|
|
|
|
return m.group(1)
|
|
|
|
return None
|