kumquat-buildroot/support/testing/tests/package/test_python.py

138 lines
4.4 KiB
Python
Raw Normal View History

import os
import infra.basetest
class TestPythonBase(infra.basetest.BRTest):
config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
"""
BR2_TARGET_ROOTFS_CPIO=y
# BR2_TARGET_ROOTFS_TAR is not set
"""
interpreter = "python"
def login(self):
cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
self.emulator.boot(arch="armv5",
kernel="builtin",
options=["-initrd", cpio_file])
self.emulator.login()
def version_test(self, version, timeout=-1):
cmd = self.interpreter + " --version 2>&1 | grep '^{}'".format(version)
self.assertRunOk(cmd, timeout)
def math_floor_test(self, timeout=-1):
cmd = self.interpreter + " -c 'import math; math.floor(12.3)'"
self.assertRunOk(cmd, timeout)
def libc_time_test(self, timeout=-1):
cmd = self.interpreter + " -c '"
cmd += "import ctypes;"
support/testing/infra/basetest.py: use Bootlin toolchain stable 2022.08-1 This patch updates the default toolchain used for runtime tests. The last time this toolchain was updated was in commit 0207a65323fc73fe9570f3d7e03bbfed4ab6d477 3 years ago. Since then, multiple things have changed: Firstly, it used uclibc-ng as the libc whereas since commit 4057e36ca9665edd5248512e4edba2c243b8f4be, glibc is used as the default library. And secondly, since commit 531b2a10cda6bae78ecc141c9fe48f2024f71dbc, buildroot dropped the support for gcc 8 and it cannot be built internally anymore. So the testsuite was executed using a toolchain that can't be built by the Buildroot internal toolchain backend anymore. This new Bootlin toolchain stable 2022.08-1 is based on gcc 11.3.0, linux headers 4.9.327, glibc 2.35 and binutils 2.38. The previous toolchain bleeding edge 2018.11-1 is based on gcc 8.2.0, linux headers 4.14.80, uclibc 1.0.30 and binutils 2.31.1 Nowadays Bootlin toolchains are packaged in Buildroot and we can directly select them from BASIC_TOOLCHAIN_CONFIG and avoid setting the toolchain parameters (BR2_TOOLCHAIN_EXTERNAL_CUSTOM...). The switch to Glibc requires to update some tests for the following reasons: - TestPython3Py, TestPython3Pyc and TestPython3PyPyc has been updated since they use the libc binary file name in their test (uClibc: libc.so.1 vs Glibc: libc.so.6). - TestTmux needs at least one locale to pass (as stated in tmux help text "tmux needs a working UTF-8 locale"), so use "C.UTF-8". - TestOpenSsh needs a toolchain >= 5.x due to a openssh issue (Similar to: https://bugs.busybox.net/show_bug.cgi?id=13671) Use the Bootlin toolchain bleeding-edge 2022.08-1 rhat provide kernel headers 5.4 - TestShadow needs a toolchain >= 4.14 Use the Bootlin toolchain bleeding-edge 2022.08-1 rhat provide kernel headers 5.4 Runtime tested on the gcc farm server. Signed-off-by: Sebastian Weyer <sebastian.weyer@smile.fr> Signed-off-by: Romain Naour <romain.naour@gmail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-06-09 01:28:48 +02:00
cmd += "libc = ctypes.cdll.LoadLibrary(\"libc.so.6\");"
cmd += "print(libc.time(None))'"
self.assertRunOk(cmd, timeout)
def zlib_test(self, timeout=-1):
cmd = self.interpreter + " -c 'import zlib'"
_, exit_code = self.emulator.run(cmd, timeout)
self.assertEqual(exit_code, 1)
class TestPython3Pyc(TestPythonBase):
config = TestPythonBase.config + \
"""
BR2_PACKAGE_PYTHON3=y
"""
def test_run(self):
self.login()
self.version_test("Python 3")
self.math_floor_test()
self.libc_time_test()
self.zlib_test()
support/testing: create default test case for python packages Test cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So create a new helper class named TestPythonPackageBase that holds all the logic to run a script on the target. TestPythonPackageBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then automatically logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter. An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds). A simple test case for a package that only supports Python 2 will look like this: |from tests.package.test_python import TestPythonPackageBase | | |class TestPythonPy2<Package>(TestPythonPackageBase): | __test__ = True | config = TestPythonPackageBase.config + \ | """ | BR2_PACKAGE_PYTHON=y | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15 Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Asaf Kahlon <asafka7@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-10 03:16:02 +01:00
class TestPython3Py(TestPythonBase):
config = TestPythonBase.config + \
"""
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON3_PY_ONLY=y
"""
def test_run(self):
self.login()
self.version_test("Python 3")
self.math_floor_test()
self.libc_time_test()
self.zlib_test()
class TestPython3PyPyc(TestPythonBase):
config = TestPythonBase.config + \
"""
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON3_PY_PYC=y
"""
def test_run(self):
self.login()
self.version_test("Python 3")
self.math_floor_test()
self.libc_time_test()
self.zlib_test()
support/testing: create default test case for python packages Test cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So create a new helper class named TestPythonPackageBase that holds all the logic to run a script on the target. TestPythonPackageBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then automatically logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter. An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds). A simple test case for a package that only supports Python 2 will look like this: |from tests.package.test_python import TestPythonPackageBase | | |class TestPythonPy2<Package>(TestPythonPackageBase): | __test__ = True | config = TestPythonPackageBase.config + \ | """ | BR2_PACKAGE_PYTHON=y | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15 Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Asaf Kahlon <asafka7@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-10 03:16:02 +01:00
class TestPythonPackageBase(TestPythonBase):
"""Common class to test a python package.
Build an image containing the scripts listed in sample_scripts, start the
emulator, login to it and for each sample script in the image run the python
interpreter passing the name of the script and check the status code is 0.
Each test case that inherits from this class must have:
__test__ = True - to let nose2 know that it is a test case
config - defconfig fragment with the packages to run the test
It also can have:
sample_scripts - list of scripts to add to the image and run on the target
timeout - timeout to the script to run when the default from the
test infra is not enough
When custom commands need be issued on the target the method
run_sample_scripts can be overridden.
"""
__test__ = False
config_sample_scripts = \
"""
BR2_ROOTFS_POST_BUILD_SCRIPT="{}"
BR2_ROOTFS_POST_SCRIPT_ARGS="{}"
""".format(infra.filepath("tests/package/copy-sample-script-to-target.sh"),
"{sample_scripts}")
sample_scripts = None
timeout = -1
def __init__(self, names):
"""Add the scripts to the target in build time."""
super(TestPythonPackageBase, self).__init__(names)
if self.sample_scripts:
scripts = [infra.filepath(s) for s in self.sample_scripts]
self.config += self.config_sample_scripts.format(sample_scripts=" ".join(scripts))
def check_sample_scripts_exist(self):
"""Check the scripts were really added to the image."""
scripts = [os.path.basename(s) for s in self.sample_scripts]
cmd = "md5sum " + " ".join(scripts)
_, exit_code = self.emulator.run(cmd)
self.assertEqual(exit_code, 0)
def run_sample_scripts(self):
"""Run each script previously added to the image."""
for script in self.sample_scripts:
cmd = self.interpreter + " " + os.path.basename(script)
self.assertRunOk(cmd, timeout=self.timeout)
support/testing: create default test case for python packages Test cases for python packages are very similar among each other: run a simple script in the target that minimally tests the package. So create a new helper class named TestPythonPackageBase that holds all the logic to run a script on the target. TestPythonPackageBase adds in build time one or more sample scripts to be run on the target. The test case for the python package must explicitly list them in the "sample_scripts" property. The test case then automatically logins to the target, checks the scripts are really in the rootfs (it calls "md5sum" instead of "ls" or "test" in an attempt to make the logfile more friendly, since someone analysing a failure can easily check the expected script was executed) and then calls the python interpreter passing the sample script as parameter. An optional property "timeout" exists for the case the sample script needs more time to run than the default timeout from the test infra (currently 5 seconds). A simple test case for a package that only supports Python 2 will look like this: |from tests.package.test_python import TestPythonPackageBase | | |class TestPythonPy2<Package>(TestPythonPackageBase): | __test__ = True | config = TestPythonPackageBase.config + \ | """ | BR2_PACKAGE_PYTHON=y | BR2_PACKAGE_PYTHON_<PACKAGE>=y | """ | sample_scripts = ["tests/package/sample_python_<package>.py"] | timeout = 15 Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Asaf Kahlon <asafka7@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Yegor Yefremov <yegorslists@googlemail.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-10 03:16:02 +01:00
def test_run(self):
self.login()
self.check_sample_scripts_exist()
self.run_sample_scripts()