kumquat-buildroot/support/scripts/boot-qemu-image.py

85 lines
2.5 KiB
Python
Raw Normal View History

2020-02-17 21:50:29 +01:00
#!/usr/bin/env python3
# This script expect to run from the Buildroot top directory.
support/scripts/boot-qemu-image.py: wait before using expect As reported by a gitlab runtime test [1] and on the mailing list [2], some runtime tests are failing on slow host machines when the qemu-system-<arch> is missing on the host. The boot-qemu-image.py script need to wait some time after calling pexpect.spawn() in order to make sure that the qemu process has been executed in start-qemu.sh. If start-qemu.sh failed due to missing qemu-system binary an exception will be thrown by child.expect() and should be catched by the error handling (pexpect.EOF). After spending a lot of time to investigate with Yann E. MORIN [3]. It seems that short-lived child processes are a corner-case that is not very correctly handled... Without adding a sleep(1), child.expect() can trigger an exception before setting the exitstatus of the spawned process. This issue can be reproduced on a gitlab runner or by adding "exit 1" in the first line of start-qemu.sh (after the shebang). There is even the same workaround in some pexpect examples [4]. Thanks to Yann for the help while investigating the issue. Tested: https://gitlab.com/kubu93/buildroot/pipelines/138472925 [1] https://gitlab.com/kubu93/buildroot/pipelines/135487475 [2] http://lists.busybox.net/pipermail/buildroot/2020-April/280037.html [3] http://patchwork.ozlabs.org/project/buildroot/patch/20200418161023.1221799-1-romain.naour@gmail.com/ [4] https://github.com/pexpect/pexpect/blob/master/examples/ssh_tunnel.py#L80 Fixes: https://gitlab.com/kubu93/buildroot/-/jobs/509053135 Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> [yann.morin.1998@free.fr: reorder imports] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-22 01:00:05 +02:00
import os
2020-02-17 21:50:29 +01:00
import pexpect
import sys
support/scripts/boot-qemu-image.py: wait before using expect As reported by a gitlab runtime test [1] and on the mailing list [2], some runtime tests are failing on slow host machines when the qemu-system-<arch> is missing on the host. The boot-qemu-image.py script need to wait some time after calling pexpect.spawn() in order to make sure that the qemu process has been executed in start-qemu.sh. If start-qemu.sh failed due to missing qemu-system binary an exception will be thrown by child.expect() and should be catched by the error handling (pexpect.EOF). After spending a lot of time to investigate with Yann E. MORIN [3]. It seems that short-lived child processes are a corner-case that is not very correctly handled... Without adding a sleep(1), child.expect() can trigger an exception before setting the exitstatus of the spawned process. This issue can be reproduced on a gitlab runner or by adding "exit 1" in the first line of start-qemu.sh (after the shebang). There is even the same workaround in some pexpect examples [4]. Thanks to Yann for the help while investigating the issue. Tested: https://gitlab.com/kubu93/buildroot/pipelines/138472925 [1] https://gitlab.com/kubu93/buildroot/pipelines/135487475 [2] http://lists.busybox.net/pipermail/buildroot/2020-April/280037.html [3] http://patchwork.ozlabs.org/project/buildroot/patch/20200418161023.1221799-1-romain.naour@gmail.com/ [4] https://github.com/pexpect/pexpect/blob/master/examples/ssh_tunnel.py#L80 Fixes: https://gitlab.com/kubu93/buildroot/-/jobs/509053135 Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> [yann.morin.1998@free.fr: reorder imports] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-22 01:00:05 +02:00
import time
2020-02-17 21:50:29 +01:00
def main():
if not (len(sys.argv) == 2):
print("Error: incorrect number of arguments")
print("""Usage: boot-qemu-image.py <qemu_arch_defconfig>""")
sys.exit(1)
# Ignore non Qemu defconfig
if not sys.argv[1].startswith('qemu_'):
sys.exit(0)
if not os.path.exists('output/images/start-qemu.sh'):
print('qemu-start.sh is missing, cannot test.')
sys.exit(0)
2020-02-17 21:50:29 +01:00
qemu_start = os.path.join(os.getcwd(), 'output/images/start-qemu.sh')
child = pexpect.spawn(qemu_start, ['serial-only'],
timeout=5, encoding='utf-8',
env={"QEMU_AUDIO_DRV": "none"})
# We want only stdout into the log to avoid double echo
child.logfile = sys.stdout
support/scripts/boot-qemu-image.py: wait before using expect As reported by a gitlab runtime test [1] and on the mailing list [2], some runtime tests are failing on slow host machines when the qemu-system-<arch> is missing on the host. The boot-qemu-image.py script need to wait some time after calling pexpect.spawn() in order to make sure that the qemu process has been executed in start-qemu.sh. If start-qemu.sh failed due to missing qemu-system binary an exception will be thrown by child.expect() and should be catched by the error handling (pexpect.EOF). After spending a lot of time to investigate with Yann E. MORIN [3]. It seems that short-lived child processes are a corner-case that is not very correctly handled... Without adding a sleep(1), child.expect() can trigger an exception before setting the exitstatus of the spawned process. This issue can be reproduced on a gitlab runner or by adding "exit 1" in the first line of start-qemu.sh (after the shebang). There is even the same workaround in some pexpect examples [4]. Thanks to Yann for the help while investigating the issue. Tested: https://gitlab.com/kubu93/buildroot/pipelines/138472925 [1] https://gitlab.com/kubu93/buildroot/pipelines/135487475 [2] http://lists.busybox.net/pipermail/buildroot/2020-April/280037.html [3] http://patchwork.ozlabs.org/project/buildroot/patch/20200418161023.1221799-1-romain.naour@gmail.com/ [4] https://github.com/pexpect/pexpect/blob/master/examples/ssh_tunnel.py#L80 Fixes: https://gitlab.com/kubu93/buildroot/-/jobs/509053135 Signed-off-by: Romain Naour <romain.naour@gmail.com> Cc: Yann E. MORIN <yann.morin.1998@free.fr> [yann.morin.1998@free.fr: reorder imports] Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-22 01:00:05 +02:00
# Let the spawn actually try to fork+exec to the wrapper, and then
# let the wrapper exec the qemu process.
time.sleep(1)
2020-02-17 21:50:29 +01:00
try:
child.expect(["buildroot login:", pexpect.TIMEOUT], timeout=60)
except pexpect.EOF as e:
# Some emulations require a fork of qemu-system, which may be
# missing on the system, and is not provided by Buildroot.
# In this case, spawn above will succeed at starting the wrapper
# start-qemu.sh, but that one will fail (exit with 127) in such
# a situation.
exit = [int(l.split(' ')[1])
for l in e.value.splitlines()
if l.startswith('exitstatus: ')]
if len(exit) and exit[0] == 127:
print('qemu-start.sh could not find the qemu binary')
sys.exit(0)
print("Connection problem, exiting.")
sys.exit(1)
except pexpect.TIMEOUT:
print("System did not boot in time, exiting.")
sys.exit(1)
child.sendline("root\r")
try:
child.expect(["# ", pexpect.TIMEOUT], timeout=60)
except pexpect.EOF:
print("Cannot connect to shell")
sys.exit(1)
except pexpect.TIMEOUT:
print("Timeout while waiting for shell")
sys.exit(1)
child.sendline("poweroff\r")
try:
child.expect(["System halted", pexpect.TIMEOUT], timeout=60)
child.expect(pexpect.EOF)
except pexpect.EOF:
pass
except pexpect.TIMEOUT:
# Qemu may not exit properly after "System halted", ignore.
print("Cannot halt machine")
sys.exit(0)
if __name__ == "__main__":
main()