NetCube-Systems-Austria/kumquat-buildroot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

support/testing: improve python-magic-wormhole test reliability

Browse Source 
The python-magic-wormhole runtime test can randomly fail on slow
runners, see [1].

The issue is that the sending command is started first in background
_without_ redirecting its output to /dev/null. The receiving command
is started after, expecting the message to be printed on the first
standard output line. On slower systems, the sending command still
print messages while the test controller expect output from the
receiving command. The expected string finally appear, but not on the
first line. This makes the test fail.

This commit fixes the issue by redirecting all outputs (stdout, stderr)
of the sending command to /dev/null. To help even more, the sleep time
is moved from the emulator to the test controller. The sleep time is
also multiplied by the timeout_multiplier.

Fixes: [1]

[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/6888691508

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
This commit is contained in:
Julien Olivain 2024-05-21 21:44:41 +02:00 committed by Romain Naour
parent 1263adf7a9
commit 9a734700d7
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
support/testing/tests/package/test_python_magic_wormhole.py
View File

@ -1,4 +1,5 @@
import os import os
import time
from tests.package.test_python import TestPythonPackageBase from tests.package.test_python import TestPythonPackageBase
@ -46,9 +47,12 @@ class TestPythonPy3MagicWormhole(TestPythonPackageBase):
wormhole_cmd = "wormhole --relay-url={} --transit-helper={}".format( wormhole_cmd = "wormhole --relay-url={} --transit-helper={}".format(
relay_url, transit_helper) relay_url, transit_helper)
cmd = wormhole_cmd + " send --code={} --text=\"{}\" & ".format(code, text) cmd = wormhole_cmd
cmd += "sleep 25" cmd += f" send --code={code} --text=\"{text}\""
self.assertRunOk(cmd, timeout=30) cmd += " &> /dev/null &"
self.assertRunOk(cmd)
time.sleep(30 * self.timeout_multiplier)
wormhole_env = "_MAGIC_WORMHOLE_TEST_KEY_TIMER=100 " wormhole_env = "_MAGIC_WORMHOLE_TEST_KEY_TIMER=100 "
wormhole_env += "_MAGIC_WORMHOLE_TEST_VERIFY_TIMER=100 " wormhole_env += "_MAGIC_WORMHOLE_TEST_VERIFY_TIMER=100 "