support/testing: improve python-magic-wormhole test reliability

The python-magic-wormhole runtime test can randomly fail on slow
runners, see [1].

The issue is that the sending command is started first in background
_without_ redirecting its output to /dev/null. The receiving command
is started after, expecting the message to be printed on the first
standard output line. On slower systems, the sending command still
print messages while the test controller expect output from the
receiving command. The expected string finally appear, but not on the
first line. This makes the test fail.

This commit fixes the issue by redirecting all outputs (stdout, stderr)
of the sending command to /dev/null. To help even more, the sleep time
is moved from the emulator to the test controller. The sleep time is
also multiplied by the timeout_multiplier.

Fixes: [1]

[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/6888691508

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
This commit is contained in:
Julien Olivain 2024-05-21 21:44:41 +02:00 committed by Romain Naour
parent 1263adf7a9
commit 9a734700d7

View File

@ -1,4 +1,5 @@
import os
import time
from tests.package.test_python import TestPythonPackageBase
@ -46,9 +47,12 @@ class TestPythonPy3MagicWormhole(TestPythonPackageBase):
wormhole_cmd = "wormhole --relay-url={} --transit-helper={}".format(
relay_url, transit_helper)
cmd = wormhole_cmd + " send --code={} --text=\"{}\" & ".format(code, text)
cmd += "sleep 25"
self.assertRunOk(cmd, timeout=30)
cmd = wormhole_cmd
cmd += f" send --code={code} --text=\"{text}\""
cmd += " &> /dev/null &"
self.assertRunOk(cmd)
time.sleep(30 * self.timeout_multiplier)
wormhole_env = "_MAGIC_WORMHOLE_TEST_KEY_TIMER=100 "
wormhole_env += "_MAGIC_WORMHOLE_TEST_VERIFY_TIMER=100 "