Commit Graph

664 Commits

Author SHA1 Message Date
Julien Olivain
2f507f1da5 support/testing: fix trace-cmd runtime test
The trace-cmd runtime test has a typo and fails with output:

    Traceback (most recent call last):
      File "/build/buildroot/support/testing/tests/package/test_trace_cmd.py", line 53, in test_run
        self.assertEquals(exit_code, 0)
        ^^^^^^^^^^^^^^^^^
    AttributeError: 'TestTraceCmd' object has no attribute 'assertEquals'. Did you mean: 'assertEqual'?

The issue can be reproduced with the command:

    support/testing/run-tests \
        -d dl -o output_test \
        tests.package.test_trace_cmd

This commit fixes the issue by removing the extra 's'.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-04-01 22:05:01 +02:00
Julien Olivain
dc101babfd support/testing: add ed runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-04-01 16:28:10 +02:00
Adam Duskett
a92e7c3cec support/testing/tests/package/test_flutter.py: use flutter-markdown-example
The maintainers of the flutter-gallery package archived the project as of
February 16, 2024. In addition, the flutter-gallery package is incompatible
with Flutter 3.19.x. Because of these problems, using the flutter-gallery
package as the testing application for Flutter is no longer reasonable nor
maintainable.

However, it is reasonable to use the flutter-markdown-example package from
flutter-packages, as it is a first-party application updated regularly and
often automatically, ensuring compatibility with the latest versions of
Flutter.

  - Switch the package used for Flutter testing from flutter-gallery to
    flutter-markdown-example

  - Rename flutter-gallery.service to flutter-markdown-example.service

  - Change /usr/share/flutter/gallery/release/ to
    /usr/share/flutter/flutter_markdown_example/release/

  - Run `systemctl is-active flutter-markdown-example` instead of
   `systemctl is-active flutter-gallery`

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-26 00:05:37 +01:00
Julien Olivain
bc76d786a4 support/testing: add bitcoin runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-24 17:56:35 +01:00
Julien Olivain
2bf3dc5b84 support/testing: add iptables runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-24 16:05:47 +01:00
Fabrice Fontaine
53a8c5150e toolchain: drop codesourcery arm/aarch64 toolchains
codesourcery arm/aarch64 toolchains are old (2014) and use glibc
2.18/2.20 which are not compatible with 64-bit time_t raising the
following build failure with libcgroup since commit
1c2dbcdcf0:

In file included from ./libcgroup-internal.h:25:0,
                 from parse.y:21:
/home/buildroot/autobuild/run/instance-3/output-1/host/arm-buildroot-linux-gnueabi/sysroot/usr/include/fts.h:41:3: error: #error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64"
 # error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64"
   ^

Fixes: 1c2dbcdcf0
 - http://autobuild.buildroot.org/results/e28f955f2b360f6e7bb231a5a3800cfbd17a23d7

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: add Config.in.legacy entries]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-14 19:47:51 +01:00
Peter Korsgaard
216e99e9b4 Merge branch 'next' 2024-03-05 21:42:55 +01:00
Marcus Hoffmann
2ce680b991 package/python-jc: new package
All dependencies are optional, and thus only mentioned in the package
help text.

Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
[Arnout:
 - add to DEVELOPERS;
 - add BSD-3-Clause license for vendored pbPlist.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-03 21:32:30 +01:00
Julien Olivain
1140bfd950 support/testing: mdadm: improve test robustness on slow runners
As expected by Peter in [1], the hardcoded 3 seconds for waiting the
RAID array to rebuild are not enough on slow test host runners. This
test already failed at least once for that reason, in [2].

In order to fix those failures, this commit adds extra logic to allow
several attempts, before failing. The timeout is currently set at 10
attempts, waiting 3 seconds between each attempts. To help even more,
those 3 seconds are also scaled with the timeout_multiplier.

Fixes: [2]

[1] https://lists.buildroot.org/pipermail/buildroot/2024-February/685034.html
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/6137469690

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-03 17:30:52 +01:00
Julien Olivain
9047058cd8 support/testing: add sox runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-25 08:44:22 +01:00
Romain Naour
a9990552eb support/testing: TestCheckPackage: update expected strings
Commit dfed5acb56 ("utils/check-package: use https for the manual URL")
replaced the default url to the Buildroot manual while it was used
by TestCheckPackage test.

Update TestCheckPackage with https url.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6224243484

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-21 18:04:43 +01:00
Marcus Hoffmann
dc3f1faa8b support/testing: remove hardcoded sleep from python-django test
Instead of waiting for a hardcoded time of 30s we check periodically every
second if the server is already up. If it isn't up after the full timeout
(which is the same as before) expired the test fails.

We need to redirect all output of the background started task to
/dev/null now as it otherwise confuses the emulator.run() exit code
parsing logic (as it gets out of order messages from the emulator).

Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
yann.morin.1998@free.fr: simplify assert test]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-20 17:44:18 +01:00
Marcus Hoffmann
cbe0f4de67 support/testing: remove hardcoded sleep from python-flask test
Similar to the new fastapi test, instead of waiting for a hard coded
amount of time we can retry every second until the server is available
and abort if after the timeout we still didn't manage to connect.

Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 23:12:03 +01:00
Julien Olivain
e728889e8c support/testing: add iozone runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 22:49:38 +01:00
Julien Olivain
72c4c5740d support/testing: add ghostscript runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 22:49:37 +01:00
Julien Olivain
3efc611f53 support/testing: libgpgme: increase gpg key generation timeout
gpg key generation can take longer than the default timeout on a
loaded or slow test host. The commit increase the timeout for the
key generation command to prevent the test to randomly fail.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-11 18:14:34 +01:00
Julien Olivain
74ffd0a4df support/testing: add jailhouse runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 16:11:15 +01:00
Julien Olivain
f44c0807c6 support/testing: gnupg2: increase a timeout
The symmetric encryption test can sometimes take longer than the default
timeout. This commit increase the timeout to 10 seconds for that
command.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 20:49:15 +01:00
Julien Olivain
cd56ac9eb6 support/testing: add optee-os runtime test
Cc: Etienne Carriere <etienne.carriere@foss.st.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 16:36:31 +01:00
Romain Naour
dc25b8d99f support/testing: use core2duo cpu emulation for TestGlxinfo
TestGlxinfo fail with a new runtime error:

  # glxinfo -B -display :0
  name of display: :0
  traps: glxinfo[84] trap invalid opcode ip:b73c7027 sp:bf8433c0 error:0 in swrast_dri.so[b6e4c000+64f000]
  Illegal instruction

The x86-core2 Bootlin toolchains are built for a core2 CPU [0],
this means that the Bootlin toolchains may use core2-specific
instructions.

The TestGlxinfo test is setup for BR2_x86_core2, so our
executables will also contain core2 instructions.

However, the default Qemu x86 is not guaranteed to emulate all the
instructions specific to core2, causing runtime issues as reported
above.

A similar issue has been fixed by adding Nehalem cpu emulation on
the qemu command line. See 4f565b5222 ("support/testing: use Nehalem
cpu emulation for TestGrubX8664EFI").

Set core2duo cpu emulation for TestGlxinfo on the qemu command line.

[0] https://gitlab.com/buildroot.org/toolchains-builder/-/blob/kubu/toolchain-builder-2023.08/configs/arch/x86-core2.config?ref_type=heads

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 16:18:27 +01:00
Marcus Hoffmann
8a232ee101 support/testing: add python-uvloop runtime test
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-07 12:38:15 +01:00
Julien Olivain
22f9ce63a3 support/testing: add libcamera runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 23:26:02 +01:00
Adam Duskett
dcf0c3612d package/python-crossbar: drop package
The current package has not recieved an update since Sat Oct 9 2021
33ece2446e and is not python 3.12 compatible.

Furthermore, the current version requires at least 42 new packages worth of
depedencies of which several require patches to be python 3.12 compatible.
As nobody has stepped up to maintain the package and its ever-growing list of
dependencies, along with the other problems, it is time to drop the package.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-06 23:21:20 +01:00
Julien Olivain
c7bbea9a6c support/testing: add thttpd runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 22:52:29 +01:00
Julien Olivain
26b7e0f282 support/testing: new acl runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 22:49:08 +01:00
Julien Olivain
506073c3ed support/testing: add numactl runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 22:08:50 +01:00
Julien Olivain
6338bdfc1c support/testing: add mdadm runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 21:54:20 +01:00
Julien Olivain
9add4cbfea support/testing: add a postgresql runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 21:49:49 +01:00
Julien Olivain
feab8290a4 support/testing: new jq runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 18:33:49 +01:00
Marcus Hoffmann
4e5a25a112 support/testing: add fastapi runtime test
Add a runtime test for fastapi. Use uvicorn as the asgi server
application as does the fastapi hello world example [1].

Fastapi depends on PydanticV2 now which is written in rust so we need to
run the test on armv7.

[1] https://fastapi.tiangolo.com/tutorial/first-steps/

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
[Arnout:
 - fix flake8 errors
support/testing/tests/package/sample_python_fastapi.py:5:1: E302 expected 2 blank lines, found 1
support/testing/tests/package/sample_python_fastapi.py:8:1: W391 blank line at end of file
 - Remove BR2_CCACHE (as requested by Marcus).
 - Add a comment explaining that this also tests uvicorn and pydantic.
 - Re-try wget in a loop instead of a fixed timeout of 30 seconds.
 - Add a DEVELOPERS entry.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-02-06 13:31:48 +01:00
Marcus Hoffmann
420d34a8ad support/testing: remove unused import from sample_python_crossbar
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-05 23:49:15 +01:00
Julien Olivain
b63c213794 support/testing: new sqlite runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 23:38:20 +01:00
Julien Olivain
e03513c773 support/testing: add a "file" package runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 23:27:35 +01:00
Julien Olivain
9ce02d84ad support/testing: add usbutils runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
[Peter: add comment about USB keyboard/mouse]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 23:20:25 +01:00
Julien Olivain
3b342b7793 support/testing: add trace-cmd runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 23:20:23 +01:00
Julien Olivain
d64b36ba15 support/testing: add ltrace runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 23:20:22 +01:00
Julien Olivain
f72ba67d08 support/testing: netcat: improve test robustness
Under some conditions (mostly slow execution due to test host load),
the netcat runtime test can randomly fail. This is due to several
facts:

- the sleep time between the server and client is too short,
- the use of netcat option -c could close the connection before the
  server could receive all the data.

This commit improves the test robustness by increasing the sleep time,
and by reducing the amount of transferred data (from 1MB of random data,
to a simpler string of few bytes). Also, to make sure netcat cannot wait
on DNS resolution, this commit also adds the -n option.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6093854664

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
2024-02-05 23:19:26 +01:00
Julien Olivain
cc2ff8a38e support/testing: add fping runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
[Peter: scale delay with timout_multiplier]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 22:42:11 +01:00
Julien Olivain
1dd39dcf12 support/testing: add cryptsetup runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-05 22:36:42 +01:00
Romain Naour
4f565b5222 support/testing: use Nehalem cpu emulation for TestGrubX8664EFI
In commit 7e0e6e3b86 (toolchain/toolchain-external/toolchain-external-bootlin:
update to 2023.11-1) the TestGrubX8664EFI fails with this error:

  # efivar -l
  traps: efivar[86] trap invalid opcode ip:7fc187f4c7f4 sp:7fff9bbaa930 error:0 in libefivar.so.1.38[7fc187f4c000+16000]
  Illegal instruction

This error can be reproduced by installing other packages like "file".

The 2023.11-1 Bootlin toolchains are built for a corei7 CPU [0], which
is in fact a Nehalem CPU; we switched to the new names in commit
653fa001f3 (arch/Config.in.x86: add "newer" names for several Intel
x86 CPU variants). This means that the Bootlin toolchains may use
Nehalem-specific instructions.

The TestGrubX8664EFI test is also setup for BR2_x86_corei7, so our
executables will also contain Nehalem instructions.

However, the default Qemu x86_64 is not guaranteed to emulate all the
instructions specific to Nehalem, causing runtime issues as reported
above.

A similar issue has been fixed in toolchain-builder by adding Nehalem
cpu emulation on the qemu command line [0].

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/6093853712

[0] f2b253732b

Signed-off-by: Romain Naour <romain.naour@smile.fr>
[yann.morin.1998@free.fr:
  - reword the commit log to explain corei7 -> Nehalem equivalence
  - note that the toolchain-builder *and* our test target corei7, thus
    Nehalem
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-05 17:02:24 +01:00
Julien Olivain
0f50c4cd2b support/testing: add pciutils runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-05 16:12:36 +01:00
Marcus Hoffmann
8f7e112ab2 support/testing: add minimalmodbus runtime test
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
[yann.morin.1998@free.fr: add the comment to explain failure is success]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-05 16:06:19 +01:00
Julien Olivain
c47da268be support/testing: add coremark test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 15:55:54 +01:00
Julien Olivain
b89435db60 support/testing: add tesseract-ocr test
Cc: Gilles Talis <gilles.talis@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 15:55:53 +01:00
Julien Olivain
094fb84320 support/testing: micropython: add micropython-lib test
Commit 8f403f0 "package/micropython-lib: merge with, and install
as part of micropython" brought micropython library within the
package.

This commit improves the micropython runtime test by enabling the
micropython-lib and by also adding a runtime test using one of its
module. We choose to use the gzip module, and check whther decodign q
simple gziped file works; the micropython-lib gzip can only decompress,
so we need to prepare it from the shell.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-05 12:10:03 +01:00
Julien Olivain
035b4ee8a9 support/testing: add tcpdump runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 11:34:11 +01:00
Julien Olivain
988a2115c2 support/testing: add strace runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-05 11:31:27 +01:00
Romain Naour
393f549418 support/testing: TestPythonPy3SciPy: increase rootfs size to 250M
The rootfs generated TestPythonPy3SciPy by doesn't fit anymore in 120M.

Increase the rootfs size to 250M and checking the amount of space
available on the file system.

  Filesystem                Size      Used Available Use% Mounted on
  /dev/root               232.3M    134.5M     85.3M  61% /

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/5880448601

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Reviewed-by: Antoine Coutant <antoine.coutant@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
2024-02-04 22:16:02 +01:00
Julien Olivain
4316a4c081 support/testing: add sed runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-01-21 16:52:09 +01:00
Julien Olivain
0c6ffa5513 support/testing: add micropython runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
[me: rework the test mostly for eye-candyness]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-01-21 14:45:40 +01:00