Commit Graph

1839 Commits

Author SHA1 Message Date
Julien Olivain
2b8ca3cfbe support/testing: add fwts runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 7c1faf997f)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-16 11:41:28 +02:00
Julien Olivain
1f383f120e support/testing: add iperf runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 6317f6c9a4)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-16 11:35:30 +02:00
Romain Naour
0b892513d0 support/testing: TestF2FS: update kernel to 4.19.310
Since the toolchain Bootlin update to 2023.11-1 [1], the arm Linux
kernel build is broken with binutils >= 2.41 with:

  arch/arm/mm/proc-v7.S: Assembler messages:
  arch/arm/mm/proc-v7.S:640: Error: junk at end of line, first unrecognized character is `#'

A similar issue has already be fixed for qemu m68k [2].

Bump to the latest kernel 4.19 that already include the backport
of 790756c7e022 ("ARM: 8933/1: replace Sun/Solaris style flag on section directive")

[1] 7e0e6e3b86
[2] a1ce9474e4

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6703222383

Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit 7e126bd38d)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-15 15:25:16 +02:00
Romain Naour
c8a229c850 support/testing: TestATFAllwinner: add missing openssl dependency
The commit [1] updated the u-boot version with the one used by
orangepi_zero_plus2_defconfig but the dependency on openssl
was forgotten.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6703221868

[1] eb16148ddd

Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit bc75b09b1c)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-15 15:23:39 +02:00
Robert Marko
f5a347c1dc kconfig/lxdialog: fix check() with GCC14
GCC14 now treats implicit int types as error so when check() from
check-lxdialog.sh is called to check whether we can link against ncurses
it will fail silently and the help text indicating to install ncurses is
printed.

However, this is not due to missing ncurses but once the stderr redirect
to /dev/null is removed we can see the root cause:
<stdin>:2:1: error: return type defaults to ‘int’ [-Wimplicit-int]

So, in order for menuconfig to work with GCC14 lets just specify the
return type of main() as int.

Npte that the upstream kconfig in the linux kernel source tree no longer
carries or uses the check-lxdialog.sh script since commit 1c5af5cf9308
(kconfig: refactor ncurses package checks for building mconf and nconf),
so there is no commit we can backport to our kconfig copy.

Signed-off-by: Robert Marko <robimarko@gmail.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>
Tested-by: Petr Vorel <petr.vorel@gmail.com>
[yann.morin.1998@free.fr: add note about upstream kernel]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit a6210d28db)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-15 15:18:09 +02:00
Peter Korsgaard
0155095c55 Update for 2024.02.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-07 09:30:06 +02:00
Romain Naour
0bbae94e22 support/testing: remove TestATFVexpress
TestATFVexpress is using vexpress_aemv8a_juno as as u-boot defconfig
but the Buildroot defconfig of this board was removed in 2022.11 [1]

Since both TestATFVexpress and TestATFAllwinner are now using mainline
ATF, we don't really need several ATF test anymore. Initially [2],
several runtime test were added to test ATF/U-Boot combinations when
ATF was provided by a vendor: vexpress (mainline), Allwinner and
Marvell.

Keep TestATFAllwinner as ATF mainline test.

[1] 347c108738
[2] 8cf3ce04e9

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 107bcd536d)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-06 23:01:24 +02:00
Romain Naour
2e92960756 support/testing: sync TestATFAllwinner with orangepi_zero_plus2_defconfig
u-boot-2021.04 seems to be broken when pylibfdt support is enabled
and the latest python3/setuptools are used.

Since the TestATFAllwinner is using bananapi_m64 as u-boot defconfig
but the Buildroot defconfig of this board was removed in 2022.11 [1]
update TestATFAllwinner to use a newer BSP. Use the one provided
by orangepi_zero_plus2_defconfig.

[1] daf3c6661f

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6477656317 (TestATFAllwinner)

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit eb16148ddd)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-06 22:59:50 +02:00
Romain Naour
3948fb3327 support/testing: TestFileCapabilities: update kernel to 4.19.310
Since the toolchain Bootlin update to 2023.11-1 [1], the arm Linux
kernel build is broken with binutils >= 2.41 with:

  arch/arm/mm/proc-v7.S: Assembler messages:
  arch/arm/mm/proc-v7.S:640: Error: junk at end of line, first unrecognized character is `#'

A similar issue has already be fixed for qemu m68k [2].

Bump to the latest kernel 4.19 that already include the backport
of 790756c7e022 ("ARM: 8933/1: replace Sun/Solaris style flag on section directive")

[1] 7e0e6e3b86
[2] a1ce9474e4

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6414160106 (TestFileCapabilities)

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 07ef00df9b)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-06 22:50:57 +02:00
Sen Hastings
7a5eda5009 support/scripts/pkg-stats: fix/improve git hash sorting
sortGrid()'s handling of git hashes and other large hex numbers
has been inconsistent, they can be detected as strings or numbers
depending on what type of character they start with.
This patch fixes the behaviour by using a regex to capture everything
that looks like a big hex number and treat it as a string.
This means when you sort by current version ascending all the version
strings with big hex numbers should show up first, sorted 0-9,a-f.

First we check for a string length >= 39, and then apply a regex
to return an array with every char from that string that matched
the regex. If the length of this array is still >= 39 we can assume
we are looking at something containing a git hash.

The reason why the length is defined as ">= 39" and not "40" or
"39 or 40" is twofold:

Firstly, 39 was chosen as a minimum to match stuff with 39 char git
hashes, like the rockchip-mali package.

Secondly, there is no max because we actually want to catch not
just explicitly git hashes, but any verson string with big gnarly
hex numbers in it.
Stuff like: "1.4.2-168-ged3039cdbeeb28fc0011c3585d8f7dfb91038292"

Why? Well, the idea is less about git hashes and sorting
and more about grouping similarly formatted version strings.

It would be impossble (or at least annoyingly complicated) and of
dubious utility to get a real sequential sort out of the
current version column, so the attempt here is to at the very
least collect all the similarly formatted things together.

This isn't perfect, but it's a (arguably) more useful sorted
output than before.

A demo is available here:
https://sen-h.codeberg.page/pkg-stats-demos/@pages/fix-improve-git-hash-sorting.html

Signed-off-by: Sen Hastings <sen@hastings.org>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit ce7363524c)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-02 18:30:18 +02:00
Sen Hastings
59c153dbcd support/scripts/pkg-stats: fix bug when sorting by hashfile
Because the div_class variable was not reassigned a value,
cells in the latest_version column were still being assigned
hash_file classes and so were being picked up as elements in
the hash_file sort. This lead to execessive grid gap elements
stacking up and creating huge blank spaces at the top of the page.
This is very noticable on pages with a large number of packages,
like the ones the autobuilder creates.

original behaviour(click the "Hash file" column label twice):
http://autobuild.buildroot.org/stats/master.html

demo of fixed behaviour:
https://sen-h.codeberg.page/pkg-stats-demos/@pages/fix-bug-when-sorting-by-hash-file.html

Signed-off-by: Sen Hastings <sen@hastings.org>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit 6e3d79f52e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-02 18:28:38 +02:00
Julien Olivain
bcafe4be05 support/testing: add lvm2 runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit 79f7d71640)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-05-01 20:49:28 +02:00
Julien Olivain
a2ade71578 support/testing: fix trace-cmd runtime test
The trace-cmd runtime test has a typo and fails with output:

    Traceback (most recent call last):
      File "/build/buildroot/support/testing/tests/package/test_trace_cmd.py", line 53, in test_run
        self.assertEquals(exit_code, 0)
        ^^^^^^^^^^^^^^^^^
    AttributeError: 'TestTraceCmd' object has no attribute 'assertEquals'. Did you mean: 'assertEqual'?

The issue can be reproduced with the command:

    support/testing/run-tests \
        -d dl -o output_test \
        tests.package.test_trace_cmd

This commit fixes the issue by removing the extra 's'.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 2f507f1da5)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-04-28 22:11:41 +02:00
Yann E. MORIN
4a27953e8b support/download/check-hash: fix shellcheck errors
The hash files do not use trailing backslash \ to continue lines, so
we don't want them to be interpreted thusly, so we use 'read -r'
(SC2162).

The h_file is used twice in the same loop, once for reading from it,
and once just to print it, so there is no conflict (SC2094).

Integrer variables need not be quoted (SC2086). In any case, should
there be an actual issue and they be set empty, that would cause a
runtime issue, wether they be quoted or not.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 9cb421c16f)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-04-28 21:45:59 +02:00
Yann E. MORIN
1f46f56346 support/scripts/mkusers: fix shellcheck errors
the user tables do not use trailing backslash \ to continue lines,
so we don't want them to be interpreted thusly, so we use 'read -r'
(SC2162).

Integer variables need not be quoted (SC2086). In any case, should
there be an actual issue and they be set empty, that would cause a
runtime issue, wether they be quoted or not.

The binary -o and -a ar perfectly defined in bash's test (SC2166).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 01b3053cec)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-04-28 21:44:56 +02:00
Yann E. MORIN
9698702615 support/scripts/mkusers: fix UID/GID confusion
Commit 41ea61d59c (support/scripts/mkusers: allow option for system
uid/gid) confused GID and UID variables: the GID limits were used to
create UIDs.

Fix that.

Note that this fixes a shellcheck error; although there are many more
shellcheck errors, these fixes are semantically a bug that need to be
fixed separately from the coding style issues reported by shellcheck.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Norbert Lange <nolange79@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit ebbcf5a0a7)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-04-28 21:44:20 +02:00
Julien Olivain
3c2463c9cf support/testing: add ed runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit dc101babfd)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-04-28 21:38:14 +02:00
Peter Korsgaard
9a3d34139b Update for 2024.02.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-26 10:28:48 +01:00
Julien Olivain
456b9b3fd7 support/testing: add bitcoin runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit bc76d786a4)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-25 19:14:03 +01:00
Julien Olivain
37b2a882e9 support/testing: add iptables runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit 2bf3dc5b84)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-25 09:57:36 +01:00
Yann E. MORIN
00b9f2a2f2 support/scripts: use FKIE git tree
Currently, we grab the per-year CVE feeds, in two passes: first, we grab
the meta files, and check whether something has changed since last we
downloaded it; second, we download the feed proper, unless the meta file
has not changed, in which case we use the locally cached feed.

However, it has appeared that the FKIE releases no longer provide the
meta files, which means that (once again), our daily reports are broken.

The obvious fix would be to drop the use of the meta file, and always
and unconditionally download the feeds. That's relatively trivial to do,
but the feeds are relatively big (even as xz-xompressed).

However, the CVE database from FKIE is available as a git tree. Git is
pretty good at only sending delta when updating a local copy. In
addition, the git tree, contains each CVE as an individual file, so it
is relatively easier to scan and parse.

Switch to using a local git clone.

Slightly surprisingly (but not so much either), parsing the CVE files is
much faster when using the git working copy, than it is when parsing the
per-year feeds: indeed, the per-year feeds are xz-compressed, and even
if python is slow-ish to scan a directory and opening files therein, it
is still much faster than to decompress xz files. The timing delta [0]
is ~100s before and ~10s now, about a ten time improvement, over the
whole package set.

The drawback, however, is that the git tree is much bigger on-disk, from
~55MiB for the per-year compressed feeds, to 2.1GiB for the git tree
(~366MiB) and a working copy (~1.8GiB)... Given very few people are
going to use that, that's considered acceptable...

Eventually, with a bit of hacking [1], the two pkg-stats, before and
after this change, yield the same data (except for the date and commit
hash).

[0] hacking support/scripts/pkg-stats to display the time before/after
the CVE scan, and hacking support/scripts/cve.py to do no download so
that only the CVE scan happens (and also because the meta files are no
longer available).

[1] sorting the CVE lists in json, sorting the json keys, and using the
commit from the FKIE git tree that was used for the current per-year
feeds.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit fee7efafd0)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-23 13:06:14 +01:00
Fabrice Fontaine
86d7922952 toolchain: drop codesourcery arm/aarch64 toolchains
codesourcery arm/aarch64 toolchains are old (2014) and use glibc
2.18/2.20 which are not compatible with 64-bit time_t raising the
following build failure with libcgroup since commit
1c2dbcdcf0:

In file included from ./libcgroup-internal.h:25:0,
                 from parse.y:21:
/home/buildroot/autobuild/run/instance-3/output-1/host/arm-buildroot-linux-gnueabi/sysroot/usr/include/fts.h:41:3: error: #error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64"
 # error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64"
   ^

Fixes: 1c2dbcdcf0
 - http://autobuild.buildroot.org/results/e28f955f2b360f6e7bb231a5a3800cfbd17a23d7

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: add Config.in.legacy entries]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 53a8c5150e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-21 21:43:14 +01:00
Peter Korsgaard
87e979e457 Update for 2024.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-05 14:46:21 +01:00
Julien Olivain
1140bfd950 support/testing: mdadm: improve test robustness on slow runners
As expected by Peter in [1], the hardcoded 3 seconds for waiting the
RAID array to rebuild are not enough on slow test host runners. This
test already failed at least once for that reason, in [2].

In order to fix those failures, this commit adds extra logic to allow
several attempts, before failing. The timeout is currently set at 10
attempts, waiting 3 seconds between each attempts. To help even more,
those 3 seconds are also scaled with the timeout_multiplier.

Fixes: [2]

[1] https://lists.buildroot.org/pipermail/buildroot/2024-February/685034.html
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/6137469690

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-03-03 17:30:52 +01:00
Fabrice Fontaine
fe1b081836 support/dependencies/dependencies.sh: libopenssl needs perl bigint on s390x
Commit 7ea38660e0 confused bigint [1] with
Math::BigInt [2]:

Can't locate bigint.pm in @INC (you may need to install the bigint module) (@INC contains: /home/buildroot/autobuild/instance-1/output-1/build/libopenssl-3.2.1/crypto/chacha/asm/../.. /home/buildroot/autobuild/instance-1/output-1/host/lib/perl /usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at /home/buildroot/autobuild/instance-1/output-1/build/libopenssl-3.2.1/crypto/chacha/asm/../../perlasm/s390x.pm line 16.
BEGIN failed--compilation aborted at /home/buildroot/autobuild/instance-1/output-1/build/libopenssl-3.2.1/crypto/chacha/asm/../../perlasm/s390x.pm line 16.

[1] https://perldoc.perl.org/bigint
[2] https://perldoc.perl.org/Math::BigInt

Fixes: 7ea38660e0
 - http://autobuild.buildroot.org/results/777d86a1e53dc3d6a16c829348673f1c33245a6c

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-03 13:13:40 +01:00
Peter Korsgaard
880e3ea32c Update for 2023.11.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit f05246df4f)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-03-01 14:42:14 +01:00
Yann E. MORIN
5f253e3e04 support/scripts/cve: fix running on older ijson versions
Commit 22b6945552 (support/scripts/cve.py: switch from NVD to FKIE for
the JSON files) had to change the decompressor from gz to xz, as the new
location is using xz compression.

That commit mentioned that it was spawning an external xz process to do
the decompression, on the pretence that "there is no xz decompressor in
Python stdlib."

Before version 3.1, ijson.items() only accepted a file-like object as
input (that file-like object could yield bytes() or str(), both were
supported). Starting with version 3.1, ijson.items() also accepts that
it be directly passed bytes() or str() directly. subprocess.check_output()
means we are now passing bytes() to ijson.items(), so it fails on ijson
versions before 3.1, with failures such as:

    [...]
      File "/usr/lib/python3/dist-packages/ijson/backends/python.py", line 25, in Lexer
        if type(f.read(0)) == bytetype:
    AttributeError: 'bytes' object has no attribute 'read'

Ubuntu 20.04, on which the pkg-stats run to generate the daily report,
only has ijson 2.3. More recent distros have more recent versions of
ijson, like Fedora 39 that has 3.2.3, recent enough to support being fed
bytes(). Commit 22b6945552 was tested on Fedora 39, so did not catch
the issue.

However, the reasoning in 22b6945552 is wrong: there *is* the lzma
module, at least since python 3.3 (that is, aeons ago), which is able to
read xz-compressed files; it also has an API similar to the gzip module,
and can provide a file-like object that exposes the decompressed data.

So, do just that: provide an lzma-wrapped file-like object to ijson, so
that we can eventually recover our daily reports that everything is
broken! :-]

Note that this construct still works on recent versions!

Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-29 18:47:16 +01:00
Yann E. MORIN
4983f9e73e support/scripts/pkg-stats: fix running on older python versions
Commit f71d9f49e5 (support/scripts/pkg-stats: fix datetime deprecation
warning) forgot to consider that the datetime.UTC suggested by python
3.12, was only introduced with python 3.11.

However, we are still generating the daily report on a python 3.8
version, which fails at runtime:
    AttributeError: module 'datetime' has no attribute 'UTC'

It turns out that datetime.UTC is just an alias for datetime.timezone.utc,
which seems to have existed since before python3...

Use datetime.timezone.utc instead of its alias.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-29 17:04:29 +01:00
Yann E. MORIN
75a3562324 support/scripts/pkg-stats: sort CVEs in HTML output
While the old NVD JSON feed provided data files where the CVEs were
sorted by ID, the new feed from FKIE does not have sorted CVEs.

Add a method to sort a list of CVE IDs (i.e. CVE ID strings, not CVE
objects!), and use that when emiting the HTML output.

The JSON output need not be sorted, because it is supposed to be used
for post-processing, and we do not care about the ordering there; a
consumer interested in sorting should sort on their side.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-02-28 20:16:54 +01:00
Yann E. MORIN
54f8d97c91 support/scripts/pkg-stats: adapt to NVD v2 json format
Commit 22b6945552 (support/scripts/cve.py: switch from NVD to FKIE for
the JSON files) missed the fact that the layout of the FKIE data files
are different from the original NVD ones. They are formatted according
to the NVD v2 API.

Most differences are relatively trivial fields renaming, and those are
easily spotted in this patch.

There is however one key difference in the layout of the configurations.
Where the NVD had "configurations" as an object with a "nodes" key, the
FKIE has a "configurations" as a list of objects with a single "nodes"
key; i.e. it is one-level deeper.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-02-28 20:12:08 +01:00
Yann E. MORIN
f71d9f49e5 support/scripts/pkg-stats: fix datetime deprecation warning
Abide by the warning reported with python 3.12:

    .../support/scripts/pkg-stats:1289: DeprecationWarning:
    datetime.datetime.utcnow() is deprecated and scheduled for removal
    in a future version. Use timezone-aware objects to represent
    datetimes in UTC: datetime.datetime.now(datetime.UTC).

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2024-02-28 20:11:13 +01:00
Sen Hastings
4533e42622 support/scripts/pkg-stats: make current version cell scrollable
Currently, when the version string is "too long", it is arbitrarily
truncated.

This works well for commit hashes, because usually the truncation is
long enough to provide a short hash that is still unique in the
upstream VCS.

However, there are non-hash-like versions strings that get truncated
and wihch the discriminant part is toward the end.

Yet, adapting the version cell to the widest versions string (most
probably a git hash) is not very interesting; the table is already very
large.

Make the cell with the version string scrollable: we get to keep the
best of both worlds: a narrow version cell, and a full-length version
string that can be copy-pasted if needed.

Signed-off-by: Sen Hastings <sen@hastings.org>
[yann.morin.1998@free.fr: reword commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-25 18:45:32 +01:00
Julien Olivain
9047058cd8 support/testing: add sox runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-25 08:44:22 +01:00
Romain Naour
a9990552eb support/testing: TestCheckPackage: update expected strings
Commit dfed5acb56 ("utils/check-package: use https for the manual URL")
replaced the default url to the Buildroot manual while it was used
by TestCheckPackage test.

Update TestCheckPackage with https url.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/6224243484

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-21 18:04:43 +01:00
Marcus Hoffmann
dc3f1faa8b support/testing: remove hardcoded sleep from python-django test
Instead of waiting for a hardcoded time of 30s we check periodically every
second if the server is already up. If it isn't up after the full timeout
(which is the same as before) expired the test fails.

We need to redirect all output of the background started task to
/dev/null now as it otherwise confuses the emulator.run() exit code
parsing logic (as it gets out of order messages from the emulator).

Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
yann.morin.1998@free.fr: simplify assert test]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-20 17:44:18 +01:00
Marcus Hoffmann
cbe0f4de67 support/testing: remove hardcoded sleep from python-flask test
Similar to the new fastapi test, instead of waiting for a hard coded
amount of time we can retry every second until the server is available
and abort if after the timeout we still didn't manage to connect.

Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 23:12:03 +01:00
Julien Olivain
e728889e8c support/testing: add iozone runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 22:49:38 +01:00
Julien Olivain
72c4c5740d support/testing: add ghostscript runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 22:49:37 +01:00
Julien Olivain
3efc611f53 support/testing: libgpgme: increase gpg key generation timeout
gpg key generation can take longer than the default timeout on a
loaded or slow test host. The commit increase the timeout for the
key generation command to prevent the test to randomly fail.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-11 18:14:34 +01:00
Julien Olivain
74ffd0a4df support/testing: add jailhouse runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-11 16:11:15 +01:00
Julien Olivain
f44c0807c6 support/testing: gnupg2: increase a timeout
The symmetric encryption test can sometimes take longer than the default
timeout. This commit increase the timeout to 10 seconds for that
command.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 20:49:15 +01:00
Arnout Vandecappelle
6b2d0b466c support/scripts/cve.py: start in 1999
When the CVE lookup was added in commit
4a157be9ef, the starting year of the JSON
files was set to 2002. However, there are also CVEs from 1999, 2000 and
2001. It is not clear why these were skipped back then.

Set the start year to 1999 to capture these old CVEs too.

Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 17:29:41 +01:00
Arnout Vandecappelle
22b6945552 support/scripts/cve.py: switch from NVD to FKIE for the JSON files
NVD will deprecate the v1.1 API which allows us to download the full
database as individual JSON files. Instead, there's a horribly crappy
API that is extremely slow and subject to race conditions.

Fortunately, there is a project, Fraunhofer FKIE - Cyber Analysis and
Defense [1], that goes through the effort of adapting to this new API
and regenerating the convenient JSON files. The JSON files and meta
files are re-generated daily.

Instead of implementing the NVD v2 API, we decided to just use the JSON
files generatd by fkie-cad. That saves us the effort of solving the race
conditions, devising a cache mechanism that works, handling the frequent
gateway timeouts on the NVD servers, dealing with the rate limiting, and
keeping up with changes in the API.

Switch to this repository on github as NVD_BASE_URL. The file name is
also slightly different (CVE-20XX.json instead of nvdcve-1.1-20XX.json).

The fkie-cad repository compresses with xz instead of gz. Therefore:
 - rename the filename variables to _xz instead of _gz;
 - use xz as a subprocess because there is no xz decompressor in Python
   stdlib.

[1] https://www.fkie.fraunhofer.de/en/departments/cad.html

Cc: Daniel Lang <dalang@gmx.at>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 17:29:34 +01:00
Arnout Vandecappelle
2bf75464d6 support/scripts/pkg-stats: remove checking the CPE database
We currently check the CPE database to see if the CPE ID we use
(including the version) is already in the database.

However, the version part of the CPE ID is not actually used for CVE
matching. Instead, the CVEs have a range of versions associated with
them and we match against those ranges.

In addition, NVD is moving to a new API for accessing the CPE database.
It will not longer be possible to simply download all the CPE IDs, and
due to rate limiting, the download will have to be done in several
queries.

Since all of this is anyway of limited use, drop the CPE database lookup
entirely. Instead, as long as a CPE ID is defined in a package, it is
considered OK, without any checks.

Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 17:29:13 +01:00
Julien Olivain
cd56ac9eb6 support/testing: add optee-os runtime test
Cc: Etienne Carriere <etienne.carriere@foss.st.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 16:36:31 +01:00
Antoine Coutant
3118b7e222 package/meson: bump version to 1.3.1
The meson upgrade from 1.1.1 to 1.2.0 have been reverted because
host-qemu (version v8.0.3) was searching libraries in the build machine
instead of buildroot host directory [1].

    $ grep -r LINK_ARGS  * | grep "\-L/usr"
    build.ninja: LINK_ARGS =
        -L/usr/lib/gcc/x86_64-linux-gnu/10 \
        -L/usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu \
        -L/usr/lib/gcc/x86_64-linux-gnu/10/../../../../lib -L/usr/lib \
        -L/lib/x86_64-linux-gnu \
        -L/lib/../lib \
        -L/usr/lib/../lib \
        -L/usr/lib/gcc/x86_64-linux-gnu/10/../../.. \
        -L/lib \
        -I/[...]/host/include \
        -L/[...]/host/lib \
        -I/[...]/host/include/libfdt \
        ...

Upgrading qemu to version v8.1.0 solves the problem:

    $ grep -r LINK_ARGS  * | grep "\-L/usr"
    build.ninja: LINK_ARGS =
        -I/[...]/host/include \
        -L/[...]/host/lib \
        -I/[...]/host/include/libfdt \
        ...

Those two tests were done with the same buildroot version (2023.08) on
the same machine were the bug appeared. Using, git bisect, in this
environment, it was found that the problem was fixed in qemu by the
commit [2].

Thus, it is possible that the problem was introduced by improper use of
meson by qemu <= v8.0.3. Also, meson has been updated to version 1.2.1
and 1.3.0 in yocto and it doesn't seem to create any issue. [3]

The original problem is no longer present and some projects require
meson > 1.1.1 (e.g. rusticl requires meson 1.2.0 since mesa 23.3.0 and
meson 1.3.1 since mesa 24.0.0).

"pkgconfig" field is deprecated by meson 1.3.0 and replaced by
"pkg-config". See: [4]

The patch that adds the pkg_config_static property has been rebased.

[1] acfdf21f0b
[2] https://gitlab.com/qemu-project/qemu/-/commit/Fc9a809e0d28417fa1e7f7efc19f845bda4c1be9
[3] https://github.com/yoctoproject/poky/commits/master/meta/recipes-devtools/meson
[4] https://mesonbuild.com/Release-notes-for-1-3-0.html#machine-files-pkgconfig-field-deprecated-and-replaced-by-pkgconfig

Signed-off-by: Antoine Coutant <antoine.coutant@smile.fr>
[yann.morin.1998@free.fr:
  - keep our formatting in the patch
  - reflow commit log (to keep busy while test-building!)
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-07 16:19:35 +01:00
Romain Naour
dc25b8d99f support/testing: use core2duo cpu emulation for TestGlxinfo
TestGlxinfo fail with a new runtime error:

  # glxinfo -B -display :0
  name of display: :0
  traps: glxinfo[84] trap invalid opcode ip:b73c7027 sp:bf8433c0 error:0 in swrast_dri.so[b6e4c000+64f000]
  Illegal instruction

The x86-core2 Bootlin toolchains are built for a core2 CPU [0],
this means that the Bootlin toolchains may use core2-specific
instructions.

The TestGlxinfo test is setup for BR2_x86_core2, so our
executables will also contain core2 instructions.

However, the default Qemu x86 is not guaranteed to emulate all the
instructions specific to core2, causing runtime issues as reported
above.

A similar issue has been fixed by adding Nehalem cpu emulation on
the qemu command line. See 4f565b5222 ("support/testing: use Nehalem
cpu emulation for TestGrubX8664EFI").

Set core2duo cpu emulation for TestGlxinfo on the qemu command line.

[0] https://gitlab.com/buildroot.org/toolchains-builder/-/blob/kubu/toolchain-builder-2023.08/configs/arch/x86-core2.config?ref_type=heads

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-07 16:18:27 +01:00
Marcus Hoffmann
8a232ee101 support/testing: add python-uvloop runtime test
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2024-02-07 12:38:15 +01:00
Julien Olivain
22f9ce63a3 support/testing: add libcamera runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2024-02-06 23:26:02 +01:00
Adam Duskett
dcf0c3612d package/python-crossbar: drop package
The current package has not recieved an update since Sat Oct 9 2021
33ece2446e and is not python 3.12 compatible.

Furthermore, the current version requires at least 42 new packages worth of
depedencies of which several require patches to be python 3.12 compatible.
As nobody has stepped up to maintain the package and its ever-growing list of
dependencies, along with the other problems, it is time to drop the package.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2024-02-06 23:21:20 +01:00