Commit Graph

1348 Commits

Author SHA1 Message Date
Peter Korsgaard
aa433d1c5c Update for 2021.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-09-04 11:53:24 +02:00
Thomas Petazzoni
c938698b4a support/testing/tests: add test for check_bin_arch
This tests build a bogus package that installs a binary built for the
host architecture into $(TARGET_DIR), which should cause a build
failure, at least as long as the host architecture isn't ARM.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
[yann.morin.1998@free.fr: drop uneeded subprocess import to fix flake8]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-08-29 11:37:32 +02:00
Thomas Petazzoni
1259d40b0a support/testing/infra: add log_file_path() function
Some tests will need to grep through the build log to verify that some
features are working are expected. In order to allow them to open the
build log, we provide a new function called log_file_path(), which
returns the path to the log file if available.

We also use this function in open_log_file().

Note that open_log_file() cannot be used directly to grep through the
log file at the end of a build: because it opens in "a+" mode, it
greps starting from the end of the file.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-08-29 11:37:15 +02:00
Romain Naour
12c7a05da1 utils/test-pkg: add gitlab-ci support
The gitlab-ci support in test-pkg allows to parallelize the test-pkg
work into several gitlab jobs. It's much faster than local serialized
testing.

To trigger this, a developer will have to add, in the latest commit of
their branch, a token on its own line, followed by a configuration
fragment, e.g.:

    test-pkg config:
    SOME_OPTION=y
    # OTHER_OPTION is not set
    SOME_VARIABLE="some value"

This configuration fragment is used as input to test-pkg.

To be able to generate one job per test to run, we need the list of
tests in the parent pipeline, and the individual .config files (one per
test) in the child pipeline. We use the newly-introduced --prepare-only
mode to test-pkg, and collect all the generated .config files as
artefacts; those are inherited in the child pipeline via the
"needs::pipeline" and "needs::job" directives. This is a bit tricky,
and is best described by the Gitlab-CI documentation [0].

We also list those .config files to generate the actual list of jobs to
run in the child pipeline.

Notes:
  - if the user provides an empty fragment, this is considered an error:
    indeed, without a fragment (and the package name), there is no way
    to know what to test;
  - if that fragment yields an empty list of tests, then there is
    nothing to test either, so that is also considered an error.

[0] https://docs.gitlab.com/ee/ci/yaml/README.html#artifact-downloads-to-child-pipelines

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
  - split the change to test-pkg to its own patch
  - generate the actual yml snippet in support/scripts/generate-gitlab-ci-yml,
    listing the .config files created by test-pkg
  - some code-style-candies...
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-28 17:03:13 +02:00
Romain Naour
e5494f1fac support/testing: test_atf: bump the custom version to v2.5
This version bump is needed to pass the ATF test with
hardening option enabled (-fstack-protector-strong)

With the version v2.2, ATF fail due to undefined references:

./build/juno/release/bl2u/arm_tzc400.o: In function `arm_tzc400_setup':
arm_tzc400.c:(.text.arm_tzc400_setup+0x10): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0x18): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xb8): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xcc): undefined reference to `__stack_chk_fail'

Since commit ccac9a5bbb, Buildroot no
longer forces ENABLE_STACK_PROTECTOR. However, we rely on the ATF build
system to handle it correctly, and this wasn't the case in v2.2.

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1524842591

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-28 14:24:25 +02:00
Arnout Vandecappelle (Essensium/Mind)
7dac56b76e Merge branch 'master' into next
Conflicts:
	package/go/go.hash
	package/go/go.mk

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-26 23:40:57 +02:00
Romain Naour
5d60e07e27 support/testing: test_docker_compose: bump the kernel to 4.19.204
gcc 10.x is now used by default but the kernel 4.19 used by
test_docker_compose doesn't build with it.

Bump the kernel to 4.19.204 release that contains a lot of
fixes for newer gcc.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-24 23:59:09 +02:00
Romain Naour
f4ff135e78 support/testing: revert the last change of check_network()
check_network() must check the error code of the command
used to check the network configuration with the value
passed as argument "exitCode".

But this argument is ignored since this commit [1].

Revert the last change of check_network().

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1522848308
https://gitlab.com/kubu93/buildroot/-/jobs/1522848306

[1] afc1ed4d51

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-24 23:54:53 +02:00
Romain Naour
26f6110740 support/testing: remove TestPythonPy2{Cryptography, ServiceIdentity, Treq, Twisted, Txtorcon}
The python2 support has been removed since the python-idna bump to version 3.2 [1]

[1] 0c7e30b43a

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-24 00:26:05 +02:00
Peter Korsgaard
5284a826ae Update for 2021.05.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d5127a4de7)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-08-10 09:48:00 +02:00
Romain Naour
208e98beb2 support/misc/gitlab-ci.yml.in: templatise the make command
In a followup commit, the make command used to log and display the last
lines on error will be used in another job.
Factorize it by introducing .run_make template.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-08-05 22:30:50 +02:00
Raphaël Mélotte
58c99c957b package/python-rsa: new package
Python-RSA is a pure-Python RSA implementation.

Signed-off-by: Raphaël Mélotte <raphael.melotte@essensium.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-08-04 22:27:28 +02:00
Gleb Mazovetskiy
6fbdf51596 Makefile: Parallelize glibc locale generation
Parallelizes locale generation based on `BR2_JLEVEL` setting.

Locale generation always runs during the finalize stage and can consume
a significant amount of time. Parallelizing it greatly reduces that time
on multi-core machines.

To parallelize it, we first invoke `localedef` for every locale in
parallel with the `--no-archive` option. This creates the intermediate
locale data instead of writing to the finally archive directly.

Then, we invoke `localedef` again once to create the archive from the
intermediate compiled locale data files.

We have to do it this way because `localedef` does not do any locking
when writing to the archive file, so calling it without `--no-archive`
concurrently could result in a corrupt archive file or an archive file
that is missing some locales.

While we're at it, make two additional improvements:
- Remove locale-archive before adding to it. Otherwise, repeated
  applications of target-finalize will keep on growing the file.
- Sort the locales when creating locale-archive so its contents are
  reproducible.

We use `find` to collect the installed locales rather than LOCALES. This
makes it possible for something else (skeleton, overlay, custom package)
to create and install additional locales and still have them added to
locale-archive.

Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
[Arnout:
 - Remove -j$(PARALLEL_JOBS), it's already part of $(MAKE)
 - Remove HOST_DIR, TARGET_DIR, STAGING_DIR, they're already exported
 - Extend commit message
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-07-25 16:46:30 +02:00
Adam Duskett
db1ded1084 support/testing: add polkit tests
This test script tests polkit with and without systemd.

The Systemd test does the following:
  - The brtest user attempts to restart the systemd-timesyncd service and is
    denied.

  - A systemd-timesyncd-restart.rules file provided by polkit-rules-test
    is copied from /root/ to /etc/polkit-1/rules.d

  - The brtest user attempts to restart the systemd-timesyncd service and should
    now succeed.

The initd test does the following:
- The brtest user attempts to run the test application "hello-polkit" with the
  command "pkexec hello-polkit" and is denied.

- A hello-polkit.rules file provided by polkit-rules-test is copied from /root/
  to /etc/polkit-1/rules.d

- The brtest user attempts to re-run the test hello-polkit binary with
  "pkexec hello-polkit" and succeeds.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-07-24 23:29:21 +02:00
Yann E. MORIN
d0edfec1e2 boot/uboot: add missing dependency to host-pkgconf
Commit 2eaa6d0f36 (boot/uboot: fix uboot building host tools on x86
architecture) added use of $(PKG_CONFIG_HOST_BINARY), but forgot to add
the corresponding build-ordr dependency.

Add this missing depenency now.

Additionally, the associated test had an explicit host pkgconf enbled in
its configuration. This is superfluous now that uboot properly depends
on host-pkgconf, so drop that from the test.

Note: it hapenned to work, because host-pkgconf, when explicitly enabled
in the configuration, and without per-package directories, would build
before uboot and thus be available. This would fail with PPD, though,
and thus would break for TLPB.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-07-17 13:18:04 +02:00
Francois Perrad
d3e2e7e9a6 package/lua-lunix: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-07-13 22:19:07 +02:00
Kory Maincent
2eaa6d0f36 boot/uboot: fix uboot building host tools on x86 architecture
The make all command run the tools/makefile on the process.
This makefile use "pkg-config" command to support static link.
The issue is the use of pkg-config configured for crosscompiling
to build binaries tools for host architecture.
To fix it, I add pkg-config environment variable to configure it for host.

Add a test to avoid future regress on the build of U-boot.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
  - fix mixed space-TAB indentation
  - fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-30 22:45:18 +02:00
Yann E. MORIN
e7f0408f49 support/tests: fix squashfs test
Commit 3cf2782906 (support/testing/infra/emulator.py: update pre-built
kernels) bumped the default kernels used by the testing infra.

However, the newer armv7 kernel (at least) no longer has support for
lz4-compressed squashfs filesystems.

This breaks the squashfs test:

    Filesystem uses "lz4" compression. This is not supported
    List of all partitions:
    1f00          131072 mtdblock0
     (driver?)
    1f01           32768 mtdblock1
     (driver?)
    b300            2048 mmcblk0
     driver: mmcblk
    No filesystem could mount root, tried:
     squashfs

    Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,0)

Updating the kernel again is a little bit cumbersome, while fixing the
actual test is relatively trivial, so this is what we do: we switch
over to lzo, which is supported by the new kernel:

    # zcat /proc/config.gz |grep SQUA
    CONFIG_SQUASHFS=y
    CONFIG_SQUASHFS_ZLIB=y
    # CONFIG_SQUASHFS_LZ4 is not set
    CONFIG_SQUASHFS_LZO=y
    # CONFIG_SQUASHFS_XZ is not set

While at it, also drop the superfluous line disabling gzip compression:
it is part of a choice, so enabling one (lzo here) forcibly disables the
others (of which gzip).

Fixes: 3cf2782906

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-27 11:50:01 +02:00
Thomas Petazzoni
f37e811cd1 support/testing/tests/fs/test_ext: add missing "format" option for Qemu
Will avoid the following warning:

   WARNING: Image format was not specified for
   '/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and
   probing guessed raw. Automatically detecting the format is
   dangerous for raw images, write operations on block 0 will be
   restricted. Specify the 'raw' format explicitly to remove the
   restrictions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 22:17:07 +02:00
Thomas Petazzoni
abe32cfdf0 support/testing/tests: fix tests to use infra.img_round_power2()
All the tests that are using if=sd as a Qemu options are changed to
use infra.img_round_power2() instead of simply extending the size of
the image to the next MB boundary, which is not longer sufficient with
Qemu >= 5.1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: drop now-useless imports]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 22:16:40 +02:00
Thomas Petazzoni
37a1af7a74 support/testing/infra: add img_round_power2() function
Since Qemu 5.1, SD card images must have a size that are a power of
two. While some filesystem (such as ext2/3/4) allow to specify the
expected size of the filesystem, others such as SquashFS do not have
this capability.

We were already extending the size of such images to the next 1 MB
boundary using "truncate -s %1M", but that is no longer sufficient. So
instead, we introduce a helper function that extends the size of an
image to the next power of two.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - use f.trunctate() rather than subprocess.call([truncate,...])
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 21:27:48 +02:00
Thomas Petazzoni
78c42cdca2 support/testing/tests/fs/test_ext: specify 16 MB as ext filesystem size
Since Qemu 5.1, the SD card size must be a power of two, so the
default size for ext2/3/4 filesystem images of 60 MB is not
suitable. Since 16 MB is used for the Ext4 test, let's use the same
value for the other tests as well (ext2, ext2r1, ext3). Without this
change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >=
5.1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 21:26:16 +02:00
Thomas Petazzoni
afc1ed4d51 support/testing: use .assertRunOk() when possible
The BRTest() class implements an assertRunOk() method that does the
very common work of running a command inside the emulator, and
checking that it is successful.

This commit changes all locations where this .assertRunOk() method can
be used, instead of open-coding the same logic.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 17:41:10 +02:00
Sergey Matyukevich
7af5a80aba support/testing/tests/boot/test_atf: update U-Boot in TestATFAllwinner
Bump U-Boot version in TestATFAllwinner. Updating U-Boot version to
2021.04 requires the following two changes.

First, after switching to binman, u-boot.itb is no more generated for
64-bit sunxi boards. Combined u-boot-sunxi-with-spl.bin image should
be used instead. This image contains SPL, U-Boot, and FIT image,
where FIT image contains other binaries such as BL31 and SCP.

Second, new U-Boot enables support for System Control Processor (SCP)
firmware. SCP firmware is included by default into FIT image in the
combined u-boot-sunxi-with-spl.bin binary. When SCP is not available
or not needed, it should be explicitly disabled by pointing to an
empty file. Support for Allwinner SCP firmware is not yet available
neither in Buildroot nor in mainline kernel. So disable it for now
using custom U-Boot build options.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 17:10:42 +02:00
Sergey Matyukevich
6d7f5c09f7 support/testing: switch TestATFAllwinner to mainline TF-A
Switch to mainline TF-A that provides basic support for H5 and A64.

Note that Allwinner platform layer in TF-A does not provide support
for GCC stack protection, so make sure to disable this TF-A feature.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-26 17:10:42 +02:00
José Luis Salvador Rufo
78f477d136 package/zfs: bump version to 2.0.5
As this version brings support for kernel up to 5.12, we update the
test cases to use the 5.12 kernel.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-06-26 15:13:03 +02:00
Thomas Petazzoni
81d1c6cf28 support/testing/tests/package/test_bmap_tools: add test for host bmap-tools
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 14:24:34 +02:00
Nicolas Carrier
1023f742b8 support/testing/tests/package/test_bmap_tools: new test
This patch implements a simple test in which a dummy file system image
is created, then `bmaptool create` and `bmaptool copy` are used to copy
it to another file.

Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: several reworks, add myself to DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 14:24:22 +02:00
Thomas Petazzoni
3cf2782906 support/testing/infra/emulator.py: update pre-built kernels
The pre-built vexpress kernel used by the testing infrastructure is a
4.0.0 kernel, which is getting old to be used with reasonably recent
toolchains.

This commit updates the pre-built kernels for both the versatile and
vexpress machines to 5.10.7 (they have already been put online).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 13:59:58 +02:00
Thomas Petazzoni
3bb260cf38 support/config-fragments/autobuild/br-arm-internal-glibc: update to bleeding edge components
The br-arm-internal-glibc.config is generally used as a configuration
to test the bleeding edge versions of components. However, it has been
lagging behind somewhat, so let's bring it up-to-date:

 - Binutils 2.36.x
 - GCC 11.x

Let the fun begin in the autobuilders!

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-25 13:44:51 +02:00
Thomas Petazzoni
6a92726981 support/testing/tests/package/test_python_augeas: new test
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-24 23:42:04 +02:00
Francois Perrad
b4fd87f993 package/lua-zlib: new package
This module is only partially compatible with lzlib (which is no longer
maintained).

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: amend commit log about limited compatibility]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-06-14 22:31:32 +02:00
Peter Korsgaard
8d07baab43 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-06-07 17:14:37 +02:00
Peter Korsgaard
69f79f2a2e Update for 2021.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-06-06 23:21:08 +02:00
Marcin Niestroj
f40219ff69 support/testing: add python-dbus-next test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 16:41:10 +02:00
Matthew Weber
50791af71f support/scripts/pkg-stats: clarify when a CVE/CPE should report as N/A
- If a package doesn't have any versioning, ignore and state that
 - If a package is virtual, CVE=ignore and CPE state virtual
 - For any of these NA cases, don't provide search link and color box
   green

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 10:21:28 +02:00
Matthew Weber
9368f3f23d support/scripts/pkg-stats: add is_actual_package() and rework has_valid_infra()
has_valid_infra() is incorrectly named; it probably should be named
is_actual_package(), and has_valid_infra() would be changed to
actually represent having an actual infra.

This resolves packages reporting as having no valid package infra and
cleans up reporting cases of CPE and CVEs where there isn't a valid version
or package definition outside Buildroot

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 10:14:31 +02:00
Matthew Weber
9b83bb1382 support/scripts/pkg-stats: verified CPE has a known id but not version
Currently a verified CPE reports the following if versions are not found
 cpe:2.3🅰️qemu:qemu:5.2.0:*:*:*:*:*:*:*
 CPE identifier unknown in CPE database (Search)

This patch clarifies the report to state the 'version' is unknown instead
of the 'identifier'.

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 09:26:50 +02:00
Ryota Kinukawa
f46e13f05f support/dependencies, scripts: accept patches with renames
Currently, patches with renames are refused, as they reqire patch 2.7
or newer. So far, we did not require that version because it was too
recent to be widely available.

But patch 2.7 has been released in 2012, almost 9 years ago now; it is
old enough that we can start relying on it.

Add a check that patch is GNU patch 2.7 or newer, and so drop the common
check for patch, and drop the check about renames in apply-patches.sh.

Signed-off-by: Ryota Kinukawa <pojiro.jp@gmail.com>
[yann.morin.1998@free.fr:
  - drop common check
  - shorten variable names
  - drop now-incorrect comment about busybox w/desktop
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 08:33:11 +02:00
Romain Naour
45a13be315 support/testing: test_hardening disable PIC/PIE
Since [1], PIC/PIE is enabled by default but the TestRelroPartial
test expect implicitely PIC/PIE being disabled.

Disable PIC/PIE from the config fragment provided by
TestRelroPartial.

[1] 810ba387be

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661757

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-05-18 22:15:36 +02:00
Matthew Weber
c2ebfe7d78 support/scripts/pkg-stats: fix flake8 E741 ambiguous variable name
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 21:06:12 +02:00
Romain Naour
4a3639bad0 support/testing: test_glxinfo load X11 modules in the right order
From [1]
"Xorg does not implement real dynamic linking and requires that its
modules get loaded in the right order."

From /var/log/Xorg.0.0.log:
 (II) LoadModule: "modesetting"
 (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
 (EE) Failed to load /usr/lib/xorg/modules/drivers/modesetting_drv.so: /usr/lib/xorg/modules/drivers/modesetting_drv.so: undefined symbol: shadowRemove

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661899

[1] https://forums.gentoo.org/viewtopic-p-8245578.html#8245578

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:49 +02:00
Romain Naour
95b722eaa8 support/testing: test_glxinfo: switch to Gallium swrast
Since the mesa3d bump to version 21.0.3 [1], the
BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST option is not supported anymore
since the mesa DRI swrast driver has been removed upstream

So, switch to Gallium swrast.

[1]15a2f9b819806d38a7d8172a20f80130b1d60e63

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:48 +02:00
Dick Olsson
6131efc62c support/testing: new boot test for EDK2
Boot a QEMU sbsa-ref machine with ATF, EDK2, GRUB2 and a minimal
kernel. This is a simple but effective test of a compliant setup.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-17 17:26:37 +02:00
Matt Weber
f3b07e2ded support/scripts/pkg-stats: add column reporting ignored CVEs
When doing analysis it is helpful to be able to view what CVE have
been patched / diagnosed to not apply to Buildroot. This exposes
that list to the reporting and prevents a step where you have to
dig into the .mk's of a pkg to check for sure what has been
ignored.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: only set background if there are ignored CVEs]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 18:22:58 +02:00
Matt Weber
c5aa3c5883 support/scripts/pkg-stats: add CPE searching links
For cases of a CPE having a unknown version or when there hasn't
been a CPE verified, proposed a search criteria to help the
user research an update.

(libcurl has NIST dict entries but not this version)
  cpe:2.3🅰️haxx:libcurl:7.76.1:*:*:*:*:*:*:*
  CPE identifier unknown in CPE database (Search)

(jitterentropy-library package doesn't have any NIST dict entries)
  no verified CPE identifier (Search)

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: fix flake8 issues]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 17:54:25 +02:00
Matt Weber
fffc553485 support/scripts/gen-missing-cpe: add new script
This script queries the list of CPE IDs for the packages of the
current configuration (based on the "make show-info" output), and:

 - for CPE IDs that do not have any matching entry in the CPE
   database, it emits a warning

 - for CPE IDs that do have a matching entry, but not with the same
   version, it generates a snippet of XML that can be used to propose
   an updated version to NIST.

Ref: NIST has a group email (cpe_dictionary@nist.gov) used to
recieve these version update and new entry xml files.  They do
process the XML and provide feedback. In some cases they will
propose back something different where the vendor or version is
slightly different.

Limitations
 - Currently any use of non-number version identifiers isn't
   supported by NIST as they use ranges to determine impact
   of a CVE
 - Any Linux version from a non-upstream is also not supported
   without manually adjusting the information as the custom
   kernel will more then likely not match the upstream version
   used in the dictionary

Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr:
  - codestyles as spotted by Arnout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 13:57:38 +02:00
Peter Korsgaard
aa31d10808 test_docker_compose.py: Test the volume mount feature
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 23:00:16 +02:00
Peter Korsgaard
4915b692c8 test_docker_compose.py: Test the port publish feature
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 22:59:58 +02:00
Romain Naour
f5ea09e2a8 support/testing: remove TestPythonPy2Colorzero
The python2 support has been removed since the python-colorzero bump version to 2.0.

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:33 +02:00
Romain Naour
9217708122 support/testing: remove TestPythonPy2Gpiozero
The python2 support has been removed since the python-colorzero bump version to 2.0.

Remove the gpiozero test with python2

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:24 +02:00
Peter Korsgaard
fdb3291b8c Update for 2021.02.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 76b4f9e9b6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-12 11:37:23 +02:00
Arnout Vandecappelle (Essensium/Mind)
74bae64dc5 support/testing: add sudo package test
Create a new user 'sudotest' to validate that sudo really works (i.e.
properly has setuid).

Creating the user and adding it to sudoers is done at runtime, otherwise
we'd need to add extra files to the config which complicates things a
little bit.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-10 22:08:15 +02:00
Adam Duskett
ab2d472cde testing/tests/package/test_openjdk.py: bump kernel version to 5.10.34
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.

Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-05 21:37:21 +02:00
Dick Olsson
c5497df7b3 support/testing: add s6-networking tests
Test that the TAICLOCK and TCP servers are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:58:51 +02:00
Dick Olsson
50c6e932dc support/testing: add s6-rc tests
Test that s6-rc service database compilation is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:56:49 +02:00
Dick Olsson
77c13ae989 support/testing: add s6-portable-utils tests
Test that a few basis utilities are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:53:52 +02:00
Dick Olsson
985d733f31 support/testing: add s6 tests
Test that directory scanning and supervision is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: properly indent, and use textwrap to dedent again.]
2021-05-04 21:50:40 +02:00
Dick Olsson
f7ea0af883 support/testing: add execline tests
Test that the interpreter can run a basic command.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:34:36 +02:00
Thomas De Schampheleire
4d6c6b1bd4 support/download/hg: fix broken method
Commit 54d3d94b6e broke the 'hg' download
method, in a similar way as it broke the 'git' download method (later fixed
with commit b70ce56651), by introducing extra
output on stdout in a case where the output is redirected.

In the case of 'hg', the 'hg archive' step uses shell redirection rather
than directly letting hg write the output file, since commit
76b51f90c0.

As a result, the extra print added by the _hg function is prepended to the
actual archive, causing an invalid archive.

Fix by using the _plain_hg function instead. The disadvantage is that the
command for 'hg archive' is no longer printed.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-28 21:51:10 +02:00
Fabrice Fontaine
3ab8aefa87 support/dependencies: set cmake version min to 3.16
domoticz requires cmake 3.16 since version 2020.2 and
275effddf0

Fixes:
 - http://autobuild.buildroot.org/results/0caec85c70341036a039dbc337ad99196b6005a9

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-24 23:10:17 +02:00
Francois Perrad
b3dd1034d5 package/perl-libwww-perl: bump to version 6.53
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:44 +02:00
Francois Perrad
9962e3020c package/perl-io-socket-ssl: bump to version 2.070
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:43 +02:00
Thomas Petazzoni
d06bf96097 support/scripts/cve.py: use proper CPE ID version when available
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-18 18:20:27 +02:00
Peter Korsgaard
f31227e628 support/scripts/cve.py: use fast ijson backend if available on old ijson versions
ijson < 2.5 (as available in Debian 10) use the slow python backend by
default instead of the most efficient one available like modern ijson
versions, significantly slowing down cve checking. E.G.:

time ./support/scripts/pkg-stats --nvd-path ~/.nvd -p avahi --html foobar.html

Goes from
174,44s user 2,11s system 99% cpu 2:58,04 total

To
93,53s user 2,00s system 98% cpu 1:36,65 total

E.G. almost 2x as fast.

As a workaround, detect when the python backend is used and try to use a
more efficient one instead.  Use the yajl2_cffi backend as recommended by
upstream, as it is most likely to work, and print a warning (and continue)
if we fail to load it.

The detection is slightly complicated by the fact that ijson.backends used
to be a reference to a backend module, but is nowadays a string (without the
ijson.backends prefix).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-17 09:14:40 +02:00
José Luis Salvador Rufo
4470bc9914 package/zfs: new package
OpenZFS is an advanced file system and volume manager which was originally
developed for Solaris and is now maintained by the OpenZFS community. This
repository contains the code for running OpenZFS on Linux and FreeBSD.

http://zfsonlinux.org/

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
[me:
  - fix test case on how to use a pre-built toolchain
  - reorder the test case config
  - add test case with glibc
  - drop superflous test timeout override
  - only select libtirpc when C library lacks native RPC
  - drop unused ZFS_MODULES variable
  - drop ZFS_CPE_ID_PREFIX and ZFS_AUTORECONF_OPTS which are defaults
  - drop NLS options, already set in a generic manner
  - drop incomplete/improper sysvinit support
  - some cosmetics
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-13 23:08:17 +02:00
Peter Korsgaard
a74cb089cb Update for 2021.02.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bb10b0dfe6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-04-07 13:34:27 +02:00
Fabrice Fontaine
2c2c0192a2 package/pkg-cmake.mk: don't unconditionally set CMAKE_CXX_COMPILER
Don't unconditionally set CMAKE_CXX_COMPILER as it will raise a build
failure on qpid-proton because "if the toolchain specifies a value for
CMAKE_CXX_COMPILER, then CMake assumes the compiler works and goes
straight ahead trying to use it":
https://cmake.org/cmake/help/latest/module/CheckLanguage.html
https://issues.apache.org/jira/browse/PROTON-2365

Fixes:
 - http://autobuild.buildroot.org/results/05f344151100219c159ca4d466a453df96bf07fa

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[yann.morin.1998@free.fr: rename placeholder]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-02 23:50:16 +02:00
Bert Outtier
6fdc878d73 support/scripts: fix pycompile for short filenames
Signed-off-by: Bert Outtier <outtierbert@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-30 22:18:44 +02:00
Jean-pierre Cartal
c9f27fdc1b support/download: fix tarball generation with symlinks pointing to ./something
When a --transform expression is provided, it is by default also applied
to the target of a symlink.

When we create tarballs (from git or svn checkouts), we use a --transform
expression to replace the leading ./ with the package name and version.

This causes issues when a package contains symlinks that points to
./something, as the leading './' is also replaced.

Fix that by using the 'S' transformation scope flag, as described in the
tar manual:
    https://www.gnu.org/software/tar/manual/html_node/transform.html#transform

    In addition, several transformation scope flags are supported, that
    control to what files transformations apply. These are:

    ‘r’ Apply transformation to regular archive members.
    ‘R’ Do not apply transformation to regular archive members.
    ‘s’ Apply transformation to symbolic link targets.
    ‘S’ Do not apply transformation to symbolic link targets.
    ‘h’ Apply transformation to hard link targets.
    ‘H’ Do not apply transformation to hard link targets.

    Default is ‘rsh’ [...].

Fixes: #13616

This has been checked to not change any of the existing hash for any of
our git-downloaded package (some are host-only, hence the few fixups):

    ---8<---
    $ m="$( git grep -l -E -- -br[[:digit:]]+.tar.gz boot package/ \
            |awk -F/ '{print $(NF-1)}' \
            |sed -r -e 's/(imx-mkimage|netsurf-buildsystem|prelink-cross|qoriq-rcw|vboot-utils)/host-\1/g' \
                    -e 's/$/-source/'
          )"
    $ make defconfig; make clean; BR2_DL_DIR=$(pwd)/trash-me make ${m}
    ---8<---

Note: it is unclear what the 'H' flag does nor how it works, because the
concept of "target of a hardlink" is not obvious; probably it has to do
with how tar internally detects and stores hardlinks. Since we do not
yet have any issue with hardlinks, just ignore the problem for now, and
postpone until we have an actual issue with a real test-case.

Signed-off-by: Jean-pierre Cartal <jpcartal@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
  - re-indent commit log
  - add scriptlet to test existing hashes
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-20 17:29:29 +01:00
Thomas De Schampheleire
b70ce56651 support/download: fix git/svn corrupted cache
Commit 54d3d94b6e ("support/download: print
command used for download") broke the git and svn download helpers, because
these helpers have invocations of the _git/_svn commands where the exact
output matters.

For example for git, this would result in:

    date: invalid date ‘GIT_DIR=.../dl/libyuv/git/.git git log -1 --pretty=format:%ci \n2019-04-12 17:48:45 +0000’
    Detected a corrupted git cache.
    Removing it and starting afresh.

Fix by splitting the _git function in two: _git and _plain_git.
The former echoes the command, and then calls the latter.
Most invocations use _git as before, but those cases where the output should
not be disturbed, directly call _plain_git.

For symmetry, all download helpers are aligned, even though only the git and
svn helpers were broken.

Fixes: #13631
Fixes:
    http://autobuild.buildroot.org/results/c2f/c2fcd4aa6660e3c2f9c6f85646ca7dfe0db56040/

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: add bug report and autobuild failure]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-19 21:22:59 +01:00
Thomas De Schampheleire
54d3d94b6e support/download: print command used for download
Even though that most download commands actually print some output, like
progress indication or other messages, the actual command used is not. This
makes it hard to analyze a build log when you are not fully familiar with
the typical output of said log.

Update the download helpers to do just that, respecting any quiet/verbose
flag so that a silent make (make -s) does not get more verbose.

Note: getting rid of the duplication of the command in the script is not
straightforward without breaking support for arguments with spaces.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: use printf, not echo]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-16 23:26:12 +01:00
Thomas De Schampheleire
3300788ce0 support/download: rename internal 'verbose' variable where applicable
Most 'verbose' variable inside the download helpers actually mean 'quiet'.
I.e. they are assigned in case quiet operation is requested, and empty in
case of non-quiet operation. Using the name 'verbose' for such a variable is
confusing, especially when you want to test the variable on emptiness or
non-emptiness (in a subsequent commit).

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-16 23:26:12 +01:00
Yann E. MORIN
e36974d9e8 support/dependencies: detect and bailout when PATH contains spaces/TABs
In Makefiles, variables are split, filtered, and otherwise mangled on
a space as a separator. In a shell, they will also be split on TABs.

We split and filter and iterate on variables in a lot of places, and
most importantly, spaces in PATH is very seldom tested, if at all, so
a lot of packages will not be working properly in such a situation.

For example, the config.guess contains constructs that are not resilient
to a space in PATH:

    PATH=$PATH:/.attbin ; export PATH

Also, our fakedate will iterate over PATH:

    for P in `echo $PATH | tr ':' ' '`; do

Those are only two cases, but the first means basically all
autotools-based packages are susceptible to subtle breakage.

Furthermore, Buildroot itself does not support that the top-level or
output directories are in a path with spaces anyway.

So, instead of chasing all cases that might be potentially broken,
let's just detect the case and bail out, like we already do when PATH
contains a \n, or when it contains the current working directory.

Reported-by: Dan Raymond <draymond@foxvalley.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-03-14 23:29:15 +01:00
Yann E. MORIN
c92be85e3a support/download: make the svn backend more reproducible
Since c043ecb20c (support/download: change format of archives
generated from svn), the svn backend uses the generic helper to
create reproducible archives.

That helper really does its job as expected, but the svn backend
is flawed in two ways:

  - the first, most obvious breakage happens with versions older
    than 1.9, as they do not support the '--show-item' option
    for the 'info' action;

  - the second is more involved, in that svn will by default
    expand the old, legacy, deprecated, cumbersome CVS-style
    keywords, in the form of revision marks like '$Date$' in a
    C-style comment in a source file. These replacements are
    done on checkout as well as on export, and they use local
    settings, like the local locale and timezone.

    This means that two people with different settings, will get
    different sources when the svn-checkout or svn-export the same
    revision from the same tree...

    Needless to say that this is not very reproducible...

While the first is easily solved, the second is more involved.

We need to ensure that what source is used initially to compute
the hash, will also be the source that are used to check the hash.

There are basically two solutions:

 1. we ensure the same environment, by forcing the timezone and
    the locale to arbitrary values

 2. we disable keyword expansion

For the first solution, this still leaves the possibility that we
miss some environment settings that have an impact on the keyword
expansion. It would mean that Yann's settings be used, as he did
introduce the hash for the only svn-downloaded package we have,
avrdude, settings which are:
    TZ=Europe/Paris
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_GB.UTF-8"
    LC_MONETARY="fr_FR.utf8"
    LC_NUMERIC="fr_FR.utf8"

The second option means that the generated archives change. That
means we'd have to bump the archive version for svn downloads, and
that we update the hashes for all the svn-downloaded packages.

We chose to go with the second option, because this is what really
makes more sense, rather than hard-coding arbitrary values in the
environment. And we also have only one svn-downloaded package,
avrdude.

And thus, we're reaching the trigger for this change: avrdude is
impacted by the CVS-keyword expansion issue:

    https://svn.savannah.gnu.org/viewvc/avrdude/trunk/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js?revision=1396&view=markup

which would give two different files when checked out on different
machines:

    diff -durN foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js
    --- foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
    +++ bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
    @@ -1,6 +1,6 @@
     /**
      * @preserve jquery.layout 1.3.0 - Release Candidate 30.51
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      * $Rev: 303005 $
      *
      * Copyright (c) 2012
    @@ -4718,7 +4718,7 @@

     /**
      * jquery.layout.state 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2010
      *   Kevin Dalman (http://allpro.net)
    @@ -5074,7 +5074,7 @@

     /**
      * jquery.layout.buttons 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2010
      *   Kevin Dalman (http://allpro.net)
    @@ -5356,7 +5356,7 @@

     /**
      * jquery.layout.browserZoom 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2012
      *   Kevin Dalman (http://allpro.net)

So we also update the hash for avrdude.

Fixes:
    http://autobuild.buildroot.org/results/e3b/e3b0508047f32008ebfa83c5255ec5994b6af120/ (time issue)
    http://autobuild.buildroot.org/results/48e/48e78e84b425e79cdb98c16ab40247a0fa7e9676/ (keyword expansion issue)

Reported-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Reviewed-by: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-11 21:54:00 +01:00
Peter Korsgaard
e2fba6457b Update for 2021.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-03-06 22:16:45 +01:00
Yann E. MORIN
b9db38d370 package: drop _CPE_ID_VALID, use _CPE_ID_VENDOR
FOO_CPE_ID_VALID really ought to be an internal implementaion detail.
Packages that really want to trigger their CPE defintitions really
should set one of the actual variables to a meaningful value.

There are two CPE-related variables that we could chose to set to
replace FOO_CPE_ID_VALID: FOO_CPE_ID_VENDOR and FOO_CPE_ID_PRODUCT.
Between those two, _VENDOR more often diverges from the default than
_PRODUCT does, so that's what we use.

    ---8<------8<------8<------8<------8<---
    #!/bin/bash
    # Replace FOO_CPE_ID_VALID = YES with FOO_CPE_ID_VENDOR = foo_project
    for i in $(git grep -l -E '[^)]_CPE_ID_VALID = YES' package support); do
        pkg="$(basename "${i%/*}")"
        sed -r -i -e "s/_CPE_ID_VALID = YES/_CPE_ID_VENDOR = ${pkg}_project/" "${i}"
    done
    ---8<------8<------8<------8<------8<---

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: update cpe-test comment to reflect pkg3 change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-03-06 17:18:07 +01:00
Peter Korsgaard
a0aff89ed2 support/scripts/gen-bootlin-toolchains: correct xtensa-lx60 toolchain dependencies
Fixes:
http://autobuild.buildroot.net/results/011/0111c2ed54618daaeedfc66b0ea04eda00a7e855/
http://autobuild.buildroot.net/results/e53/e53e3880b63a23fa3b3e6d34664d40d5ddbdff89/
..

As listed in the br_fragment file of the toolchain, this is built for a
little-endian "custom" xtensa variant rather than the (big-endian) fsf one:

BR2_xtensa=y
BR2_XTENSA_CUSTOM=y

So update the dependencies in the script and regenerate Config.in.options /
toolchain test.  Also fixup the autobuild config snippet to match.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-03-02 23:52:49 +01:00
Vincent Fazio
b50eef1dfd support/download: drop sub-second precision in tarball creation
Some download backends, like svn, will provide timestamps with a
sub-second precision, e.g.

    $ svn info --show-item last-changed-date [...]
    2021-02-19T20:22:34.889717Z

However, the PAX headers do not accept sub-second precision, leading to
failure to download from subversion:

    tar: Time stamp is out of allowed range
    tar: Exiting with failure status due to previous errors
    make[1]: *** [package/pkg-generic.mk:148: [...]/build/subversion-1886712/.stamp_downloaded] Error 1

Fix that by massaging the timestamp to drop the sub-second part. We
do that in the generic helper, rather than the svn backend, so that
all callers to the generic helper benefit from this, as this is more
an internal details of the tarball limitations, than of the backends
themselves.

Reported-by: Roosen Henri <Henri.Roosen@ginzinger.com>
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
  - add Henri as reporter
  - move it out of the svn backend, and to the generic helper
  - reword the commit log accordingly
  - use an explicit time format rather than -Iseconds
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-22 23:01:27 +01:00
Romain Naour
03c3fbd81c support/scripts/boot-qemu-image.py: properly catch timeout
As reported on IRC by sephthir, the gitlab test of the defconfig
qemu_sparc_ss10_defconfig doesn't error out while the system
is not working properly.

This is because we explicitly wait for the timeout as an expected
condition, but do not check for it. Indeed, pexpect.expect() returns
the index of the matching condition in the list of expected conditions,
but we just ignore the return code, so we are not able to differentiate
between a successful login (or prompt) from a timeout.

By default, pexepect.expect() raises the pexpect.TIMEOUT exception on a
timeout, and we are already prepared to catch and handle that exception.
But because pexpect.TIMEOUT is passed as an expected condition, the
exception is not raised.

Remove pexpect.TIMEOUT from the list of expected conditions, so that the
exception is properly raised again, and so that we can catch it.

The qemu_sparc_ss10_defconfig is already fixed by
4d16e6f532.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
[yann.morin.1998@free.fr: reword commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-20 22:44:29 +01:00
Peter Korsgaard
ecbfbabcf7 Update for 2020.11.3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 610e67b1fc)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-02-17 19:43:17 +01:00
Heiko Thiery
5b844d50d8 support/scripts/pkg-stats: add ignored_cves to json output
Add the list of <pkg>_IGNORE_CVES to the json output to show that we have a
known cause (available patch or the CVE is not valid for our package
configuration) that a affected CVE is not reported.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-02-16 23:15:06 +01:00
Arnout Vandecappelle (Essensium/Mind)
bcef4a03e7 support/scripts/cpedb.py: remove import pickle
pickle is no longer used since 09a71e6a75

Fixes:
support/scripts/cpedb.py:7:1: F401 'pickle' imported but unused

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-02-16 21:16:55 +01:00
Thomas Petazzoni
09a71e6a75 support/scripts/cpedb.py: drop CPE XML database caching
Currently, the CPE XML database is parsed into a Python dict, which is
then pickled into a local file, to speed up the processing of further
invocations.

However, it turns out that since the initial implementation, we have
switched the XML parsing from the out of tree xmltodict module to the
standard ElementTree one, which has made the parsing much faster. The
pickle caching only saves 6 seconds, on something that takes more than
13 minutes total.

In addition, this pickle caching consumes a significant amount of RAM,
causing the Python process to be OOM-killed on a server with 4 GB of
RAM.

So let's just drop this caching entirely.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-14 10:05:00 +01:00
Sergey Matyukevich
653b127d1e support/testing: TestATFAllwinner needs python3 for u-boot
New U-Boot version needs Python 3.x for pylibfdt.

Fixes:
- https://gitlab.com/buildroot.org/buildroot/-/jobs/1019385909

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-13 22:46:27 +01:00
Thomas Petazzoni
e1c0360606 support/config-fragments/autobuild: use Bootlin toolchain for RISC-V 64 musl
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/musl, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-05 09:27:37 +01:00
Thomas Petazzoni
63485a6c1e support/config-fragments/autobuild: use Bootlin toolchain for RISC-V 64 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/glibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-05 09:25:23 +01:00
Thomas Petazzoni
032cf04fe7 support/config-fragments/autobuild: update Buildroot toolchains to 2020.11.2
Most of the toolchains now use gcc 9.x and kernel headers 5.9, instead
of gcc 8.x and kernel headers 5.4.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:47:02 +01:00
Thomas Petazzoni
dcb2747d65 support/config-fragments/autobuild: use Bootlin toolchain for Xtensa uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test Xtensa/uclibc, use a pre-built Bootlin toolchain.

To be noted: that fragment was in fact already using a Bootlin
bleeding-edge toolchain, because BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is
missing from the fragment:

    $ cat support/config-fragments/autobuild/br-xtensa-full.config >.config

    $ make olddefconfig

    $ grep BOOTLIN .config
    BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
    BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_ARCH_SUPPORTS=y
    BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_XTENSA_LX60_UCLIBC_BLEEDING_EDGE=y
    # BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_XTENSA_LX60_UCLIBC_STABLE is not set

The original fragment was supposed to use a stable toolchain, so we
switch to explictly use a stable Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - add blurb about missing BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:42:51 +01:00
Thomas Petazzoni
6bed28e996 support/config-fragments/autobuild: use Bootlin toolchain for x86-64 musl
Instead of using an external toolchain built specifically for the
autobuilders to test x86-64/musl, use a pre-built Bootlin toolchain.

The previous configuration was for an Atom platform, but the Bootlin
toolchains only provide a Core i7 configuration. Since this is close
enough, we change to use this Core i7 configuration.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:42:39 +01:00
Thomas Petazzoni
5679e58c9f support/config-fragments/autobuild: use Bootlin toolchain for x86-64 uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test x86-64/uclibc, use a pre-built Bootlin toolchain.

The previous configuration was for Core2 platform, but the Bootlin
toolchains only provide a Core i7 configuration. Since this is close
enough, we change to use this Core i7 configuration.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:42:39 +01:00
Thomas Petazzoni
9e56f77fdf support/config-fragments/autobuild: use Bootlin toolchain for SPARC64 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test SPARC64/glibc, use a pre-built Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:42:39 +01:00
Thomas Petazzoni
1348c569d0 support/config-fragments/autobuild: use Bootlin toolchain for SPARC uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test SPARC/uclibc, use a pre-built Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:42:39 +01:00
Thomas Petazzoni
e022d47908 support/config-fragments/autobuild: use Bootlin toolchain for SH4 uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test SH4/uclibc, use a pre-built Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: drop BR2_sh4=y which is the default]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:41:52 +01:00
Thomas Petazzoni
fc266b1c1a support/config-fragments/autobuild: use Bootlin toolchain for RISC-V 32 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 32/glibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:09 +01:00
Thomas Petazzoni
563285a63a support/config-fragments/autobuild: use Bootlin toolchain for PowerPC e500mc uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test PowerPC e500mc/uclibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:08 +01:00
Thomas Petazzoni
15af077d8e support/config-fragments/autobuild: use Bootlin toolchain for PowerPC64le Power8 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test PowerPC64le Power8/glibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:07 +01:00
Thomas Petazzoni
739cb9c9d5 support/config-fragments/autobuild: use Bootlin toolchain for OpenRISC uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test OpenRISC/uclibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:06 +01:00
Thomas Petazzoni
b18e923174 support/config-fragments/autobuild: use Bootlin toolchain for NIOS2 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test nios2/glibc, use a pre-built Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:05 +01:00
Thomas Petazzoni
625cbe23b3 support/config-fragments/autobuild: use Bootlin toolchain for mipsel uclibc
Instead of using an external toolchain built specifically for the
autobuilders to test mipsel/uclibc, use a pre-built Bootlin toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-03 22:37:03 +01:00