Commit Graph

1261 Commits

Author SHA1 Message Date
Matthew Weber
50791af71f support/scripts/pkg-stats: clarify when a CVE/CPE should report as N/A
- If a package doesn't have any versioning, ignore and state that
 - If a package is virtual, CVE=ignore and CPE state virtual
 - For any of these NA cases, don't provide search link and color box
   green

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 10:21:28 +02:00
Matthew Weber
9368f3f23d support/scripts/pkg-stats: add is_actual_package() and rework has_valid_infra()
has_valid_infra() is incorrectly named; it probably should be named
is_actual_package(), and has_valid_infra() would be changed to
actually represent having an actual infra.

This resolves packages reporting as having no valid package infra and
cleans up reporting cases of CPE and CVEs where there isn't a valid version
or package definition outside Buildroot

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 10:14:31 +02:00
Matthew Weber
9b83bb1382 support/scripts/pkg-stats: verified CPE has a known id but not version
Currently a verified CPE reports the following if versions are not found
 cpe:2.3🅰️qemu:qemu:5.2.0:*:*:*:*:*:*:*
 CPE identifier unknown in CPE database (Search)

This patch clarifies the report to state the 'version' is unknown instead
of the 'identifier'.

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-19 09:26:50 +02:00
Romain Naour
45a13be315 support/testing: test_hardening disable PIC/PIE
Since [1], PIC/PIE is enabled by default but the TestRelroPartial
test expect implicitely PIC/PIE being disabled.

Disable PIC/PIE from the config fragment provided by
TestRelroPartial.

[1] 810ba387be

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661757

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-05-18 22:15:36 +02:00
Matthew Weber
c2ebfe7d78 support/scripts/pkg-stats: fix flake8 E741 ambiguous variable name
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 21:06:12 +02:00
Romain Naour
4a3639bad0 support/testing: test_glxinfo load X11 modules in the right order
From [1]
"Xorg does not implement real dynamic linking and requires that its
modules get loaded in the right order."

From /var/log/Xorg.0.0.log:
 (II) LoadModule: "modesetting"
 (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
 (EE) Failed to load /usr/lib/xorg/modules/drivers/modesetting_drv.so: /usr/lib/xorg/modules/drivers/modesetting_drv.so: undefined symbol: shadowRemove

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661899

[1] https://forums.gentoo.org/viewtopic-p-8245578.html#8245578

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:49 +02:00
Romain Naour
95b722eaa8 support/testing: test_glxinfo: switch to Gallium swrast
Since the mesa3d bump to version 21.0.3 [1], the
BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST option is not supported anymore
since the mesa DRI swrast driver has been removed upstream

So, switch to Gallium swrast.

[1]15a2f9b819806d38a7d8172a20f80130b1d60e63

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-18 15:25:48 +02:00
Matt Weber
f3b07e2ded support/scripts/pkg-stats: add column reporting ignored CVEs
When doing analysis it is helpful to be able to view what CVE have
been patched / diagnosed to not apply to Buildroot. This exposes
that list to the reporting and prevents a step where you have to
dig into the .mk's of a pkg to check for sure what has been
ignored.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: only set background if there are ignored CVEs]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 18:22:58 +02:00
Matt Weber
c5aa3c5883 support/scripts/pkg-stats: add CPE searching links
For cases of a CPE having a unknown version or when there hasn't
been a CPE verified, proposed a search criteria to help the
user research an update.

(libcurl has NIST dict entries but not this version)
  cpe:2.3🅰️haxx:libcurl:7.76.1:*:*:*:*:*:*:*
  CPE identifier unknown in CPE database (Search)

(jitterentropy-library package doesn't have any NIST dict entries)
  no verified CPE identifier (Search)

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: fix flake8 issues]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 17:54:25 +02:00
Matt Weber
fffc553485 support/scripts/gen-missing-cpe: add new script
This script queries the list of CPE IDs for the packages of the
current configuration (based on the "make show-info" output), and:

 - for CPE IDs that do not have any matching entry in the CPE
   database, it emits a warning

 - for CPE IDs that do have a matching entry, but not with the same
   version, it generates a snippet of XML that can be used to propose
   an updated version to NIST.

Ref: NIST has a group email (cpe_dictionary@nist.gov) used to
recieve these version update and new entry xml files.  They do
process the XML and provide feedback. In some cases they will
propose back something different where the vendor or version is
slightly different.

Limitations
 - Currently any use of non-number version identifiers isn't
   supported by NIST as they use ranges to determine impact
   of a CVE
 - Any Linux version from a non-upstream is also not supported
   without manually adjusting the information as the custom
   kernel will more then likely not match the upstream version
   used in the dictionary

Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr:
  - codestyles as spotted by Arnout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-16 13:57:38 +02:00
Peter Korsgaard
aa31d10808 test_docker_compose.py: Test the volume mount feature
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 23:00:16 +02:00
Peter Korsgaard
4915b692c8 test_docker_compose.py: Test the port publish feature
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 22:59:58 +02:00
Romain Naour
f5ea09e2a8 support/testing: remove TestPythonPy2Colorzero
The python2 support has been removed since the python-colorzero bump version to 2.0.

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:33 +02:00
Romain Naour
9217708122 support/testing: remove TestPythonPy2Gpiozero
The python2 support has been removed since the python-colorzero bump version to 2.0.

Remove the gpiozero test with python2

[1] 73bf3292e1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-14 19:24:24 +02:00
Peter Korsgaard
fdb3291b8c Update for 2021.02.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 76b4f9e9b6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-12 11:37:23 +02:00
Arnout Vandecappelle (Essensium/Mind)
74bae64dc5 support/testing: add sudo package test
Create a new user 'sudotest' to validate that sudo really works (i.e.
properly has setuid).

Creating the user and adding it to sudoers is done at runtime, otherwise
we'd need to add extra files to the config which complicates things a
little bit.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-05-10 22:08:15 +02:00
Adam Duskett
ab2d472cde testing/tests/package/test_openjdk.py: bump kernel version to 5.10.34
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.

Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-05-05 21:37:21 +02:00
Dick Olsson
c5497df7b3 support/testing: add s6-networking tests
Test that the TAICLOCK and TCP servers are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:58:51 +02:00
Dick Olsson
50c6e932dc support/testing: add s6-rc tests
Test that s6-rc service database compilation is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:56:49 +02:00
Dick Olsson
77c13ae989 support/testing: add s6-portable-utils tests
Test that a few basis utilities are working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:53:52 +02:00
Dick Olsson
985d733f31 support/testing: add s6 tests
Test that directory scanning and supervision is working.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: properly indent, and use textwrap to dedent again.]
2021-05-04 21:50:40 +02:00
Dick Olsson
f7ea0af883 support/testing: add execline tests
Test that the interpreter can run a basic command.

Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
2021-05-04 21:34:36 +02:00
Thomas De Schampheleire
4d6c6b1bd4 support/download/hg: fix broken method
Commit 54d3d94b6e broke the 'hg' download
method, in a similar way as it broke the 'git' download method (later fixed
with commit b70ce56651), by introducing extra
output on stdout in a case where the output is redirected.

In the case of 'hg', the 'hg archive' step uses shell redirection rather
than directly letting hg write the output file, since commit
76b51f90c0.

As a result, the extra print added by the _hg function is prepended to the
actual archive, causing an invalid archive.

Fix by using the _plain_hg function instead. The disadvantage is that the
command for 'hg archive' is no longer printed.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-28 21:51:10 +02:00
Fabrice Fontaine
3ab8aefa87 support/dependencies: set cmake version min to 3.16
domoticz requires cmake 3.16 since version 2020.2 and
275effddf0

Fixes:
 - http://autobuild.buildroot.org/results/0caec85c70341036a039dbc337ad99196b6005a9

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-24 23:10:17 +02:00
Francois Perrad
b3dd1034d5 package/perl-libwww-perl: bump to version 6.53
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:44 +02:00
Francois Perrad
9962e3020c package/perl-io-socket-ssl: bump to version 2.070
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-24 22:46:43 +02:00
Thomas Petazzoni
d06bf96097 support/scripts/cve.py: use proper CPE ID version when available
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-04-18 18:20:27 +02:00
Peter Korsgaard
f31227e628 support/scripts/cve.py: use fast ijson backend if available on old ijson versions
ijson < 2.5 (as available in Debian 10) use the slow python backend by
default instead of the most efficient one available like modern ijson
versions, significantly slowing down cve checking. E.G.:

time ./support/scripts/pkg-stats --nvd-path ~/.nvd -p avahi --html foobar.html

Goes from
174,44s user 2,11s system 99% cpu 2:58,04 total

To
93,53s user 2,00s system 98% cpu 1:36,65 total

E.G. almost 2x as fast.

As a workaround, detect when the python backend is used and try to use a
more efficient one instead.  Use the yajl2_cffi backend as recommended by
upstream, as it is most likely to work, and print a warning (and continue)
if we fail to load it.

The detection is slightly complicated by the fact that ijson.backends used
to be a reference to a backend module, but is nowadays a string (without the
ijson.backends prefix).

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-17 09:14:40 +02:00
José Luis Salvador Rufo
4470bc9914 package/zfs: new package
OpenZFS is an advanced file system and volume manager which was originally
developed for Solaris and is now maintained by the OpenZFS community. This
repository contains the code for running OpenZFS on Linux and FreeBSD.

http://zfsonlinux.org/

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
[me:
  - fix test case on how to use a pre-built toolchain
  - reorder the test case config
  - add test case with glibc
  - drop superflous test timeout override
  - only select libtirpc when C library lacks native RPC
  - drop unused ZFS_MODULES variable
  - drop ZFS_CPE_ID_PREFIX and ZFS_AUTORECONF_OPTS which are defaults
  - drop NLS options, already set in a generic manner
  - drop incomplete/improper sysvinit support
  - some cosmetics
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-13 23:08:17 +02:00
Peter Korsgaard
a74cb089cb Update for 2021.02.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bb10b0dfe6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-04-07 13:34:27 +02:00
Fabrice Fontaine
2c2c0192a2 package/pkg-cmake.mk: don't unconditionally set CMAKE_CXX_COMPILER
Don't unconditionally set CMAKE_CXX_COMPILER as it will raise a build
failure on qpid-proton because "if the toolchain specifies a value for
CMAKE_CXX_COMPILER, then CMake assumes the compiler works and goes
straight ahead trying to use it":
https://cmake.org/cmake/help/latest/module/CheckLanguage.html
https://issues.apache.org/jira/browse/PROTON-2365

Fixes:
 - http://autobuild.buildroot.org/results/05f344151100219c159ca4d466a453df96bf07fa

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[yann.morin.1998@free.fr: rename placeholder]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-04-02 23:50:16 +02:00
Bert Outtier
6fdc878d73 support/scripts: fix pycompile for short filenames
Signed-off-by: Bert Outtier <outtierbert@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-30 22:18:44 +02:00
Jean-pierre Cartal
c9f27fdc1b support/download: fix tarball generation with symlinks pointing to ./something
When a --transform expression is provided, it is by default also applied
to the target of a symlink.

When we create tarballs (from git or svn checkouts), we use a --transform
expression to replace the leading ./ with the package name and version.

This causes issues when a package contains symlinks that points to
./something, as the leading './' is also replaced.

Fix that by using the 'S' transformation scope flag, as described in the
tar manual:
    https://www.gnu.org/software/tar/manual/html_node/transform.html#transform

    In addition, several transformation scope flags are supported, that
    control to what files transformations apply. These are:

    ‘r’ Apply transformation to regular archive members.
    ‘R’ Do not apply transformation to regular archive members.
    ‘s’ Apply transformation to symbolic link targets.
    ‘S’ Do not apply transformation to symbolic link targets.
    ‘h’ Apply transformation to hard link targets.
    ‘H’ Do not apply transformation to hard link targets.

    Default is ‘rsh’ [...].

Fixes: #13616

This has been checked to not change any of the existing hash for any of
our git-downloaded package (some are host-only, hence the few fixups):

    ---8<---
    $ m="$( git grep -l -E -- -br[[:digit:]]+.tar.gz boot package/ \
            |awk -F/ '{print $(NF-1)}' \
            |sed -r -e 's/(imx-mkimage|netsurf-buildsystem|prelink-cross|qoriq-rcw|vboot-utils)/host-\1/g' \
                    -e 's/$/-source/'
          )"
    $ make defconfig; make clean; BR2_DL_DIR=$(pwd)/trash-me make ${m}
    ---8<---

Note: it is unclear what the 'H' flag does nor how it works, because the
concept of "target of a hardlink" is not obvious; probably it has to do
with how tar internally detects and stores hardlinks. Since we do not
yet have any issue with hardlinks, just ignore the problem for now, and
postpone until we have an actual issue with a real test-case.

Signed-off-by: Jean-pierre Cartal <jpcartal@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
  - re-indent commit log
  - add scriptlet to test existing hashes
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-20 17:29:29 +01:00
Thomas De Schampheleire
b70ce56651 support/download: fix git/svn corrupted cache
Commit 54d3d94b6e ("support/download: print
command used for download") broke the git and svn download helpers, because
these helpers have invocations of the _git/_svn commands where the exact
output matters.

For example for git, this would result in:

    date: invalid date ‘GIT_DIR=.../dl/libyuv/git/.git git log -1 --pretty=format:%ci \n2019-04-12 17:48:45 +0000’
    Detected a corrupted git cache.
    Removing it and starting afresh.

Fix by splitting the _git function in two: _git and _plain_git.
The former echoes the command, and then calls the latter.
Most invocations use _git as before, but those cases where the output should
not be disturbed, directly call _plain_git.

For symmetry, all download helpers are aligned, even though only the git and
svn helpers were broken.

Fixes: #13631
Fixes:
    http://autobuild.buildroot.org/results/c2f/c2fcd4aa6660e3c2f9c6f85646ca7dfe0db56040/

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: add bug report and autobuild failure]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-19 21:22:59 +01:00
Thomas De Schampheleire
54d3d94b6e support/download: print command used for download
Even though that most download commands actually print some output, like
progress indication or other messages, the actual command used is not. This
makes it hard to analyze a build log when you are not fully familiar with
the typical output of said log.

Update the download helpers to do just that, respecting any quiet/verbose
flag so that a silent make (make -s) does not get more verbose.

Note: getting rid of the duplication of the command in the script is not
straightforward without breaking support for arguments with spaces.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[yann.morin.1998@free.fr: use printf, not echo]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-16 23:26:12 +01:00
Thomas De Schampheleire
3300788ce0 support/download: rename internal 'verbose' variable where applicable
Most 'verbose' variable inside the download helpers actually mean 'quiet'.
I.e. they are assigned in case quiet operation is requested, and empty in
case of non-quiet operation. Using the name 'verbose' for such a variable is
confusing, especially when you want to test the variable on emptiness or
non-emptiness (in a subsequent commit).

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-16 23:26:12 +01:00
Yann E. MORIN
e36974d9e8 support/dependencies: detect and bailout when PATH contains spaces/TABs
In Makefiles, variables are split, filtered, and otherwise mangled on
a space as a separator. In a shell, they will also be split on TABs.

We split and filter and iterate on variables in a lot of places, and
most importantly, spaces in PATH is very seldom tested, if at all, so
a lot of packages will not be working properly in such a situation.

For example, the config.guess contains constructs that are not resilient
to a space in PATH:

    PATH=$PATH:/.attbin ; export PATH

Also, our fakedate will iterate over PATH:

    for P in `echo $PATH | tr ':' ' '`; do

Those are only two cases, but the first means basically all
autotools-based packages are susceptible to subtle breakage.

Furthermore, Buildroot itself does not support that the top-level or
output directories are in a path with spaces anyway.

So, instead of chasing all cases that might be potentially broken,
let's just detect the case and bail out, like we already do when PATH
contains a \n, or when it contains the current working directory.

Reported-by: Dan Raymond <draymond@foxvalley.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-03-14 23:29:15 +01:00
Yann E. MORIN
c92be85e3a support/download: make the svn backend more reproducible
Since c043ecb20c (support/download: change format of archives
generated from svn), the svn backend uses the generic helper to
create reproducible archives.

That helper really does its job as expected, but the svn backend
is flawed in two ways:

  - the first, most obvious breakage happens with versions older
    than 1.9, as they do not support the '--show-item' option
    for the 'info' action;

  - the second is more involved, in that svn will by default
    expand the old, legacy, deprecated, cumbersome CVS-style
    keywords, in the form of revision marks like '$Date$' in a
    C-style comment in a source file. These replacements are
    done on checkout as well as on export, and they use local
    settings, like the local locale and timezone.

    This means that two people with different settings, will get
    different sources when the svn-checkout or svn-export the same
    revision from the same tree...

    Needless to say that this is not very reproducible...

While the first is easily solved, the second is more involved.

We need to ensure that what source is used initially to compute
the hash, will also be the source that are used to check the hash.

There are basically two solutions:

 1. we ensure the same environment, by forcing the timezone and
    the locale to arbitrary values

 2. we disable keyword expansion

For the first solution, this still leaves the possibility that we
miss some environment settings that have an impact on the keyword
expansion. It would mean that Yann's settings be used, as he did
introduce the hash for the only svn-downloaded package we have,
avrdude, settings which are:
    TZ=Europe/Paris
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_GB.UTF-8"
    LC_MONETARY="fr_FR.utf8"
    LC_NUMERIC="fr_FR.utf8"

The second option means that the generated archives change. That
means we'd have to bump the archive version for svn downloads, and
that we update the hashes for all the svn-downloaded packages.

We chose to go with the second option, because this is what really
makes more sense, rather than hard-coding arbitrary values in the
environment. And we also have only one svn-downloaded package,
avrdude.

And thus, we're reaching the trigger for this change: avrdude is
impacted by the CVS-keyword expansion issue:

    https://svn.savannah.gnu.org/viewvc/avrdude/trunk/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js?revision=1396&view=markup

which would give two different files when checked out on different
machines:

    diff -durN foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js
    --- foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
    +++ bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
    @@ -1,6 +1,6 @@
     /**
      * @preserve jquery.layout 1.3.0 - Release Candidate 30.51
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      * $Rev: 303005 $
      *
      * Copyright (c) 2012
    @@ -4718,7 +4718,7 @@

     /**
      * jquery.layout.state 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2010
      *   Kevin Dalman (http://allpro.net)
    @@ -5074,7 +5074,7 @@

     /**
      * jquery.layout.buttons 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2010
      *   Kevin Dalman (http://allpro.net)
    @@ -5356,7 +5356,7 @@

     /**
      * jquery.layout.browserZoom 1.0
    - * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
    + * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
      *
      * Copyright (c) 2012
      *   Kevin Dalman (http://allpro.net)

So we also update the hash for avrdude.

Fixes:
    http://autobuild.buildroot.org/results/e3b/e3b0508047f32008ebfa83c5255ec5994b6af120/ (time issue)
    http://autobuild.buildroot.org/results/48e/48e78e84b425e79cdb98c16ab40247a0fa7e9676/ (keyword expansion issue)

Reported-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Reviewed-by: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-11 21:54:00 +01:00
Peter Korsgaard
e2fba6457b Update for 2021.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-03-06 22:16:45 +01:00
Yann E. MORIN
b9db38d370 package: drop _CPE_ID_VALID, use _CPE_ID_VENDOR
FOO_CPE_ID_VALID really ought to be an internal implementaion detail.
Packages that really want to trigger their CPE defintitions really
should set one of the actual variables to a meaningful value.

There are two CPE-related variables that we could chose to set to
replace FOO_CPE_ID_VALID: FOO_CPE_ID_VENDOR and FOO_CPE_ID_PRODUCT.
Between those two, _VENDOR more often diverges from the default than
_PRODUCT does, so that's what we use.

    ---8<------8<------8<------8<------8<---
    #!/bin/bash
    # Replace FOO_CPE_ID_VALID = YES with FOO_CPE_ID_VENDOR = foo_project
    for i in $(git grep -l -E '[^)]_CPE_ID_VALID = YES' package support); do
        pkg="$(basename "${i%/*}")"
        sed -r -i -e "s/_CPE_ID_VALID = YES/_CPE_ID_VENDOR = ${pkg}_project/" "${i}"
    done
    ---8<------8<------8<------8<------8<---

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Matthew Weber <matthew.weber@rockwellcollins.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[Peter: update cpe-test comment to reflect pkg3 change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-03-06 17:18:07 +01:00
Peter Korsgaard
a0aff89ed2 support/scripts/gen-bootlin-toolchains: correct xtensa-lx60 toolchain dependencies
Fixes:
http://autobuild.buildroot.net/results/011/0111c2ed54618daaeedfc66b0ea04eda00a7e855/
http://autobuild.buildroot.net/results/e53/e53e3880b63a23fa3b3e6d34664d40d5ddbdff89/
..

As listed in the br_fragment file of the toolchain, this is built for a
little-endian "custom" xtensa variant rather than the (big-endian) fsf one:

BR2_xtensa=y
BR2_XTENSA_CUSTOM=y

So update the dependencies in the script and regenerate Config.in.options /
toolchain test.  Also fixup the autobuild config snippet to match.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-03-02 23:52:49 +01:00
Vincent Fazio
b50eef1dfd support/download: drop sub-second precision in tarball creation
Some download backends, like svn, will provide timestamps with a
sub-second precision, e.g.

    $ svn info --show-item last-changed-date [...]
    2021-02-19T20:22:34.889717Z

However, the PAX headers do not accept sub-second precision, leading to
failure to download from subversion:

    tar: Time stamp is out of allowed range
    tar: Exiting with failure status due to previous errors
    make[1]: *** [package/pkg-generic.mk:148: [...]/build/subversion-1886712/.stamp_downloaded] Error 1

Fix that by massaging the timestamp to drop the sub-second part. We
do that in the generic helper, rather than the svn backend, so that
all callers to the generic helper benefit from this, as this is more
an internal details of the tarball limitations, than of the backends
themselves.

Reported-by: Roosen Henri <Henri.Roosen@ginzinger.com>
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
  - add Henri as reporter
  - move it out of the svn backend, and to the generic helper
  - reword the commit log accordingly
  - use an explicit time format rather than -Iseconds
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-22 23:01:27 +01:00
Romain Naour
03c3fbd81c support/scripts/boot-qemu-image.py: properly catch timeout
As reported on IRC by sephthir, the gitlab test of the defconfig
qemu_sparc_ss10_defconfig doesn't error out while the system
is not working properly.

This is because we explicitly wait for the timeout as an expected
condition, but do not check for it. Indeed, pexpect.expect() returns
the index of the matching condition in the list of expected conditions,
but we just ignore the return code, so we are not able to differentiate
between a successful login (or prompt) from a timeout.

By default, pexepect.expect() raises the pexpect.TIMEOUT exception on a
timeout, and we are already prepared to catch and handle that exception.
But because pexpect.TIMEOUT is passed as an expected condition, the
exception is not raised.

Remove pexpect.TIMEOUT from the list of expected conditions, so that the
exception is properly raised again, and so that we can catch it.

The qemu_sparc_ss10_defconfig is already fixed by
4d16e6f532.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
[yann.morin.1998@free.fr: reword commit log]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-20 22:44:29 +01:00
Peter Korsgaard
ecbfbabcf7 Update for 2020.11.3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 610e67b1fc)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-02-17 19:43:17 +01:00
Heiko Thiery
5b844d50d8 support/scripts/pkg-stats: add ignored_cves to json output
Add the list of <pkg>_IGNORE_CVES to the json output to show that we have a
known cause (available patch or the CVE is not valid for our package
configuration) that a affected CVE is not reported.

Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-02-16 23:15:06 +01:00
Arnout Vandecappelle (Essensium/Mind)
bcef4a03e7 support/scripts/cpedb.py: remove import pickle
pickle is no longer used since 09a71e6a75

Fixes:
support/scripts/cpedb.py:7:1: F401 'pickle' imported but unused

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-02-16 21:16:55 +01:00
Thomas Petazzoni
09a71e6a75 support/scripts/cpedb.py: drop CPE XML database caching
Currently, the CPE XML database is parsed into a Python dict, which is
then pickled into a local file, to speed up the processing of further
invocations.

However, it turns out that since the initial implementation, we have
switched the XML parsing from the out of tree xmltodict module to the
standard ElementTree one, which has made the parsing much faster. The
pickle caching only saves 6 seconds, on something that takes more than
13 minutes total.

In addition, this pickle caching consumes a significant amount of RAM,
causing the Python process to be OOM-killed on a server with 4 GB of
RAM.

So let's just drop this caching entirely.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-14 10:05:00 +01:00
Sergey Matyukevich
653b127d1e support/testing: TestATFAllwinner needs python3 for u-boot
New U-Boot version needs Python 3.x for pylibfdt.

Fixes:
- https://gitlab.com/buildroot.org/buildroot/-/jobs/1019385909

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-13 22:46:27 +01:00
Thomas Petazzoni
e1c0360606 support/config-fragments/autobuild: use Bootlin toolchain for RISC-V 64 musl
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/musl, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-05 09:27:37 +01:00
Thomas Petazzoni
63485a6c1e support/config-fragments/autobuild: use Bootlin toolchain for RISC-V 64 glibc
Instead of using an external toolchain built specifically for the
autobuilders to test RISC-V 64/glibc, use a pre-built Bootlin
toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-02-05 09:25:23 +01:00