Commit Graph

25 Commits

Author SHA1 Message Date
Fabrice Fontaine
f7922a57f5 Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE
Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE to better
"comply" with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 23:21:38 +01:00
Fabrice Fontaine
4b6202f721 Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT
Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT to better "comply"
with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-21 22:43:24 +01:00
Yann E. MORIN
728484538a support/testing: fix flake8 issue in CPE ID test
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-01-02 18:06:24 +01:00
Thomas Petazzoni
3950c53cd0 support/testing/tests/core/test_cpeid: new test
This commit adds a number of test cases to verify that the CPE_ID_*
variables are properly handled by the generic package infrastructure
and that the "make show-info" JSON output matches what we expect.

A total of 5 different example packages are used to exercise different
scenarios of CPE_ID_* variables usage.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-11-26 16:36:56 +01:00
Romain Naour
f52a9835fa support/testing: update to the new text representation of capabilities
Since libcap 2.41, the text representation of capabilities now use
the '=' instead of '+' [1].

This break our capabilities tests since we still use the old
representation.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/morgan/libcap.git/commit/?id=177cd418031b1acfcf73fe3b1af9f3279828681c

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981737

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:12:48 +01:00
Romain Naour
7b9762f4ab support/testing/test_hardening: add missing Kconfig symbol
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is needed to use the
custom external toolchain x86-i686--glibc--bleeding-edge-2018.11-1.tar.bz2

Otherwise the symbol BR2_TOOLCHAIN_EXTERNAL_URL is lost.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981738
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981739
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981740
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981741
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981742
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981743

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:11:12 +01:00
Thomas Petazzoni
0a5c9f824a support/testing/tests: fix flake8 warnings
Fixes the following flake8 warnings:

support/testing/tests/core/test_selinux.py:21:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:38:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:51:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:62:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:65:14: E127 continuation line over-indented for visual indent
support/testing/tests/init/test_systemd_selinux.py:53:1: E302 expected 2 blank lines, found 1
support/testing/tests/init/test_systemd_selinux.py:64:1: E302 expected 2 blank lines, found 1

Interestingly, the "continuation line over-indented for visual indent"
shows up only once, while the same pattern is there at multiple places
in the file. We fix all places with that over-indentation pattern.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-06 08:11:25 +01:00
Antoine Tenart
751b660c1f support/testing/tests/core/test_selinux: new tests for the packages SELinux functionalities
Add tests to ensure the packages SELinux functionalities (being able to
select an extra SELinux module in the refpolicy, and being able to
provide a custom SELinux module) are working as expected.

We use a BR2_EXTERNAL folder, provided in the tests, to use a custom
SELinux enabled package.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
0f3b931624 support/testing/tests/core/test_selinux: new test for BR2_PACKAGE_REFPOLICY_CUSTOM_GIT
Add a test for BR2_PACKAGE_REFPOLICY_CUSTOM_GIT (which allows to select
a custom location for the SELinux refpolicy). The test uses the official
refpolicy as a test (we only want to test the functionality is working,
not that another refpolicy is correctly building; that is an user
problematic).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
4fe09f4cab support/testing/tests/core/test_selinux: new test for BR2_REFPOLICY_EXTRA_MODULES_DIRS
Add a test for the BR2_REFPOLICY_EXTRA_MODULES_DIRS functionality (which
allows to provide custom SELinux modules).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Antoine Tenart
e383f2835f support/testing/tests/core/test_selinux.py: new test for BR2_REFPOLICY_EXTRA_MODULES
This patch adds a test for the BR2_REFPOLICY_EXTRA_MODULES
functionality (which allows to select extra modules within the SELinux
refpolicy using Kconfig).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-06 15:30:12 +02:00
Thomas Petazzoni
cc061128de support/testing/tests/core/test_timezone.py: fix indentation
Fixes:

support/testing/tests/core/test_timezone.py:7:9: E117 over-indented

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-08-14 21:56:08 +02:00
Ricardo Martincoski
4a40d36f13 support/testing: switch to Python 3 only
Python 2.7 will not be maintained past 2020.

Many scripts on the tree are used during the build and should keep
Python 2 compatibility for a while.
This is not the case for the runtime test infra. It's meant to be run in
modern distros only, so it can safely switch to support Python 3 only.

An advantage of this approach is to have less scenarios to test in.
Otherwise every change to the test infra or runtime tests would need to
be tested against both versions of the interpreter, increasing the
effort of the developers, to ensure the compatibility to Python 2 was
not broken.

In order to accomplish the change to Python 3:
 - change the shebang for run-tests;
 - use Python 3 urllib as a drop-in replacement for Python 2 urllib2;
 - when writing the downloaded binary files, explicitly open the output
   file as binary;
 - when subprocess is used to retrieve the text output from commands,
   explicitly ask for text output. For this, use 'universal_newlines'
   because 'text' was added only on Python 3.7;
 - when pexpect is used to retrieve the text output from qemu or git,
   explicitly ask for text output using 'encoding';
 - the code using csv currently follows the example in the documentation
   for the Python 2 module, change it to follow the example in the
   documentation for the Python 3 module;
 - fix the relative import for test_git.py to be Python 3 compliant.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-10-28 22:14:07 +01:00
Victor Huesca
152776b420 support/testing: new test for root password
Add support to test that the root passowrd is working as expected.
- Buildtime test: Check the hash present in the generated '/etc/shadow'.
- Runtime test: Build an armv7 image and try to login with a password.

Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-13 22:30:41 +02:00
Ricardo Martincoski
620c98a214 support/testing: factor out run_cmd_on_host
Currently many test cases call subprocess.check_output on their own.
Factor out that code to an infra method so the call get standardized.

This will be handful when switching the test infra to use Python 3.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-11 22:11:58 +02:00
Ricardo Martincoski
5c39c90ae1 support/testing: fix hardening tests
Since "2467822c85 package/checksec: bump to version 2.1.0" the hardening
tests fail because upstream slightly changed the way the script is
called.
According to README.md: "- All options now require `--$option=$value`
instead of `--$option $value`"

Instead of just replacing '--output json' with '--output=json' take into
account that upstream also changed the usage example to show --format
instead of --output. Both options do exactly the same, but following the
usage example seems to be more future-proof.

Upstream also improved the json output. Now when a file is passed as
parameter, the json has the file name as the main key, instead of the
string "file". Adjust the test cases accordingly.

Fixes:
tests.core.test_hardening.TestFortifyConserv
tests.core.test_hardening.TestFortifyNone
tests.core.test_hardening.TestRelro
tests.core.test_hardening.TestRelroPartial
tests.core.test_hardening.TestSspNone
tests.core.test_hardening.TestSspStrong

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-08-05 22:15:18 +02:00
Yann E. MORIN
409d4c3fe9 fs: don't use an intermediate tarball
Since 118534fe54 (fs: use a common tarball as base for the other
filesystems), the filesystem creation is split in two steps, using an
intermediate tarball to carry the generic, common finalisations to the
per-filesystem finalisation and image creation.

However, this intermediate tarball causes an issue with capabilities:
they are entirely missing in the generated filesystems.

Capabilities are stored in the extended attribute security.capability,
which tar by default will not store/restore, unless explicitly told to,
e.g. with --xattrs-include='*', which we don't pass.

Now, passing this option when creating and extracting the intermediate
tarball, both done under fakeroot, will cause fakeroot to report an
invalid filetype for files with capabilities. mksquashfs would report
such unknown files as a warning, while mkfs.ext2 would fail (with a
similar error message), e.g.:

    File [...]/usr/sbin/getcap has unrecognised filetype 0, ignoring

This is due to a poor interaction between tar and fakeroot; running as
root the exact same commands we run under fakeroot, works as expected.
Unfortunately, short of fixing fakeroot (which would first require
understanding the problem in there), we don't have much options.

The intermediate tarball was made to avoid redoing the same actions over
and over again for each filesystem to build. However, most of the time,
only one or two such filesystems would be enabled [0], and those actions
are usually pretty lightweight. So, using an intermediate tarball does
not provide a big optimisation.

The main reason to introduce the intermediate tarball, however, is that
it allows to postpone per-filesystem finalisations to be applied only
for the corresponding filesystem, not for all of them.

So, we get rid of the intermediate tarball, and simply move all of the
code to run under fakeroot to the per-filesystem fakeroot script.
Instead of extracting the intermediate tarball, we just rsync the
original target/ directory, and apply the filesystem finalisations on
that copy. The only thing still done in the rootfs-common step is to
generate the intermediate files (users file, devices file) that are used
in the fakeroot script.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=11216

Note: an alternate solution would have been to keep the intermediate
tarball to keep most of the common finalisations, and move only the
permissions to each filesystem, but that was getting a bit more complex
and changed the ordering of permissions and post-fakeroot scripts. Once
we bite the bullet of having some common finalisation done in each
filesystem, it's easier to just move all of them.

[0] Most probsably, users would enable the real filesystem to put on
their device, plus the 'tar' filesystem, to be able to easily inspect
the content on their development machine.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2018-11-13 00:47:36 +01:00
Matt Weber
aa2eb4bd64 support/testing: test_hardening fix flake8 whitespace
Resolves:
support/testing/tests/core/test_hardening.py:25:42: E231 missing whitespace after ','

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-20 17:27:28 +02:00
Matt Weber
963f824511 support/testing/tests/core: SSP & hardening flags
Catch the commonly used options of SSP, Relro, and fortify.
Using the package targets of busybox and lighttpd.  This
can easily be expanded to a larger list.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-20 13:25:27 +02:00
Ricardo Martincoski
5f201de0dc support/testing: add test for file capabilities
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-08-10 08:25:51 +02:00
Yann E. MORIN
bb2a57a17a fs: run packages' filesystem hooks in a copy of target/
Currently, some packages may register hooks to be run just before and
just after the generic tarball image is generated, because they need to
prepare the filesystem for read-only or read-write operation.

However, this means that, if any of the hooks or the image generation
fails, the target directory is left in a dangling, inconsistent state.

We fix that by doing a copy of target/, run the hooks on that copy,
generate the generic tarball image out of that, and get rid of the copy.

This way, we can guarantee consistency of the target directory, and we
can even ditch support for post-fs hooks (those that restore target/).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
02fc52fe59 tests: add test for post-fakeroot script
Currently, only post-build and post-image scripts were tested, each with
their own test-script.

The two test-scripts only differ in the name of the log file they
create, and it is based on the name of the script, so it is easy to
share the script.

This allows us to easily re-use it for testing post-fakeroot scripts.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-15 21:37:45 +01:00
Ricardo Martincoski
ef8d1f1b15 support/testing: fix code style
Fix the trivial warnings from flake8:
 - remove modules imported but unused;
 - use 2 lines before class or module level method;
 - remove blank line at end of file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-10-06 18:56:49 +02:00
Ricardo Martincoski
2927f412be support/testing: standardize defconfig fragments style
Change all defconfig fragments to take advantage of
"cf3cd4388a support/tests: allow properly indented config fragment".

Make each defconfig fragment:
 - start after a backslash;
 - be declared as a multi-line string literal;
 - be indented one level more than the variable that contains it.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-10-06 18:56:06 +02:00
Thomas Petazzoni
96e21b617d support/testing: add core tests
This commit adds a few Buildroot "core" tests, testing functionalities
such as:

 - post-build and post-image scripts
 - root filesystem overlays
 - timezone support

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-05-07 22:04:54 +02:00