Commit Graph

1569 Commits

Author SHA1 Message Date
Julien Olivain
ea47e177f0 package/rdma-core: new package
This is the userspace components for the Linux Kernel's
drivers/infiniband subsystem.

https://github.com/linux-rdma/rdma-core

Tested-by: Shamraiz Ashraf <shamraizashraf092@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
[yann.morin.1998@free.fr:
  - select iproute2 as it provides the 'rdma' utility
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-02-05 13:51:13 +01:00
Peter Korsgaard
a796cfb2e0 Update for 2022.11.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d48a8beb39)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-01-18 17:50:12 +01:00
Yann E. MORIN
fbddd12601 support/download: fix concurrent cargo vendor
Commit 8450b76918 (package/pkg-cargo: move CARGO_HOME into DL_DIR)
allowed for a shared cargo cache of crates. Internally, cargo is
supposed to lock themselves when accessing that cache, and that commit
even had some research in that area, pointing at [0] for complaints
about too-coarse the lock, so it was deemed safe to have a shared cargo
home.

However, in practice, the locking as implemented by cargo, fails to
properly protect the concurrent accesses to the crates cache, with random
failures that manifest themselves like so:

        Blocking waiting for file lock on package cache
        Blocking waiting for file lock on package cache
     Downloading crates ...
    error: failed to sync
    Caused by:
      failed to download packages
    Caused by:
      failed to download `autocfg v1.1.0`
    Caused by:
      unable to get packages from source
    Caused by:
      failed to unpack package `autocfg v1.1.0`
    Caused by:
      failed to unpack entry at `autocfg-1.1.0/src/tests.rs`
    Caused by:
      No such file or directory (os error 2) while canonicalizing [...]

with the last few errors sometime being:

    Caused by:
      failed to parse manifest at `[...]/aho-corasick-0.7.18/Cargo.toml`
    Caused by:
      can't find library `aho_corasick`, rename file to `src/lib.rs` or specify lib.path

So, as we do not systematically use our own cargo build (we can use a
pre-built one with host-rust-bin), we can't patch cargo (even if we knew
what to do!).

Instead, we implement a lock ourselves, by wrapping the call to "cargo
vendor" with a flock(1) on cargo home.

Note: the download wrapper is already flock-ed, but it is a per-package
lock, so it does not prevent different packages from being downloaded in
parallel; if those packages need cargo vendoring, that will not be
protected by the flock on the dl wrapper. So we really do need a flock
on cargo home.

[0] https://github.com/rust-lang/cargo/issues/6930

Fixes: 8450b76918

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Moritz Bitsch <moritz@h6t.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-01-14 21:00:16 +01:00
Yann E. MORIN
ac2b26fba2 support/download: fix cargo vendoring
Commit de4cf25375 (package/{rust, rust-bin}: bump to version 1.66.0)
forgot, despite the big comment above the version strings, to confirm
that the vendoring was still working.

Previously, we were adding the vendoring equivalence manually, but in
commit 04154a6517 (support/download/cargo-post-process: cargo output
for vendor config), we switched to using the output of "cargo vendor"
(on stdout) to support cases were the vendoring equivalence would be
more complex (e.g. when using crates not hosted on crates.io).

With rust until and including 1.65.0, "cargo vendor" would output (for
crates.io crates) the same output as our manual fixups, except it was
preceded by an empty line. So, to avoid recompting all our hashes, we
added a tweak to strip away the leading empty line in 04154a6517.

But rust 1.66.0 includes [0] which changes the output (on stdout) of
"cargo vendor", where the first empty line is no longer emitted.

This means that our tweak for rust 1.65.0 now strips out an important
part of the cargo vendor output, which renders the archives invalid, and
thus generates different archives, which fail to validate against our
hashes.

Fix this by doing what the comment in the post-process helper states,
and just keep the whole output of "cargo vendor", by just removing the
"tail --lines=+2". Since that comment is no longer meaningful, we drop
it too.

Now, all our 6 cargo-based packages, as well as our 5 python packages
that have rust code, can be vendored again, without changing our hashes,
but most importantly, with valid archives.

Still, we keep the comment above the versions strings, in the hope that
a future bumper will notice and be more careful at validating the
vendoring.

[0] https://github.com/rust-lang/cargo/pull/11273

Fixes:
    http://autobuild.buildroot.org/results/bea/beac7674bbc9fd2f8777b5861f65afee9c485753/  (bat)
    http://autobuild.buildroot.org/results/d1e/d1ec1ebbde115628a4b8b9099544347242a97c1c/  (dust)
    http://autobuild.buildroot.org/results/f96/f968be895be9ca98b314fdd688ef8d3bdf4e5dfb/  (hyerfine)
    http://autobuild.buildroot.org/results/a0c/a0cdb6cc9493f5248d98f98b13da854e12adc2be/  (ripgrep)
    ... and so many others...

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Simon Richter <simon.richter@ptwdosimetry.com>
Reviewed-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-01-10 17:58:05 +01:00
Marcin Niestroj
82522d2547 package/python-dbus-fast: new package
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Reviewed-by: James Hilliard <james.hilliard1@gmail.com>
[yann.morin.1998@free.fr:
  - select host-cython in Config.in
  - introduce obj_path in test sample
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-01-02 22:33:55 +01:00
Yegor Yefremov
2f0a2a43bb support/testing: add test for python-munch
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-12-29 10:38:11 +01:00
Yann E. MORIN
44161560dd support/tests: print failed command and output on assertRunOK error
Currently, when asserting that a command succeeded, we just capture the
return code of the command. If that is not zero, the assertion fails,
but the error message is not very splicit:
    AssertionError: 1 != 0

Replace the error message with an explicit message that dumps the failed
command, the error code, and the resulting output.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-12-27 21:40:31 +01:00
Yann E. MORIN
0627cb0e8a support/testing: remove leftover kernel config fragment
Commit 86d32208b6 (support/testing/tests/init/test_systemd.py: use
downloaded kernel) stopped building a custom kernel for the systemd
tests, but forgot to drop the associated kernel config fragment.

That fragment is now not used in any test case, so we can drop it.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-12-22 11:47:26 +01:00
Yegor Yefremov
b54e0dde30 support/testing: add test for python-dicttoxml2
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-12-11 21:18:57 +01:00
Julien Olivain
6ad775fc81 support/testing/tests/package/test_ncdu.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-12-11 11:50:15 +01:00
Peter Korsgaard
6ca0edcb5f Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-12-05 10:01:26 +01:00
Peter Korsgaard
40bd4a32aa Update for 2022.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-12-05 08:23:19 +01:00
Julien Olivain
2ad68ff8df package/z3: new package
Z3, also known as the Z3 Theorem Prover, is a cross-platform
satisfiability modulo theories (SMT) solver.

https://github.com/Z3Prover/z3

Signed-off-by: Julien Olivain <ju.o@free.fr>
[yann.morin.1998@free.fr:
  - python bindings 'depends on' python, not 'select' it
  - fix check-package in test_z3.py
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-11-20 14:54:58 +01:00
Peter Korsgaard
12a33b54b3 Update for 2022.08.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25680e6aa8)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-11-17 08:24:23 +01:00
Gleb Mazovetskiy
f7a6011f78 Makefile: unexport PERL_MM_OPT
Compilation of Perl-related packages fails if `PERL_MM_OPT` is defined.

We previously issued an error in this case.
Instead, simply `unexport` the variable.

Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-07 23:10:47 +01:00
Thomas Petazzoni
b9dd9ee857 support/testing/tests/package/test_gdb.py: drop version-specific tests
Back when support/testing/tests/package/test_gdb was introduced, there
was a significant difference in how gdb < 10 and gdb >= 10 were
handled in gdb.mk, which explained why we were testing both gdb 9.x
and gdb 11.x.

However, support for gdb 9.x has now been dropped, and we only support
gdb >= 10.x, so testing gdb 9.x and 11.x separately no longer make
much sense. In addition:

 - other GDB tests in the same file already test the default version,
 which is now 11.x, meaning we in fact have duplicated tests between
 the ones testing the default version and the ones testing 11.x
 specifically

 - GDB 9.x has been removed, which means all the tests testing GDB 9.x
 are failing, with a Config.in.legacy build error.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828456 (TestGdbHostOnly9x)
  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828454 (TestGdbHostGdbserver9x)
  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828451 (TestGdbHostGdbTarget9x)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:08:10 +01:00
Thomas Petazzoni
e7930708a3 support/testing/tests/package/test_python_crossbar: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_crossbar test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828587

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:07:58 +01:00
Thomas Petazzoni
3884cfc057 support/testing/tests/package/test_python_flask*: increase time after server startup
It seems like on Gitlab CI, the runners are quite slow, and the Flask
server does not startup in the 15 seconds we give it. So increase this
to 30 seconds before trying to contact the Flask server.

Hopefully fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828594

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:07:54 +01:00
Christian Stewart
5d8371a3fe support/testing/tests/package/test_docker-compose: update kernel to 4.19.262
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:15:01 +01:00
Christian Stewart
f486521b27 support/testing/tests/package/test_docker-compose: quiet overly verbose output
Docker compose up outputs terminal control characters intended for
interactive output viewing.

Wget similarly can use the -q option to produce quieter logs.

Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:57 +01:00
Christian Stewart
685d8a0318 support/testing/tests/package/test_docker-compose: fix docker compose container name
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:46 +01:00
Christian Stewart
2e99a233bd support/testing/tests/package/test_docker-compose: fix docker compose binary name
Docker compose is now invoked as "docker compose" not "docker-compose."

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828442

Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:29 +01:00
Simon Richter
04154a6517 support/download/cargo-post-process: cargo output for vendor config
Use the output of `cargo vendor` to generate the vendor configuration.

Fixes the need to patch the generated configuration if there are
non-crates.io dependencies.

Note:
  `cargo vendor` currently prints a newline before it prints the
  needed configuration.

  This is fixed in +nightly, will end up in +stable soon and must
  be considered when updating cargo.
  See: https://github.com/rust-lang/cargo/pull/11273

  Until then it is needed to remove this first line to make sure
  that the contents of .cargo/config will be the same as they were
  generated with the earlier version of the script. Thus, the
  hashes of the packages that use this script remain the same.

Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
[yann.morin.1998@free.fr: add comment in rust-bin.mk and rust.mk]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-31 10:06:17 +01:00
yann.morin@orange.com
9d948e1b34 toolchain: support gconv modules from glibc >= 2.34
Startig with glibc 2.34, the gconv modules description has been split in
two:
  - a common definition in the old location, /usr/lib/gconv/gconv-modules
  - specific definitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/

This is done so as to simplify the handling of glibc gconv modules, and
eventually to segregate those outside of glibc, and so that third-parties
may also provide their own gconv converters and their definitions.

And starting with that same glibc version, most of the gconv modules
definitions are moved to an extra configuration file in that
sub-directory.

It is thus no longer possible to use special code pages, like cp850,
which are very useful to access FAT-formatted devices.

Add support for this new gconv layout, while keeping support for older
glibc versions. Note that the modules themselves are not moved or
renamed, just the definition files have changed.

Instead of passing the one old gonv modules definitions file on stdin,
we pass the base directory to that file, and move into the script the
responsibility to find all the gconv definition files.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-21 21:02:40 +02:00
yann.morin@orange.com
822cc1ebc4 support/scripts: don't require gawk to generate glibc gconv modules
When only a subset of the glibc gconv modules are installed, we need to
generate a trimmed-down list of available modules. We currently use gawk
for that.

However, we are not using any GNU extension in that awk script, and it
happens to work as expected when using mawk (which has no GNU
extension).

Commit 11c1076db9 (toolchain: add option to copy the gconv libraries)
did not explain why it used gawk explicitly, and given the age for that
commit, we doubt we'd be able to have the involved participants recall
anything from that period...

Besides, gawk is not a requirement for Buildroot.

Switch over to using plain awk.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-21 21:01:25 +02:00
José Luis Salvador Rufo
1b77c2170d support/testing/tests/package/test_zfs: increase timeout for the ZFS tests
Newest versions requires a bit more time to finish.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-10-06 20:46:12 +02:00
Peter Korsgaard
1648c2ec36 Update for 2022.08.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25d865996d)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-10-03 08:49:17 +02:00
Michael Klein
a54a7bf805 support/scripts/size-stats: count compiled python (.pyc) files
Any .pyc files generated by the pycompile script during target
finalization are currently counted in the "Unknown" package,
because packages-file-list.txt only contains the source .py file.

If a .py file is added to filesdict, add the corresponding .pyc
file as well.

Signed-off-by: Michael Klein <m.klein@mvz-labor-lb.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-24 10:34:30 +02:00
Lang Daniel
2040961b48 package/polkit: test /usr/share/polkit-1/rules.d
Polkit has two directories that are used to store rules.
Add the second directory to the existing tests, to ensure
that both work in the future.

Signed-off-by: Daniel Lang <d.lang@abatec.at>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-17 23:36:10 +02:00
Yann E. MORIN
c25b445488 support/tests/cpio: extend runtime tests
Check that dependencies that are DT_NEEDED by a program are
automatically copied by dracut. We use cramfs, the package,
as it is small and just depends on libz.

Test more than one dracut config file.

Reported-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
[Arnout: remove test for features that haven't been merged yet]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-17 22:13:23 +02:00
Yann E. MORIN
c8a618cff7 support/test/cpio: test for pv already done by listing the cpio archive
Since we do not pivot_root/switch_root from the cpio, whatever we get
in the rootfs was exactly what we got by listing the cpio archive.

Drop the test for the presence of pv, it's redundant.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-17 22:13:22 +02:00
Julien Olivain
680ad2d44b support/testing/tests/package/test_hwloc.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-17 15:06:08 +02:00
Thomas Petazzoni
43fc826d82 support/config-fragments/autobuild/bootlin-armv7m-uclibc.config: specify configuration more precisely
Due to a bug in the CodeSourcery ARM toolchain packaging, this
toolchain currently appears as available even for noMMU
configurations, which is obviously wrong. Due to this, the
bootlin-armv7m-uclibc.config fragment ends up using the CodeSourcery
ARM toolchain, which is obviously wrong for an ARM noMMU
configuration, causing a build failure when matching the toolchain
capabilities with the configuration.

Even though we will separately fix the CodeSourcery ARM toolchain
packaging, it makes sense to ensure that the
bootlin-armv7m-uclibc.config fragment explicitly selects the Bootlin
toolchain.

Reported-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-17 14:45:47 +02:00
Peter Korsgaard
be914b97ad Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-11 09:57:08 +02:00
Peter Korsgaard
0003fdbed3 Update for 2022.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-10 17:00:09 +02:00
Marcus Hoffmann
54e5a64200 support/testing/tests/package/test_python_paho_mqtt: new runtime test
This tests valdates that we can publish a message and read it back.

Signed-off-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
  - don't manually start mosquitto, there's a startup script for that
  - don't pass custom timeout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-31 21:37:50 +02:00
Peter Korsgaard
41b1b65c5a Update for 2022.05.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bf0d8c9659)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-08-31 21:06:05 +02:00
Arnout Vandecappelle
6c7061f3a8 support/scripts/pkg-stats: fix flake8 errors
Fixes flake8 errors:
support/scripts/pkg-stats:1013:133: E501 line too long (164 > 132 characters)
support/scripts/pkg-stats:1018:36: F541 f-string is missing placeholders
support/scripts/pkg-stats:1110:199: E261 at least two spaces before inline comment

For the first and the last one, we chose to split the long lines rather
than adding noqa: 501. Indeed, the long lines make it very unreadable,
and there are relatively natural places where the line can be broken.
Also split a line just below the second one in a similar way.

The f-string on 1018 doesn't need to be an f-string.

Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-08-27 23:02:38 +02:00
Thomas Petazzoni
d68628a538 support/scripts/pkg-stats: remove remaining double quote escaping
This is done either by switching to single quoted f-strings, triple
double quoted f-strings when needed, or simply single-quoted strings.

The renderer HTML is exactly identical before/after this commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
2022-08-24 13:36:12 +02:00
Thomas Petazzoni
98a3fba940 support/scripts/pkg-stats: remove useless escaping of double quotes
Within single-quoted f-strings, and within triple double quoted
strings, escaping all the double quotes is completely useless and
makes the code more difficult to read. Get rid of all this useless
escaping.

The renderer HTML is exactly identical before/after this commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
2022-08-24 13:35:49 +02:00
James Hilliard
fc429c4dc6 package/pkg-python: clean conflicting pep517 packages before install
The python installer package isn't able to overwrite files of packges
that already exist, this causes problems when doing a rebuild or
update without a full clean.

To fix this we can use functionality from importlib to identify and
remove any conflicting python package files before installation.

We also need to use internals from python-installer, as we want to use
the same logic as pyinstaller uses internally for getting the scheme so
that we ensure we clean the correct package scheme (we want it to be the
same as the one we're installing)

Fixes:
Traceback (most recent call last):
  File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 69, in <module>
    main()
  File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 61, in main
    install(
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/_core.py", line 109, in install
    record = destination.write_file(
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 207, in write_file
    return self.write_to_fs(scheme, path_, stream, is_executable)
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 167, in write_to_fs
    raise FileExistsError(message)
FileExistsError: File already exists: /home/buildroot/buildroot/output/target/usr/lib/python3.10/site-packages/tinycss2/__init__.py

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
[yann.morin.1998@free.fr:
  - extend commit log about the use of the installer internals (the
    symbols prefixed with '_')
  - check path.files against explicitly None
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-22 11:54:25 +02:00
James Hilliard
d752f73b0c package/pkg-python: use pyinstaller.py for host python packages
The python installer cli isn't able to overwrite files of packages
that already exist, this causes problems when doing a rebuild or
update without a full clean.

Since we need to add functionality to our pyinstaller.py script to fix
this issue we must also use pyinstaller.py for host python packages.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-22 11:39:27 +02:00
Fabrice Fontaine
ba2659401f support/dependencies: check for find and xargs
Commit 7652817c93 updated the
documentation but forgot to update support/dependencies

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-21 15:50:15 +02:00
Thierry Bultel
bd0b2db231 support/testing/tests/fs/test_cpio.py: new runtime test
It includes a simple test for the full cpio image, and a test of the
dracut image. To validate that the dracut image is a subset of the full
image, 'pv' is added to the image, and the test verifies that pv is not
part of the image. Note that the real rootfs is not mounted at the
moment, so pv is never available in the running image.

Systemd and other init systems are currently untested.

Signed-off-by: Thierry Bultel <thierry.bultel@linatsea.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-18 22:49:53 +02:00
Thomas Petazzoni
7ad5e6cede support/testing/tests/package/test_python_s3transfer: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_s3transfer test is
too large, and doesn't fit as an initramfs in the 256MB of RAM
available in the versatilepb machine. This causes a "Initramfs
unpacking failed: write error" when booting, and many files being
missing from the root filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635126

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:56:46 +02:00
Thomas Petazzoni
0813ec1aa0 support/testing/tests/package/test_python_botocore: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_botocore test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635042

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:55:50 +02:00
Thomas Petazzoni
a9df206190 support/testing/tests/package/test_python_boto3: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_boto3 test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635041

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:55:19 +02:00
Thomas Petazzoni
0214ee94e0 support/testing/tests/fs/test_f2fs: fix test after f2fs-tools bump
In commit
9267b0f14d ("package/f2fs-tools: bump to
version 1.15.0"), f2fs-tools was bumped from 1.14.0 to 1.15.0.

It turns out that this version bump causes the output of dump.f2fs to
slightly change.

In version 1.14.0, it looked like this:

Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 262144 (128 MB)
Info: MKFS version
  "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
  from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
    to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = 70c101c3
Info: checkpoint state = 181 :  trimmed nat_bits unmount

In version 1.15.0, it looked like this:

Info: MKFS version
  "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
  from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
    to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: Segments per section = 1
Info: Sections per zone = 1
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = b89f8bb
Info: checkpoint state = 181 :  trimmed nat_bits unmount

You will notice that the message "Info: total sectors = 262144 (128
MB)" is no longer present, and only "Info: total FS sectors =
262144 (128 MB)" is not present.

Except our test case was precisely looking for this "Info: total
sectors" string in the output, causing the test to fail.

We fix this by simply matching on "Info: total FS sectors" now.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884634814

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 18:08:02 +02:00
Julien Olivain
4b291f1bac support/testing: octave: increase test timeout
Octave package test can occasionally fail due to timeout while testing
some octave modules. This commit slightly increase the timeout value
to reduce those failures.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-08 23:07:23 +02:00
Emile Cormier
bfafb08c62 package/python-crossbar: adjust dependencies based on requirements-min.txt
This commit makes sure that the python-crossbar package pulls in the
right dependencies, based on the requirements-min.txt.

It does so by:

 - Changing the 0002-Remove-idna-requirement patch by a more thorough
   patch that drops all indirect dependencies from
   requirements-min.txt, making it easier to have a 1:1 mapping
   between lines in requirements-min.txt and Buildroot selects.

 - Changing the
   0003-crossbar-webservice-wap-use-markupsafe-instead-of-we patch to
   update requirements-min.txt to indicate the new MarkupSafe
   dependency. Here again, to have a 1:1 mapping between lines in
   requirements-min.txt and Buildroot selects.

 - Updating the Buildroot selects to match requirements-min.txt, with
   relevant comments when it does not.

 - Fixing up the Crossbar test case to no longer force autobahn to use
   umsgpack. Instead, we now use the default of msgpack that is
   expected by autobahn.

Fixes bug #14556, https://bugs.busybox.net/show_bug.cgi?id=14556.

Signed-off-by: Emile Cormier <emile.cormier.jr@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-08 22:27:35 +02:00