Commit Graph

1374 Commits

Author SHA1 Message Date
Yann E. MORIN
be9ffe3a4e support/misc/utils: introduce $(tab)=\t and $(escape)=\x1b
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-30 21:31:27 +01:00
Yann E. MORIN
5b00c382fc support/utils: make-comma-list does just that, not quoting
Currently, we have two functions that build a comma-separated list
of items; one is double-quoting the items, while the other is
single-quoting them. Their naming is not very consistent.

Besides, in a followup change, we will need to build a comma-separated
list of items that are already double-quoted.

Introduce a macro that does just build a comma-separated list, and
use that in the two other macros; rename the existing macro so the
naming is consistent.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-30 21:31:16 +01:00
Thomas Petazzoni
b3c66481e1 toolchain: re-generate Bootlin toolchain descriptions
Following the releases of 2021.11 Bootlin toolchains, this commit
represents the result of re-running the gen-bootlin-toolchains script.

The only part that isn't auto-generated are the contents of
Config.in.legacy, which account for the replacement of the RISC-V LP64
toolchain by RISC-V LP64D toolchains.

The complete set of runtime test cases was verified on Gitlab CI:

  https://gitlab.com/tpetazzoni/buildroot/-/pipelines/437767674

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-12-30 18:43:23 +01:00
Thomas Petazzoni
bbfcb19c55 support/scripts/gen-bootlin-toolchains: handle RISC-V 64-bit toolchain change
toolchains.bootlin.com no longer provides a LP64 RISC-V 64-bit
toolchain, but a more useful LP64D RISC-V 64-bit toolchain. Of course,
the old tarballs remain available, but no new versions of the LP64
toolchain will be produced.

This commit reflects this change in the gen-bootlin-toolchains script.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-12-30 18:43:14 +01:00
Thomas Petazzoni
65d2f04c01 support/scripts/generate-gitlab-ci-yml: make it possible to test multiple defconfigs
Already supported:

 - Pushing a branch called "<foo>-defconfigs" tests all defconfigs.

 - Pushing a branch called "<foo>-defconfig-<defconfig-name>" will
   test one particular defconfig

This commit adds support for:

 - Pushing a branch called "<foo>-defconfigs-<pattern>" which will
   test all defconfigs whose name start with the pattern. For example
   "<foo>-defconfigs-qemu_" will test all Qemu defconfigs

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-12-30 18:41:53 +01:00
Francois Perrad
a15cb73b49 package/lua-argon2: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-27 22:25:30 +01:00
Arnout Vandecappelle (Essensium/Mind)
7a947133ab support/testing: test_php_lua: fix flake8 error
support/testing/tests/package/test_php_lua.py:35:1: E302 expected 2 blank lines, found 1

Add the missing line before class definition.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-20 18:56:15 +01:00
Herve Codina
403b62943b package/php-pam: new package
The php-pam package provides a PHP PAM (Pluggable Authentication
Modules) integration.

https://pecl.php.net/package/PAM

Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-17 22:46:55 +01:00
Herve Codina
d49127dd4c package/php-lua: new package
The php-lua package provides a PHP extension that embeds the lua
interpreter and offers an OO-API to lua variables and functions.

https://pecl.php.net/package/lua

Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>

Two patches are present and were retrieved from the following
upstream pull request in order to support PHP8:
https://github.com/laruence/php-lua/pull/47

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-17 22:46:55 +01:00
Herve Codina
8ddeeffa18 package/php-apcu: new package
APCu is an in-memory key-value store for PHP.
Keys are of type string and values can be any PHP variables.
APCu only supports userland caching of variables

https://pecl.php.net/package/APCU

Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-12-17 22:46:55 +01:00
Cyril Bur
c988867fd2 support/scripts/pkg-stats: Don't print directory in 'make' call
It is possible that some users of buildroot have put it in a repository
and call into it from another Makefile such as:
.DEFAULT:
	$(MAKE) O=$(abspath $(O)) -C buildroot $(@)

This technique works well except that Make tells us that it changes into
the buildroot directory:
make[1]: Entering directory 'buildroot'

Because this line doesn't have an equals within it, python raises a
ValueError exception within pkg-stats.

This patch has python tell the invoked make not to print directories

Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-11 21:35:19 +01:00
Romain Naour
8cce2ce1d2 support/testing: add gitlab download helper testing
Like for the github helper, add some tests to test the download of
Gitlab's generated tarball.

[1] f83826c90d

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-06 22:12:24 +01:00
Romain Naour
1ca6ab6ace support/testing: add github download helper testing
Back in 2013, a github download helper has been introduced to cope with
changes in github download-URL's [1][2].

Since then a testing infrastructure has been introduced in Buildroot
but no tests has been added to check if the github download helper is
still working.

It was reported recently [3] that the github helper doesn't work anymore
using tags. Buildroot is not the only project having the issue, see
Github feedback discussions [4].

Add tests for direct archive download (archives uploaded by maintainers),
download from a git tag and git hash using the github helper.
Make sure that Buildroot doesn't use BR2_BACKUP_SITE
(http://sources.buildroot.net).

[1] https://bugs.busybox.net/show_bug.cgi?id=6302
[2] c7c7d0697c
[3] https://bugs.busybox.net/show_bug.cgi?id=14396
[4] https://github.com/github/feedback/discussions/8149

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-12-06 22:12:20 +01:00
Peter Korsgaard
e6e12337f1 Update for 2021.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-12-05 20:59:16 +01:00
Yann E. MORIN
77304e5143 support/dependencies: avoid spurious warning on print-vars
When calling 'printvars', the 'suitable-host-package' macro is printed
(a macro is just a variable like the others, after all, just with some
parameters). Because it is printed as a variable, it is missing its
parameters, but it still tries to evaluate the $(shell) construct.

This causes spurious warning:

    make[1]: support/dependencies/check-host-.sh: Command not found

Only try and call the script if there is actually a tool to check for.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-12-04 20:09:04 +01:00
Yann E. MORIN
37004bde66 support/docker: drop use of Docker hub, use Gitlab registry
Now that our pipelines are using the Docker image from the Gitlab
registry, there is no longer any reason to push the image to the
Docker hub.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-11-28 14:39:47 +01:00
Yann E. MORIN
9f6a2dbe03 support/docker: don't use sudo to build
On a properly setup machine, it is totally useless to use sudo to run
docker; it is very bad practice. Instead, users really should add
themselves to the docker group.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-11-28 14:39:40 +01:00
Yann E. MORIN
b919d5dbba support/pkg-stats: support values with an equal sign in them
The heuristic to extract the various variables of interest is pretty
crude: we filter on variables ending with certain suffixes (like
'%_VERSION' to get the version strings).

However, in doing so, we may dump variables that are not actual package
versions (especially with br2-external trees), and those may contain one
or more equal sign.  And anyway, an actual package version string may
very well contain an equal sign too.

But the current situation is that the output of 'printvars' is split on
all equal signs, which will not fit in the 2-tuple we assign the result,
thus causing an exception.

Fix that by limiting to a single split.

Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-11-13 14:59:42 +01:00
Romain Naour
bff1c6ada5 support/testing: test_luvi: make luvi test reproducible
As explained by Jörg [1], iteration with pairs() does not result in the
same order since luajit 2.1.

From [2]
"Table iteration with pairs() does not result in the same order?

The order of table iteration is explicitly undefined by the Lua
language standard. Different Lua implementations or versions may use
different orders for otherwise identical tables. Different ways of
constructing a table may result in different orders, too. Due to
improved VM security, LuaJIT 2.1 may even use a different order on
separate VM invocations or when string keys are newly interned.

If your program relies on a deterministic order, it has a bug.
Rewrite it, so it doesn't rely on the key order.
Or sort the table keys, if you must."

Note: The "luvi -v" return 255 even on success.

[1] http://lists.busybox.net/pipermail/buildroot/2021-November/627938.html
[2] https://luajit.org/faq.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Francois Perrad <francois.perrad@gadz.org>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-11-12 23:26:17 +01:00
Romain Naour
1477b60d4e support/testing: test_luvi: switch to armv5 to boot with rng support enabled
Since the commit replacing moonjit by luajit [1] luvi doesn't work without
rng support enabled.

Switch to armv5 to use virtio-rng-pci on the qemu command line [2].

[1] 9450b53c8e
[2] https://git.buildroot.net/buildroot/tree/support/testing/infra/emulator.py?h=2021.08.1

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-11-12 23:23:44 +01:00
Romain Naour
8ea6eead60 support/scripts: don't build board defconfigs with Gitlab's pipelines trigged on tag
Currently when a tag is added to the Buildroot git tree, the gitlab-ci
create a pipeline with several hundred of jobs (~750) to build all
defconfigs and execute the Buildroot testsuite.

However, there is only a limited number of gitlab-ci runner (9 runners)
and some jobs reach the timeout limit (24h) while waiting for a runner
[1]. Indeed, the Buildroot project doesn't use the Gitlab's shared
runners.

In addition to the pipeline created when a new tag is added to the
git repository, two pipelines are created each weeks to execute the
Buildroot testsuite (on monday [2]) and build all defconfigs (on
Thursday [3]).

At some point there are too many jobs waiting in gitlab due board
defconfigs builds. Indded a board defconfig requires a lot of time
(~30min) compared to other jobs in order to build a toolchain and a
kernel linux along with a basic rootfs. There is currently 262
defconfigs.

This is even worse when several pipelines are trigged at the same
time (new git tag and scheduled pipeline trigger).

In order to reduce the number of long jobs, don't build board
defconfigs with pipelines trigged on tag, keeping only the runtime
tests and the Qemu's defconfigs.

[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/1758966541
[2] https://gitlab.com/buildroot.org/buildroot/-/pipelines/404035190
[3] https://gitlab.com/buildroot.org/buildroot/-/pipelines/401685550

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-11-11 09:24:47 +01:00
Peter Korsgaard
19e2b2606f Update for 2021.08.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d815599e37)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-11-10 16:43:48 +01:00
Pierre-Jean Texier
38c1c535ad support/testing/tests/package/sample_python_unittest_xml_reporting.py: fix flake8 warnings
Fixes:

support/testing/tests/package/sample_python_unittest_xml_reporting.py:4:1: E302 expected 2 blank lines, found 1
support/testing/tests/package/sample_python_unittest_xml_reporting.py:8:1: E305 expected 2 blank lines after class or function definition, found 1
1     E302 expected 2 blank lines, found 1
1     E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Pierre-Jean Texier <texier.pj2@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-11-08 22:12:22 +01:00
Pierre-Jean Texier
7140ebbda5 support/testing/tests/package/test_python_unittest_xml_reporting.py: fix flake8 warnings
Fixes:

support/testing/tests/package/test_python_unittest_xml_reporting.py:3:1: F401 'time' imported but unused
1     F401 'time' imported but unused
make: *** [Makefile:1220: check-flake8] Error 123

Signed-off-by: Pierre-Jean Texier <texier.pj2@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-11-08 22:12:13 +01:00
Romain Naour
d2f92512f6 support/testing: test_jffs2.py: update logical eraseblock size for qemu >= 2.9
The test_jffs2 test fail for the same reason as test_ubi test with qemu >= 2.9
due to a qemu 2.8 bug. See commit d8447c38f5.

Divide the erase block size by two.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1687590514

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-11-05 22:30:36 +01:00
Thomas Petazzoni
9d3428beca package/python-unitest-xml-reporting: new package
Based on initial work from Nicolas Carrier
<nicolas.carrier@orolia.com>, with the following additions:

- Updated to a newer version
- Added proper license file handling
- Added runtime test case
- Restricted to Python 3.x

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-11-05 17:04:32 +01:00
Romain Naour
8b4a675854 support/testing: fix the TestPythonPy[2,3]Incremental test cases
The recent python-incremental version bump (commit
fd03cb5a12) added a change/fix
in incremental.Version().

python-incremental 17.5.0 return: "[package, version 1.2.3rc4]")
python-incremental 21.3.0 return: "[package, version 1.2.3.rc4]")

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1710552692
https://gitlab.com/buildroot.org/buildroot/-/jobs/1710552693

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-10-31 18:42:55 +01:00
Herve Codina
7f9ad0ed7f package/php-pecl-dbus: new package
The php-pecl-dbus package provides a PHP extension for interaction
with D-Bus busses.

https://github.com/derickr/pecl-dbus

Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-27 22:10:38 +02:00
Thomas Petazzoni
23186356a1 support/scripts/generate-gitlab-ci-yml: allow multiple tests in one pipeline
The current Gitlab CI mechanism allows to trigger all tests in a CI
pipeline by pushing a branch named <something>-runtime-tests, or to
trigger a single test in a CI pipeline by pushing a branch name
<something>-tests.<name of test>.

However, there are cases where it is useful to run a suite of tests,
for example to run all tests in tests.init.test_busybox.

This commit makes that possible by extending the current semantic of
<something>-tests.<name of test> to not expect a complete test name,
but instead to accept all tests that starts with the given pattern.

This allows to do:

  git push gitlab HEAD:foobar-tests.init.test_busybox.TestInitSystemBusyboxRo

like it was the case before. But it now also allows to do:

  git push gitlab HEAD:foobar-tests.init.test_busybox

to run all Busybox tests.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-27 21:31:49 +02:00
Raphaël Mélotte
5562eb58e8 support/testing: add new test for python-boto3
Add a rudimentary test inspired from the "Using boto3" section in
the package README ([1]).

Note that it doesn't try to do anything with the instanciated
resource, as this would require a network connection when the test
runs.

[1]: https://github.com/boto/boto3

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-26 21:34:27 +02:00
Raphaël Mélotte
9060f1a6b8 support/testing: add new test for python-botocore
Add a rudimentary test inspired from the "Using botocore" section in
the package README ([1]).

Note that it doesn't try to use the instantiated client, as this would
require a network connection when the test runs.

[1]: https://github.com/boto/botocore

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-26 21:34:27 +02:00
Raphaël Mélotte
0514ced833 support/testing: test_python_rsa: increase timeout
On some developers machines, the default timeout (5 seconds) is not
enough for the test to succeed.

Increase it to 20 seconds, to let more time for the rsa keys to be
generated.

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-26 21:34:15 +02:00
Adam Duskett
1e363383b6 support/testing/tests/package/test_python_pyndiff: new test
A simple test that runs nmap twice to create the files scanme-1.xml and
scanme2.xml, then runs pyndiff on both files.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-21 22:48:44 +02:00
Thomas Petazzoni
11efcb39b2 support/scripts/pkg-stats: use the new 'stable_versions' field of release-monitoring.org
The pkg-stats script queries release-monitoring.org to find the latest
upstream versions of our packages. However, up until recently,
release-monitoring.org had no notion of stable
vs. development/release-candidate versions, so for some packages the
"latest" version was in fact a development/release-candidate version
that we didn't want to package in Buildroot.

However, in recent time, release-monitoring.org has gained support for
differentiating stable vs. development releases of upstream
projects. See for example
https://release-monitoring.org/project/10024/ for the glib library,
which has a number of versions marked "Pre-release".

The JSON blurb returned by release-monitoring.org has 3 relevant
fields:

 - "version", which we are using currently, which is a string
   containing the reference of the latest version, including
   pre-release.

 - "versions", which is an array of strings listing all versions,
   pre-release or not.

 - "stable_versions", which is an array of string listing only
   non-pre-release versions. It is ordered newest first to oldest
   last.

So, this commit changes from using 'version' to using
'stable_versions[0]'.

As an example, before this change, pkg-stats reports that nfs-utils
needs to be bumped to 2.5.5rc3, while after this patch, it reports
that nfs-utils is already at 2.5.4, and that this is the latest stable
version (modulo an issue where Buildroot has 2.5.4 and
release-monitoring.org has 2-5-4, this will be addressed separately).

Note that part of this change was already done in commit f7b0e0860, but
it was incomplete.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-21 21:13:00 +02:00
Thomas Petazzoni
9602fd94e7 support/scripts/pkg-stats: fix the status for packages found by guess
The pkg-stats scripts tries to match packages against
release-monitoring.org in two ways:

- First by using the "Buildroot" distribution registered on
  release-monitoring.org, in which we have added a lot of mappings
  between Buildroot package names and release-monitoring.org package
  names. If there is a match using this distribution, the package
  status is RM_API_STATUS_FOUND_BY_DISTRO, which means that the
  resulting HTML has a "found by distro" statement.

- Then, if the first solution didn't work, by using the pattern
  matching, as done in the check_package_get_latest_version_by_guess()
  function.

However, there is a bug in this later case: it sets the package status
to RM_API_STATUS_FOUND_BY_DISTRO as well, while it should have been
RM_API_STATUS_FOUND_BY_PATTERN. Due to this bug, in the resulting HTML
file from a pkg-stats run, all packages are marked as "found by
distro" even the ones that are "found by guess".

This commit fixes that by setting the correct package status.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-21 21:04:58 +02:00
Francois Perrad
f7b0e08605 support/scripts/pkg-stats: prefers stable version from release-monitoring.org
For example with libpng: 1.6.37 instead of 1.7.0beta89

Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: coalesce into a single line]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-19 21:57:22 +02:00
Arnout Vandecappelle (Essensium/Mind)
d721c95b8b support: fix flake8 error E741 ambiguous variable name
A recent update of flake8 in CI introduced a new check E741. It
basically checks that variables are at least 3 characters long. Up to
now, however, we have used shorter names in some places - all of them
turn out to be "l" for a line of text.

Replace all those "l" variables with "line".

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/scripts/boot-qemu-image.py:47:21: E741 ambiguous variable name 'l'
support/scripts/check-dotconfig.py:20:38: E741 ambiguous variable name 'l'
support/scripts/size-stats:76:13: E741 ambiguous variable name 'l'
support/testing/tests/core/test_bad_arch.py:17:32: E741 ambiguous variable name 'l'
support/testing/tests/package/test_python_treq.py:10:30: E741 ambiguous variable name 'l'
support/testing/tests/toolchain/test_external.py:30:42: E741 ambiguous variable name 'l'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-18 22:31:11 +02:00
Arnout Vandecappelle (Essensium/Mind)
1217817ac2 support/testing: sample_python_dbus_next: ignore F821 flake8 error
The dbus-next package uses the Python type annotation for dbus types. This is
not compatible with the python typing assumption that flake8 makes.

Exclude F821 from this line.

Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1687009829
partially:
support/testing/tests/package/sample_python_dbus_next.py:17:36: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:48: F821 undefined name 's'
support/testing/tests/package/sample_python_dbus_next.py:17:56: F821 undefined name 's'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-18 22:30:44 +02:00
Romain Naour
e11c3cbe26 support/testing: test_ubi: allow to boot several times using the same ubi image.
Since the ubi/ubifs test has been introduced, it's not possible to
boot the same ubi image twice [1]:

"TODO: if you boot Qemu twice on the same UBI image, it fails to
attach the image the second time, with "ubi0 error:
ubi_read_volume_table: the layout volume was not found"."

For some reason, the kernel corrupt the ubi image if the ubifs
rootfs is mounted with write access. Use a custom config file
to mount the rootfs readonly (vol_type=static). Doing so requires
to add the flash size (vol_size=64MiB).

At least it allows to boot several times the same ubi image.

[1] bf4a6490e4

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-17 21:47:04 +02:00
Romain Naour
d8447c38f5 support/testing: update logical eraseblock and physical eraseblock size for qemu >= 2.9
The current ubi/ubifs test (test_ubi.py) rely on a Qemu bug present in
2.8.0 that was fixed in Qemu 2.9.0 [1]. The ubi/ubifs settings is
updated to run with Qemu >= 2.9.0 using the new multiple chip handling.

If needed, the old behavior can be enabled using the pflash01 property
"old-multiple-chip-handling" [2].

The issue was not detected until now since we are sill using an old
qemu (2.8 from Debian stretch) for testing in gitlab (using the
Buildroot Docker image used by gitlab-ci.yml).

First the logical eraseblock size (LEB) must be updated to the value
0x3ff80 reported by the kernel when using qemu >= 2.9.0.

  UBIFS (ubi0:0): Mounting in unauthenticated mode
  UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: LEB size mismatch: 524160 in superblock, 262016 real
  UBIFS error (ubi0:0 pid 1): ubifs_read_superblock: bad superblock, error 1

But the system is still failing to boot:

 UBIFS error (ubi0:0 pid 1): ubifs_scan: garbage
 UBIFS error (ubi0:0 pid 1): ubifs_recover_master_node: failed to recover master node

ubifs is reading garbage since Qemu >= 2.9.0 report a sector
length per device divided by the number of devices (see commit [1]).

The kernel detect two flash devices (dmesg):

  Concatenating MTD devices:
  (0): "40000000.flash"
  (1): "40000000.flash"
  into device "40000000.flash"

Divide the physical eraseblock (PEB) size by two.

Tested with qemu 2.9.0, 5.1.0.

Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1543100932

[1] https://git.qemu.org/?p=qemu.git;a=commitdiff;h=feb0b1aa11f14ee71660aba46b46387d1f923c9e
[2] http://lists.busybox.net/pipermail/buildroot/2021-September/622069.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-17 21:47:02 +02:00
Romain Naour
1ab2dd6aa5 support/testing: test_ubi: add image format on the qemu command line
Adding the Image format on the Qemu command line avoid this warning:

"WARNING: Image format was not specified for 'output/TestUbi/images/rootfs.ubi' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions."

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-17 21:46:59 +02:00
Romain Naour
6f9e83f5f7 support/testing: test_ubi: reduce the rootfs.ubi size to 64M to match the Qemu emulated flash device
The size of the cfi flash device emulated by Qemu is 64M not 128M [1].
Since Qemu >= 4.0, the size of the device must match the size of the block backend [2].

Fixes:

  qemu-system-arm: device requires 67108864 bytes, block backend provides 134217728 bytes

[1] https://git.qemu.org/?p=qemu.git;a=blob;f=hw/arm/vexpress.c;h=58481c07629aedb09864dcc72757ff7947e733bb;hb=f9baca549e44791be0dd98de15add3d8452a8af0#l50
[2] https://git.qemu.org/?p=qemu.git;a=commitdiff;h=06f1521795207359a395996c253c306f4ab7586e

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-17 21:46:57 +02:00
Yann E. MORIN
128af17dce support/testing: fix python_flask_expect_python
Commit e6ee07f41a (package/python-flask-expects-json: new package)
added a non-functional test case that, as noticed by Edgar, fails with:

    AssertionError: '%{http_code}' != '200'

That's because the % sign is self-escaped, à-la C, in the first part
of the command, probably to avoid its being %-formatted. But only the
second part of the command is %-formatted, so we do not need to
self-escape % in the first part.

Additionally, since eb3ee3078a (support/testing/infra/emulator.py:
prevent the commands from wrapping), we no longer need to play tricks
with commands that are too long to fit on the first line of the shell
prompt.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-12 22:51:25 +02:00
Herve Codina
36395e4762 package/lua-augeas: new package
The lua-augeas package provides a Lua binding for augeas

https://github.com/ncopa/lua-augeas

Based on initial work from Nicolas Carrier <nicolas.carrier@orolia.com>

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-12 22:21:12 +02:00
Herve Codina
9fc7b49ab1 package/dtbocfg: new package
dtbocfg, which stands for Device Tree Blob Overlay Configuration
File System, was developed to serve as a userspace API of Device
Tree Overlay.

https://github.com/ikwzm/dtbocfg

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-11 22:50:40 +02:00
Peter Korsgaard
39bcc03870 Update for 2021.08.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 516b837002)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-10-11 22:31:39 +02:00
Thomas Petazzoni
1ea0ae0349 support/testing: add suite of tests for gdb
The list of tests is as follows:

TestGdbHostOnlyDefault: build just minimal host-gdb, default version

TestGdbHostOnlyAllFeatures: build host-gdb, default version, with all
features enabled (TUI, Python, simulator)

TestGdbserverOnly: build just target gdbserver, default version

TestGdbFullTarget: build just target gdb, default version

TestGdbHostOnly9x: build minimal host-gdb, 9.x version

TestGdbHostGdbserver9x: build minimal host-gdb 9.x + gdbserver

TestGdbHostGdbTarget9x: build minimal host-gdb 9.x + full gdb

TestGdbHostOnly11x: build minimal host-gdb, 11.x version

TestGdbHostGdbserver11x: build minimal host-gdb 11.x + gdbserver

TestGdbHostGdbTarget11x: build minimal host-gdb 11.x + gdb

TestGdbArc: build minimal host-gdb + gdb + gdbserver, for the special
ARC architecture version

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-10 19:41:32 +02:00
Edgar Bonet
eb3ee3078a support/testing/infra/emulator.py: prevent the commands from wrapping
Traditional VT-10x terminals (and their emulators) [0] have a "magic
margins" feature that enables the last character position to be updated
without scrolling the screen: whenever a character is printed on the
last column, the cursor stays over the character, instead of moving to
the next line.

The Busybox shell, ash, attempts to defeat this feature by printing
CR,LF right after echoing a character to the last column.[1] This
doesn't play well with emulator.py. The run() method of the Emulator
class captures the output of the emulated system and assumes the first
line it reads is the echo of the command, and all subsequent lines are
the command's output. If the line made by the command + shell prompt is
longer than 80 characters, then it is echoed as two or more lines, and
all but the first one are mistaken for the command's output.

We fix this by telling the emulated system that we are using an
ultra-wide terminal with 29999 columns. Larger values would be ignored
and replaced by the default, namely 80 columns.[2]

[0] https://vt100.net/docs/vt100-ug/chapter3.html  -  DECAWM
[1] https://git.busybox.net/busybox/tree/libbb/lineedit.c?h=1_34_0#n412
[2] https://git.busybox.net/busybox/tree/libbb/xfuncs.c?h=1_34_0#n258

Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Co-authored-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-06 21:54:20 +02:00
Edgar Bonet
324612d68e support/kconfig: fix compiler warnings
Compiling on Ubuntu 20.04 generates this:

./util.c: In function ‘file_write_dep’
./util.c:54:18: warning: ‘..config.tmp’ directive writing 12 bytes into a region of size between 1 and 4097 [-Wformat-overflow=]
   54 |  sprintf(buf, "%s..config.tmp", dir);
      |                  ^~~~~~~~~~~~
./util.c:54:2: note: ‘sprintf’ output between 13 and 4109 bytes into a destination of size 4097
   54 |  sprintf(buf, "%s..config.tmp", dir);
      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and similar warnings on confdata.c, lines 778, 989, 995, 1000, 1007,
1040, 1046 and 1054. Avoid the warnings by enlarging the destination
buffer of fprintf().

Normally, we want changes to kconfig to be reflected by patches in
support/kconfig/patches. This makes it easier to resync with upstream
kconfig. However, in this case, everything that is changed here is
already changed completely (and differently) upstream, so there is no
added value in keeping the patch.

Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-10-06 21:11:09 +02:00
Kory Maincent
c656373aeb support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS
The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
Add dedicated Grub2 builtin config for the EFI compatible cases.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-03 14:45:10 +02:00