c9b07cbdd0
Currently the text for each package infra that mentions the usage of variables already provided by the generic infra diverge from each other: - some (golang, kconfig, python) add a cross-referece to the generic infra chapter; - kconfig does not list any example; - some mention _LICENSE as an example, others don't; - some (cargo, golang, python) add an 'etc.' at the end of the examples, giving the idea that can be more symbols provided by the generic infra than the ones listed; - most have the text 'works by defining a number of variables before calling the +<macro-name>+ macro', except golang and kconfig; - some actually list 'A few additional variables' but keep using some old reference as 'An additional variable'; - some say 'First, all the package metadata' and other only 'All the package metadata'; - most mention _SUBDIR as an example of variable supported by the generic infra, even the generic infra manual not mentioning it. Improve the correctness for the manual by standardizing the text among the package infras: - use the same text "All the package metadata information variables that exist in the generic package infrastructure also exist in the <name> infrastructure:" for all of them; - add the cross-reference for all of them; - remove the examples of variables inherited from the generic infra - this also solves the _SUBDIR problem, there no longer is any reference to _SUBDIR; - wrap the modified text at 80 columns; - add "macro" to golang and luarocks infra; - use "A few additional variables" for qmake and waf. At same time, add a missing format on golang manual for BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS. Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr> Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com> [Arnout: - remove the examples; - add "the" where "macro" was added; - rewrite the preceding paragraphs for kconfig to make it more consistent. ] Signed-off-by: Arnout Vandecappelle <arnout@mind.be> (cherry picked from commit 4286c89f9d987f5f3bcbb14dfd58ba440944f4c2) Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
245 lines
9.4 KiB
Plaintext
245 lines
9.4 KiB
Plaintext
// -*- mode:doc; -*-
|
|
// vim: set syntax=asciidoc:
|
|
|
|
=== Infrastructure for Python packages
|
|
|
|
This infrastructure applies to Python packages that use the standard
|
|
Python setuptools, pep517, flit or maturin mechanisms as their build
|
|
system, generally recognizable by the usage of a +setup.py+ script or
|
|
+pyproject.toml+ file.
|
|
|
|
[[python-package-tutorial]]
|
|
|
|
==== +python-package+ tutorial
|
|
|
|
First, let's see how to write a +.mk+ file for a Python package,
|
|
with an example :
|
|
|
|
------------------------
|
|
01: ################################################################################
|
|
02: #
|
|
03: # python-foo
|
|
04: #
|
|
05: ################################################################################
|
|
06:
|
|
07: PYTHON_FOO_VERSION = 1.0
|
|
08: PYTHON_FOO_SOURCE = python-foo-$(PYTHON_FOO_VERSION).tar.xz
|
|
09: PYTHON_FOO_SITE = http://www.foosoftware.org/download
|
|
10: PYTHON_FOO_LICENSE = BSD-3-Clause
|
|
11: PYTHON_FOO_LICENSE_FILES = LICENSE
|
|
12: PYTHON_FOO_ENV = SOME_VAR=1
|
|
13: PYTHON_FOO_DEPENDENCIES = libmad
|
|
14: PYTHON_FOO_SETUP_TYPE = setuptools
|
|
15:
|
|
16: $(eval $(python-package))
|
|
------------------------
|
|
|
|
On line 7, we declare the version of the package.
|
|
|
|
On line 8 and 9, we declare the name of the tarball (xz-ed tarball
|
|
recommended) and the location of the tarball on the Web. Buildroot
|
|
will automatically download the tarball from this location.
|
|
|
|
On line 10 and 11, we give licensing details about the package (its
|
|
license on line 10, and the file containing the license text on line
|
|
11).
|
|
|
|
On line 12, we tell Buildroot to pass custom options to the Python
|
|
+setup.py+ script when it is configuring the package.
|
|
|
|
On line 13, we declare our dependencies, so that they are built
|
|
before the build process of our package starts.
|
|
|
|
On line 14, we declare the specific Python build system being used. In
|
|
this case the +setuptools+ Python build system is used. The five
|
|
supported ones are +flit+, +pep517+, +setuptools+, +setuptools-rust+
|
|
and +maturin+.
|
|
|
|
Finally, on line 16, we invoke the +python-package+ macro that
|
|
generates all the Makefile rules that actually allow the package to be
|
|
built.
|
|
|
|
[[python-package-reference]]
|
|
|
|
==== +python-package+ reference
|
|
|
|
As a policy, packages that merely provide Python modules should all be
|
|
named +python-<something>+ in Buildroot. Other packages that use the
|
|
Python build system, but are not Python modules, can freely choose
|
|
their name (existing examples in Buildroot are +scons+ and
|
|
+supervisor+).
|
|
|
|
The main macro of the Python package infrastructure is
|
|
+python-package+. It is similar to the +generic-package+ macro. It is
|
|
also possible to create Python host packages with the
|
|
+host-python-package+ macro.
|
|
|
|
Just like the generic infrastructure, the Python infrastructure works
|
|
by defining a number of variables before calling the +python-package+
|
|
or +host-python-package+ macros.
|
|
|
|
All the package metadata information variables that exist in the
|
|
xref:generic-package-reference[generic package infrastructure] also
|
|
exist in the Python infrastructure.
|
|
|
|
Note that:
|
|
|
|
* It is not necessary to add +python+ or +host-python+ in the
|
|
+PYTHON_FOO_DEPENDENCIES+ variable of a package, since these basic
|
|
dependencies are automatically added as needed by the Python
|
|
package infrastructure.
|
|
|
|
* Similarly, it is not needed to add +host-python-setuptools+ to
|
|
+PYTHON_FOO_DEPENDENCIES+ for setuptools-based packages, since it's
|
|
automatically added by the Python infrastructure as needed.
|
|
|
|
One variable specific to the Python infrastructure is mandatory:
|
|
|
|
* +PYTHON_FOO_SETUP_TYPE+, to define which Python build system is used
|
|
by the package. The five supported values are +flit+, +pep517+ and
|
|
+setuptools+, +setuptools-rust+ and +maturin+. If you don't know
|
|
which one is used in your package, look at the +setup.py+ or
|
|
+pyproject.toml+ file in your package source code, and see whether
|
|
it imports things from the +flit+ module or the +setuptools+
|
|
module. If the package is using a +pyproject.toml+ file without any
|
|
build-system requires and with a local in-tree backend-path one
|
|
should use +pep517+.
|
|
|
|
A few additional variables, specific to the Python infrastructure, can
|
|
optionally be defined, depending on the package's needs. Many of them
|
|
are only useful in very specific cases, typical packages will
|
|
therefore only use a few of them, or none.
|
|
|
|
* +PYTHON_FOO_SUBDIR+ may contain the name of a subdirectory inside the
|
|
package that contains the main +setup.py+ or +pyproject.toml+ file.
|
|
This is useful, if for example, the main +setup.py+ or +pyproject.toml+
|
|
file is not at the root of the tree extracted by the tarball. If
|
|
+HOST_PYTHON_FOO_SUBDIR+ is not specified, it defaults to
|
|
+PYTHON_FOO_SUBDIR+.
|
|
|
|
* +PYTHON_FOO_ENV+, to specify additional environment variables to
|
|
pass to the Python +setup.py+ script (for setuptools packages) or
|
|
the +support/scripts/pyinstaller.py+ script (for flit/pep517
|
|
packages) for both the build and install steps. Note that the
|
|
infrastructure is automatically passing several standard variables,
|
|
defined in +PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools target
|
|
packages), +HOST_PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools host
|
|
packages), +PKG_PYTHON_PEP517_ENV+ (for flit/pep517 target packages)
|
|
and +HOST_PKG_PYTHON_PEP517_ENV+ (for flit/pep517 host packages).
|
|
|
|
* +PYTHON_FOO_BUILD_OPTS+, to specify additional options to pass to
|
|
the Python +setup.py+ script during the build step, this generally
|
|
only makes sense to use for setuptools based packages as flit/pep517
|
|
based packages do not pass these options to a +setup.py+ script but
|
|
instead pass them to +support/scripts/pyinstaller.py+.
|
|
|
|
* +PYTHON_FOO_INSTALL_TARGET_OPTS+, +PYTHON_FOO_INSTALL_STAGING_OPTS+,
|
|
+HOST_PYTHON_FOO_INSTALL_OPTS+ to specify additional options to pass
|
|
to the Python +setup.py+ script (for setuptools packages) or
|
|
+support/scripts/pyinstaller.py+ (for flit/pep517 packages) during
|
|
the target installation step, the staging installation step or the
|
|
host installation, respectively.
|
|
|
|
With the Python infrastructure, all the steps required to build and
|
|
install the packages are already defined, and they generally work well
|
|
for most Python-based packages. However, when required, it is still
|
|
possible to customize what is done in any particular step:
|
|
|
|
* By adding a post-operation hook (after extract, patch, configure,
|
|
build or install). See xref:hooks[] for details.
|
|
|
|
* By overriding one of the steps. For example, even if the Python
|
|
infrastructure is used, if the package +.mk+ file defines its own
|
|
+PYTHON_FOO_BUILD_CMDS+ variable, it will be used instead of the
|
|
default Python one. However, using this method should be restricted
|
|
to very specific cases. Do not use it in the general case.
|
|
|
|
[[scanpypi]]
|
|
|
|
==== Generating a +python-package+ from a PyPI repository
|
|
|
|
If the Python package for which you would like to create a Buildroot
|
|
package is available on PyPI, you may want to use the +scanpypi+ tool
|
|
located in +utils/+ to automate the process.
|
|
|
|
You can find the list of existing PyPI packages
|
|
https://pypi.python.org[here].
|
|
|
|
+scanpypi+ requires Python's +setuptools+ package to be installed on
|
|
your host.
|
|
|
|
When at the root of your buildroot directory just do :
|
|
|
|
-----------------------
|
|
utils/scanpypi foo bar -o package
|
|
-----------------------
|
|
|
|
This will generate packages +python-foo+ and +python-bar+ in the package
|
|
folder if they exist on https://pypi.python.org.
|
|
|
|
Find the +external python modules+ menu and insert your package inside.
|
|
Keep in mind that the items inside a menu should be in alphabetical order.
|
|
|
|
Please keep in mind that you'll most likely have to manually check the
|
|
package for any mistakes as there are things that cannot be guessed by
|
|
the generator (e.g. dependencies on any of the python core modules
|
|
such as BR2_PACKAGE_PYTHON_ZLIB). Also, please take note that the
|
|
license and license files are guessed and must be checked. You also
|
|
need to manually add the package to the +package/Config.in+ file.
|
|
|
|
If your Buildroot package is not in the official Buildroot tree but in
|
|
a br2-external tree, use the -o flag as follows:
|
|
|
|
-----------------------
|
|
utils/scanpypi foo bar -o other_package_dir
|
|
-----------------------
|
|
|
|
This will generate packages +python-foo+ and +python-bar+ in the
|
|
+other_package_directory+ instead of +package+.
|
|
|
|
Option +-h+ will list the available options:
|
|
|
|
-----------------------
|
|
utils/scanpypi -h
|
|
-----------------------
|
|
|
|
[[python-package-cffi-backend]]
|
|
|
|
==== +python-package+ CFFI backend
|
|
|
|
C Foreign Function Interface for Python (CFFI) provides a convenient
|
|
and reliable way to call compiled C code from Python using interface
|
|
declarations written in C. Python packages relying on this backend can
|
|
be identified by the appearance of a +cffi+ dependency in the
|
|
+install_requires+ field of their +setup.py+ file.
|
|
|
|
Such a package should:
|
|
|
|
* add +python-cffi+ as a runtime dependency in order to install the
|
|
compiled C library wrapper on the target. This is achieved by adding
|
|
+select BR2_PACKAGE_PYTHON_CFFI+ to the package +Config.in+.
|
|
|
|
------------------------
|
|
config BR2_PACKAGE_PYTHON_FOO
|
|
bool "python-foo"
|
|
select BR2_PACKAGE_PYTHON_CFFI # runtime
|
|
------------------------
|
|
|
|
* add +host-python-cffi+ as a build-time dependency in order to
|
|
cross-compile the C wrapper. This is achieved by adding
|
|
+host-python-cffi+ to the +PYTHON_FOO_DEPENDENCIES+ variable.
|
|
|
|
------------------------
|
|
################################################################################
|
|
#
|
|
# python-foo
|
|
#
|
|
################################################################################
|
|
|
|
...
|
|
|
|
PYTHON_FOO_DEPENDENCIES = host-python-cffi
|
|
|
|
$(eval $(python-package))
|
|
------------------------
|