2013-12-11 21:26:36 +01:00
|
|
|
// -*- mode:doc; -*-
|
|
|
|
|
// vim: set syntax=asciidoc:
|
|
|
|
|
|
manual: use one-line titles instead of two-line titles (trivial)
Asciidoc supports two syntaxes for section titles: two-line titles (title
plus underline consisting of a particular symbol), and one-line titles
(title prefixed with a specific number of = signs).
The two-line title underlines are:
Level 0 (top level): ======================
Level 1: ----------------------
Level 2: ~~~~~~~~~~~~~~~~~~~~~~
Level 3: ^^^^^^^^^^^^^^^^^^^^^^
Level 4 (bottom level): ++++++++++++++++++++++
and the one-line title prefixes:
= Document Title (level 0) =
== Section title (level 1) ==
=== Section title (level 2) ===
==== Section title (level 3) ====
===== Section title (level 4) =====
The buildroot manual is currenly using the two-line titles, but this has
multiple disadvantages:
- asciidoc also uses some of the underline symbols for other purposes (like
preformatted code, example blocks, ...), which makes it difficult to do
mass replacements, such as a planned follow-up patch that needs to move
all sections one level down.
- it is difficult to remember which level a given underline symbol (=-~^+)
corresponds to, while counting = signs is easy.
This patch changes all two-level titles to one-level titles in the manual.
The bulk of the change was done with the following Python script, except for
the level 1 titles (-----) as these underlines are also used for literal
code blocks.
This patch only changes the titles, no other changes. In
adding-packages-directory.txt, I did add missing newlines between some
titles and their content.
----------------------------------------------------------------------------
#!/usr/bin/env python
import sys
import mmap
import re
for input in sys.argv[1:]:
f = open(input, 'r+')
f.flush()
s = mmap.mmap(f.fileno(), 0)
# Level 0 (top level): ====================== =
# Level 1: ---------------------- ==
# Level 2: ~~~~~~~~~~~~~~~~~~~~~~ ===
# Level 3: ^^^^^^^^^^^^^^^^^^^^^^ ====
# Level 4 (bottom level): ++++++++++++++++++++++ =====
def replace_title(s, symbol, replacement):
pattern = re.compile(r'(.+\n)\%s{2,}\n' % symbol, re.MULTILINE)
return pattern.sub(r'%s \1' % replacement, s)
new = s
new = replace_title(new, '=', '=')
new = replace_title(new, '+', '=====')
new = replace_title(new, '^', '====')
new = replace_title(new, '~', '===')
#new = replace_title(new, '-', '==')
s.seek(0)
s.write(new)
s.resize(s.tell())
s.close()
f.close()
----------------------------------------------------------------------------
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-02 07:47:30 +02:00
|
|
|
=== Infrastructure for Python packages
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
This infrastructure applies to Python packages that use the standard
|
2024-02-06 23:19:00 +01:00
|
|
|
Python setuptools, pep517, flit or maturin mechanisms as their build
|
|
|
|
|
system, generally recognizable by the usage of a +setup.py+ script or
|
|
|
|
|
+pyproject.toml+ file.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
[[python-package-tutorial]]
|
|
|
|
|
|
manual: use one-line titles instead of two-line titles (trivial)
Asciidoc supports two syntaxes for section titles: two-line titles (title
plus underline consisting of a particular symbol), and one-line titles
(title prefixed with a specific number of = signs).
The two-line title underlines are:
Level 0 (top level): ======================
Level 1: ----------------------
Level 2: ~~~~~~~~~~~~~~~~~~~~~~
Level 3: ^^^^^^^^^^^^^^^^^^^^^^
Level 4 (bottom level): ++++++++++++++++++++++
and the one-line title prefixes:
= Document Title (level 0) =
== Section title (level 1) ==
=== Section title (level 2) ===
==== Section title (level 3) ====
===== Section title (level 4) =====
The buildroot manual is currenly using the two-line titles, but this has
multiple disadvantages:
- asciidoc also uses some of the underline symbols for other purposes (like
preformatted code, example blocks, ...), which makes it difficult to do
mass replacements, such as a planned follow-up patch that needs to move
all sections one level down.
- it is difficult to remember which level a given underline symbol (=-~^+)
corresponds to, while counting = signs is easy.
This patch changes all two-level titles to one-level titles in the manual.
The bulk of the change was done with the following Python script, except for
the level 1 titles (-----) as these underlines are also used for literal
code blocks.
This patch only changes the titles, no other changes. In
adding-packages-directory.txt, I did add missing newlines between some
titles and their content.
----------------------------------------------------------------------------
#!/usr/bin/env python
import sys
import mmap
import re
for input in sys.argv[1:]:
f = open(input, 'r+')
f.flush()
s = mmap.mmap(f.fileno(), 0)
# Level 0 (top level): ====================== =
# Level 1: ---------------------- ==
# Level 2: ~~~~~~~~~~~~~~~~~~~~~~ ===
# Level 3: ^^^^^^^^^^^^^^^^^^^^^^ ====
# Level 4 (bottom level): ++++++++++++++++++++++ =====
def replace_title(s, symbol, replacement):
pattern = re.compile(r'(.+\n)\%s{2,}\n' % symbol, re.MULTILINE)
return pattern.sub(r'%s \1' % replacement, s)
new = s
new = replace_title(new, '=', '=')
new = replace_title(new, '+', '=====')
new = replace_title(new, '^', '====')
new = replace_title(new, '~', '===')
#new = replace_title(new, '-', '==')
s.seek(0)
s.write(new)
s.resize(s.tell())
s.close()
f.close()
----------------------------------------------------------------------------
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-02 07:47:30 +02:00
|
|
|
==== +python-package+ tutorial
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
First, let's see how to write a +.mk+ file for a Python package,
|
|
|
|
|
with an example :
|
|
|
|
|
|
|
|
|
|
------------------------
|
|
|
|
|
01: ################################################################################
|
|
|
|
|
02: #
|
|
|
|
|
03: # python-foo
|
|
|
|
|
04: #
|
|
|
|
|
05: ################################################################################
|
|
|
|
|
06:
|
|
|
|
|
07: PYTHON_FOO_VERSION = 1.0
|
2014-08-06 04:41:52 +02:00
|
|
|
08: PYTHON_FOO_SOURCE = python-foo-$(PYTHON_FOO_VERSION).tar.xz
|
2013-12-11 21:26:36 +01:00
|
|
|
09: PYTHON_FOO_SITE = http://www.foosoftware.org/download
|
2017-03-30 15:43:30 +02:00
|
|
|
10: PYTHON_FOO_LICENSE = BSD-3-Clause
|
2013-12-11 21:26:36 +01:00
|
|
|
11: PYTHON_FOO_LICENSE_FILES = LICENSE
|
|
|
|
|
12: PYTHON_FOO_ENV = SOME_VAR=1
|
|
|
|
|
13: PYTHON_FOO_DEPENDENCIES = libmad
|
2024-02-06 23:09:28 +01:00
|
|
|
14: PYTHON_FOO_SETUP_TYPE = setuptools
|
2013-12-11 21:26:36 +01:00
|
|
|
15:
|
|
|
|
|
16: $(eval $(python-package))
|
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
|
|
On line 7, we declare the version of the package.
|
|
|
|
|
|
|
|
|
|
On line 8 and 9, we declare the name of the tarball (xz-ed tarball
|
|
|
|
|
recommended) and the location of the tarball on the Web. Buildroot
|
|
|
|
|
will automatically download the tarball from this location.
|
|
|
|
|
|
|
|
|
|
On line 10 and 11, we give licensing details about the package (its
|
|
|
|
|
license on line 10, and the file containing the license text on line
|
|
|
|
|
11).
|
|
|
|
|
|
|
|
|
|
On line 12, we tell Buildroot to pass custom options to the Python
|
|
|
|
|
+setup.py+ script when it is configuring the package.
|
|
|
|
|
|
|
|
|
|
On line 13, we declare our dependencies, so that they are built
|
|
|
|
|
before the build process of our package starts.
|
|
|
|
|
|
|
|
|
|
On line 14, we declare the specific Python build system being used. In
|
2024-02-06 23:19:00 +01:00
|
|
|
this case the +setuptools+ Python build system is used. The five
|
|
|
|
|
supported ones are +flit+, +pep517+, +setuptools+, +setuptools-rust+
|
|
|
|
|
and +maturin+.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
Finally, on line 16, we invoke the +python-package+ macro that
|
|
|
|
|
generates all the Makefile rules that actually allow the package to be
|
|
|
|
|
built.
|
|
|
|
|
|
|
|
|
|
[[python-package-reference]]
|
|
|
|
|
|
manual: use one-line titles instead of two-line titles (trivial)
Asciidoc supports two syntaxes for section titles: two-line titles (title
plus underline consisting of a particular symbol), and one-line titles
(title prefixed with a specific number of = signs).
The two-line title underlines are:
Level 0 (top level): ======================
Level 1: ----------------------
Level 2: ~~~~~~~~~~~~~~~~~~~~~~
Level 3: ^^^^^^^^^^^^^^^^^^^^^^
Level 4 (bottom level): ++++++++++++++++++++++
and the one-line title prefixes:
= Document Title (level 0) =
== Section title (level 1) ==
=== Section title (level 2) ===
==== Section title (level 3) ====
===== Section title (level 4) =====
The buildroot manual is currenly using the two-line titles, but this has
multiple disadvantages:
- asciidoc also uses some of the underline symbols for other purposes (like
preformatted code, example blocks, ...), which makes it difficult to do
mass replacements, such as a planned follow-up patch that needs to move
all sections one level down.
- it is difficult to remember which level a given underline symbol (=-~^+)
corresponds to, while counting = signs is easy.
This patch changes all two-level titles to one-level titles in the manual.
The bulk of the change was done with the following Python script, except for
the level 1 titles (-----) as these underlines are also used for literal
code blocks.
This patch only changes the titles, no other changes. In
adding-packages-directory.txt, I did add missing newlines between some
titles and their content.
----------------------------------------------------------------------------
#!/usr/bin/env python
import sys
import mmap
import re
for input in sys.argv[1:]:
f = open(input, 'r+')
f.flush()
s = mmap.mmap(f.fileno(), 0)
# Level 0 (top level): ====================== =
# Level 1: ---------------------- ==
# Level 2: ~~~~~~~~~~~~~~~~~~~~~~ ===
# Level 3: ^^^^^^^^^^^^^^^^^^^^^^ ====
# Level 4 (bottom level): ++++++++++++++++++++++ =====
def replace_title(s, symbol, replacement):
pattern = re.compile(r'(.+\n)\%s{2,}\n' % symbol, re.MULTILINE)
return pattern.sub(r'%s \1' % replacement, s)
new = s
new = replace_title(new, '=', '=')
new = replace_title(new, '+', '=====')
new = replace_title(new, '^', '====')
new = replace_title(new, '~', '===')
#new = replace_title(new, '-', '==')
s.seek(0)
s.write(new)
s.resize(s.tell())
s.close()
f.close()
----------------------------------------------------------------------------
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-05-02 07:47:30 +02:00
|
|
|
==== +python-package+ reference
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
As a policy, packages that merely provide Python modules should all be
|
|
|
|
|
named +python-<something>+ in Buildroot. Other packages that use the
|
|
|
|
|
Python build system, but are not Python modules, can freely choose
|
|
|
|
|
their name (existing examples in Buildroot are +scons+ and
|
|
|
|
|
+supervisor+).
|
|
|
|
|
|
|
|
|
|
The main macro of the Python package infrastructure is
|
|
|
|
|
+python-package+. It is similar to the +generic-package+ macro. It is
|
|
|
|
|
also possible to create Python host packages with the
|
|
|
|
|
+host-python-package+ macro.
|
|
|
|
|
|
|
|
|
|
Just like the generic infrastructure, the Python infrastructure works
|
|
|
|
|
by defining a number of variables before calling the +python-package+
|
|
|
|
|
or +host-python-package+ macros.
|
|
|
|
|
|
|
|
|
|
All the package metadata information variables that exist in the
|
|
|
|
|
xref:generic-package-reference[generic package infrastructure] also
|
|
|
|
|
exist in the Python infrastructure: +PYTHON_FOO_VERSION+,
|
|
|
|
|
+PYTHON_FOO_SOURCE+, +PYTHON_FOO_PATCH+, +PYTHON_FOO_SITE+,
|
|
|
|
|
+PYTHON_FOO_SUBDIR+, +PYTHON_FOO_DEPENDENCIES+, +PYTHON_FOO_LICENSE+,
|
2014-07-16 22:23:59 +02:00
|
|
|
+PYTHON_FOO_LICENSE_FILES+, +PYTHON_FOO_INSTALL_STAGING+, etc.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
Note that:
|
|
|
|
|
|
|
|
|
|
* It is not necessary to add +python+ or +host-python+ in the
|
|
|
|
|
+PYTHON_FOO_DEPENDENCIES+ variable of a package, since these basic
|
|
|
|
|
dependencies are automatically added as needed by the Python
|
|
|
|
|
package infrastructure.
|
|
|
|
|
|
2024-03-29 17:03:58 +01:00
|
|
|
* Similarly, it is not needed to add +host-python-setuptools+ to
|
2020-04-28 13:47:15 +02:00
|
|
|
+PYTHON_FOO_DEPENDENCIES+ for setuptools-based packages, since it's
|
|
|
|
|
automatically added by the Python infrastructure as needed.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
One variable specific to the Python infrastructure is mandatory:
|
|
|
|
|
|
|
|
|
|
* +PYTHON_FOO_SETUP_TYPE+, to define which Python build system is used
|
2024-02-06 23:19:00 +01:00
|
|
|
by the package. The five supported values are +flit+, +pep517+ and
|
|
|
|
|
+setuptools+, +setuptools-rust+ and +maturin+. If you don't know
|
|
|
|
|
which one is used in your package, look at the +setup.py+ or
|
|
|
|
|
+pyproject.toml+ file in your package source code, and see whether
|
|
|
|
|
it imports things from the +flit+ module or the +setuptools+
|
|
|
|
|
module. If the package is using a +pyproject.toml+ file without any
|
|
|
|
|
build-system requires and with a local in-tree backend-path one
|
|
|
|
|
should use +pep517+.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
A few additional variables, specific to the Python infrastructure, can
|
|
|
|
|
optionally be defined, depending on the package's needs. Many of them
|
|
|
|
|
are only useful in very specific cases, typical packages will
|
|
|
|
|
therefore only use a few of them, or none.
|
|
|
|
|
|
2018-10-11 11:57:54 +02:00
|
|
|
* +PYTHON_FOO_SUBDIR+ may contain the name of a subdirectory inside the
|
2022-05-04 09:07:00 +02:00
|
|
|
package that contains the main +setup.py+ or +pyproject.toml+ file.
|
|
|
|
|
This is useful, if for example, the main +setup.py+ or +pyproject.toml+
|
|
|
|
|
file is not at the root of the tree extracted by the tarball. If
|
|
|
|
|
+HOST_PYTHON_FOO_SUBDIR+ is not specified, it defaults to
|
|
|
|
|
+PYTHON_FOO_SUBDIR+.
|
2018-10-11 11:57:54 +02:00
|
|
|
|
2013-12-11 21:26:36 +01:00
|
|
|
* +PYTHON_FOO_ENV+, to specify additional environment variables to
|
2024-02-06 23:09:28 +01:00
|
|
|
pass to the Python +setup.py+ script (for setuptools packages) or
|
|
|
|
|
the +support/scripts/pyinstaller.py+ script (for flit/pep517
|
|
|
|
|
packages) for both the build and install steps. Note that the
|
|
|
|
|
infrastructure is automatically passing several standard variables,
|
|
|
|
|
defined in +PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools target
|
2022-05-04 09:07:00 +02:00
|
|
|
packages), +HOST_PKG_PYTHON_SETUPTOOLS_ENV+ (for setuptools host
|
|
|
|
|
packages), +PKG_PYTHON_PEP517_ENV+ (for flit/pep517 target packages)
|
|
|
|
|
and +HOST_PKG_PYTHON_PEP517_ENV+ (for flit/pep517 host packages).
|
2013-12-11 21:26:36 +01:00
|
|
|
|
2024-02-06 23:09:28 +01:00
|
|
|
* +PYTHON_FOO_BUILD_OPTS+, to specify additional options to pass to
|
|
|
|
|
the Python +setup.py+ script during the build step, this generally
|
|
|
|
|
only makes sense to use for setuptools based packages as flit/pep517
|
|
|
|
|
based packages do not pass these options to a +setup.py+ script but
|
|
|
|
|
instead pass them to +support/scripts/pyinstaller.py+.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
2014-09-27 21:32:41 +02:00
|
|
|
* +PYTHON_FOO_INSTALL_TARGET_OPTS+, +PYTHON_FOO_INSTALL_STAGING_OPTS+,
|
2014-09-27 21:32:39 +02:00
|
|
|
+HOST_PYTHON_FOO_INSTALL_OPTS+ to specify additional options to pass
|
2024-02-06 23:09:28 +01:00
|
|
|
to the Python +setup.py+ script (for setuptools packages) or
|
|
|
|
|
+support/scripts/pyinstaller.py+ (for flit/pep517 packages) during
|
2022-05-04 09:07:00 +02:00
|
|
|
the target installation step, the staging installation step or the
|
2023-10-01 00:16:34 +02:00
|
|
|
host installation, respectively.
|
2013-12-11 21:26:36 +01:00
|
|
|
|
|
|
|
|
With the Python infrastructure, all the steps required to build and
|
|
|
|
|
install the packages are already defined, and they generally work well
|
|
|
|
|
for most Python-based packages. However, when required, it is still
|
|
|
|
|
possible to customize what is done in any particular step:
|
|
|
|
|
|
|
|
|
|
* By adding a post-operation hook (after extract, patch, configure,
|
|
|
|
|
build or install). See xref:hooks[] for details.
|
|
|
|
|
|
|
|
|
|
* By overriding one of the steps. For example, even if the Python
|
|
|
|
|
infrastructure is used, if the package +.mk+ file defines its own
|
|
|
|
|
+PYTHON_FOO_BUILD_CMDS+ variable, it will be used instead of the
|
|
|
|
|
default Python one. However, using this method should be restricted
|
|
|
|
|
to very specific cases. Do not use it in the general case.
|
2016-02-22 10:20:59 +01:00
|
|
|
|
2016-05-31 17:03:41 +02:00
|
|
|
[[scanpypi]]
|
|
|
|
|
|
|
|
|
|
==== Generating a +python-package+ from a PyPI repository
|
|
|
|
|
|
|
|
|
|
If the Python package for which you would like to create a Buildroot
|
|
|
|
|
package is available on PyPI, you may want to use the +scanpypi+ tool
|
2017-07-01 18:40:40 +02:00
|
|
|
located in +utils/+ to automate the process.
|
2016-05-31 17:03:41 +02:00
|
|
|
|
|
|
|
|
You can find the list of existing PyPI packages
|
|
|
|
|
https://pypi.python.org[here].
|
|
|
|
|
|
|
|
|
|
+scanpypi+ requires Python's +setuptools+ package to be installed on
|
|
|
|
|
your host.
|
|
|
|
|
|
|
|
|
|
When at the root of your buildroot directory just do :
|
|
|
|
|
|
|
|
|
|
-----------------------
|
2017-07-01 18:40:40 +02:00
|
|
|
utils/scanpypi foo bar -o package
|
2016-05-31 17:03:41 +02:00
|
|
|
-----------------------
|
|
|
|
|
|
|
|
|
|
This will generate packages +python-foo+ and +python-bar+ in the package
|
|
|
|
|
folder if they exist on https://pypi.python.org.
|
|
|
|
|
|
|
|
|
|
Find the +external python modules+ menu and insert your package inside.
|
|
|
|
|
Keep in mind that the items inside a menu should be in alphabetical order.
|
|
|
|
|
|
|
|
|
|
Please keep in mind that you'll most likely have to manually check the
|
|
|
|
|
package for any mistakes as there are things that cannot be guessed by
|
|
|
|
|
the generator (e.g. dependencies on any of the python core modules
|
|
|
|
|
such as BR2_PACKAGE_PYTHON_ZLIB). Also, please take note that the
|
|
|
|
|
license and license files are guessed and must be checked. You also
|
|
|
|
|
need to manually add the package to the +package/Config.in+ file.
|
|
|
|
|
|
|
|
|
|
If your Buildroot package is not in the official Buildroot tree but in
|
2016-10-14 16:39:18 +02:00
|
|
|
a br2-external tree, use the -o flag as follows:
|
2016-05-31 17:03:41 +02:00
|
|
|
|
|
|
|
|
-----------------------
|
2017-07-01 18:40:40 +02:00
|
|
|
utils/scanpypi foo bar -o other_package_dir
|
2016-05-31 17:03:41 +02:00
|
|
|
-----------------------
|
|
|
|
|
|
|
|
|
|
This will generate packages +python-foo+ and +python-bar+ in the
|
|
|
|
|
+other_package_directory+ instead of +package+.
|
|
|
|
|
|
|
|
|
|
Option +-h+ will list the available options:
|
|
|
|
|
|
|
|
|
|
-----------------------
|
2017-07-01 18:40:40 +02:00
|
|
|
utils/scanpypi -h
|
2016-05-31 17:03:41 +02:00
|
|
|
-----------------------
|
|
|
|
|
|
2016-02-22 10:20:59 +01:00
|
|
|
[[python-package-cffi-backend]]
|
|
|
|
|
|
|
|
|
|
==== +python-package+ CFFI backend
|
|
|
|
|
|
|
|
|
|
C Foreign Function Interface for Python (CFFI) provides a convenient
|
|
|
|
|
and reliable way to call compiled C code from Python using interface
|
|
|
|
|
declarations written in C. Python packages relying on this backend can
|
|
|
|
|
be identified by the appearance of a +cffi+ dependency in the
|
|
|
|
|
+install_requires+ field of their +setup.py+ file.
|
|
|
|
|
|
|
|
|
|
Such a package should:
|
|
|
|
|
|
|
|
|
|
* add +python-cffi+ as a runtime dependency in order to install the
|
|
|
|
|
compiled C library wrapper on the target. This is achieved by adding
|
|
|
|
|
+select BR2_PACKAGE_PYTHON_CFFI+ to the package +Config.in+.
|
|
|
|
|
|
|
|
|
|
------------------------
|
|
|
|
|
config BR2_PACKAGE_PYTHON_FOO
|
|
|
|
|
bool "python-foo"
|
|
|
|
|
select BR2_PACKAGE_PYTHON_CFFI # runtime
|
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
|
|
* add +host-python-cffi+ as a build-time dependency in order to
|
|
|
|
|
cross-compile the C wrapper. This is achieved by adding
|
|
|
|
|
+host-python-cffi+ to the +PYTHON_FOO_DEPENDENCIES+ variable.
|
|
|
|
|
|
|
|
|
|
------------------------
|
|
|
|
|
################################################################################
|
|
|
|
|
#
|
|
|
|
|
# python-foo
|
|
|
|
|
#
|
|
|
|
|
################################################################################
|
|
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
|
|
|
|
PYTHON_FOO_DEPENDENCIES = host-python-cffi
|
|
|
|
|
|
|
|
|
|
$(eval $(python-package))
|
|
|
|
|
------------------------
|
