The cross-compilation improvements integrated in Python rely on the
compiler exposing a line starting with LIBRARY_PATH when called with
-E -v. This is used by Python setup.py to find the installation
locations of libraries.
However, this LIBRARY_PATH line is not shown by very old compilers,
such as the gcc 4.2.x compiler used on the AVR32 architecture. This
causes libraries installed in the sysroot, such as libffi, to not be
detected by the setup.py script.
To fix this problem, this patch adds addtional logic to setup.py,
which consists in deriving the library paths from the sysroot
location, if no LIBRARY_PATH field was found.
Fixes:
http://autobuild.buildroot.org/results/1aa/1aad2d677dbf7b0a3cb0120a35f45123124f51ab/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When no python interpreter is selected, all host-python symlink
installation were disabled.
This could lead to a non-existing $(HOST_DIR)/usr/bin/python program.
Fixes:
http://autobuild.buildroot.net/results/6e7/6e74a343b4e8e113fd34d45c9279419e233afe8a/
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This change adds a patch to python disabling the installation of the
python and python-config symlinks.
This allows Buildroot to control these symlinks' installation:
* the python symlink should be unconditionally installed in the target
tree, and the python-config symlink in the staging tree, since it is
only built and installed in the target tree if the user selected it;
* the python and python-config symlinks should only be installed in
the host tree when python(2) is the selection of the user for the
target.
Otherwise, when python3 is selected for the target, the host-python
may be required to built some packages. In such cases, the python
symlink should points to python3 (so should the python-config
symlink) to reflect the staging/target tree.
[Thomas: fix comments according to Yann's suggestions, and replaced
python(2) by python2, as suggested by Yann.]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During installation, host-python and host-python3 run the freshly built
python executable. This is done with a proper LD_LIBRARY_PATH to make
sure it picks up the libpython in the build directory. However, the
python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
Therefore, if libpython exists there, it will be used instead.
If the install step is run in parallel, it is possible that libpython
is already partially copied to $(HOST_DIR)/usr/lib when python is run.
This gives an error like:
python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short
The fix is simple: use RUNPATH instead of RPATH, which allows
LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
build directory is always used. RUNPATH is enabled by passing
--enable-new-dtags to the linker.
Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The original version uses sysconfig from the python interpreter
(normally host-python) which isn't really suited to provide information
of the target python.
Use the pure shell preprocessed backport from the upcoming python 3.4
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A confusion was made when applying commit
c3d539b53f ('host python/python3:
conditionally disable unicodedata') and then commit
73293e88c8 ('python, python3: enable
unicodedata for host-python, needed by setuptools').
The first commit added the enabling of unicodedata in the host Python
when the target Python is configured with unicodedata, which is
necessary because building the target unicodedata Python module
requires having unicodedata support in the host Python.
The second commit enabled the unicodedata unconditionally in the host
Python, because it is needed by host-setuptools.
However, when the second commit was applied, the logic that consisted
in disabling unicodedate in the host Python when not needed was kept,
which resulted in breaking the build of host-setuptools due to the
lack of unicodedata support in the host python.
This commit fixes that by only leaving the unconditional
--enable-unicodedata in the host Python configuration options, and
therefore removing the problematic conditional --disable-unicodedata.
Fixes:
http://autobuild.buildroot.org/results/b62/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The python and python3 builds mark libpython as read-only which
prevents it from being stripped out correctly for the target.
Signed-off-by: Przemyslaw Wrzos <przemyslaw.wrzos@calyptech.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
As we are going to bump setuptools to a much newer version, the host
python needs to be built with support for unicodedata.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The host python always had --disable-unicodedata, regardless of the
corresponding configuration option BR2_PACKAGE_PYTHON_UNICODEDATA.
Since the host python is used to byte-compile python modules, this meant
that such modules could not contain unicode strings. For example, following
statement in a python module:
print u"\N{SOLIDUS}"
would cause the byte-compilation to fail with message:
SyntaxError: ("(unicode error) \\N escapes not supported (can't load
unicodedata module)",
Instead, conditionally disable unicodedata based on
BR2_PACKAGE_PYTHON_UNICODEDATA, also for the host python.
This fixes bug #6542 (https://bugs.busybox.net/show_bug.cgi?id=6542)
Reported-by: Gernot Vormayr <gvormayr@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Even though jumping from 2.7.3 to 2.7.6 looks like a minor version
bump, it is in fact a fairly significant one, because a good number of
changes to help cross-compilation have been merged into Python
upstream. Therefore, most of our patches are affected by this change.
In detail, this commit:
* Renames all the patches to follow the naming convention of patches
in Buildroot: the patch file names should not have any version
number.
* The patches numbered above 100, that add configuration options to
disable certain modules of the Python standard library, are only
renamed and slightly adapted, they didn't change that much.
* The patches numbered below 100 are almost entirely rewritten: many
of the cross-compilation problems that used to exist in Python
2.7.3 no longer exist, and the number of remaining problems is
smaller, and can be fixed with smaller patches.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
As a preparation to make the Python infrastructure support both Python
and Python 3, as well as the bump of Python 2 and 3, we need the
Python package to expose the Python module path in a variable called
PYTHON_PATH. It will be used by the following commits.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch is based on the original new pkg patch submitted last Jan
and is part of the "Patchwork oldest patches cleanup #5".
[Peter: fix CONF_OPT indentation]
Signed-off-by: Matt Weber <mlweber1@rockwellcollins.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit improves the cross-compilation patches we have on top of
Python, to fix the problem of host library paths leaking into the
build of target modules, as seen at:
http://autobuild.buildroot.org/results/fcc/fccd7e08cd9d4713eb4208097dd48c5ab25749bc/build-end.loghttp://autobuild.buildroot.org/results/0bd/0bda780bf4b759b12edec26ac20b88cde617db4d/build-end.log
To do so, it ensures that the right python2.7/config/Makefile is used
when building target modules, and adjusts at runtime the paths read
from this Makefile if we are cross-compiling.
In addition, it installs the pgen program into the host directory, and
points the target python build to use python and pgen from $(HOST_DIR)
instead of from the host python source directory, which looks cleaner.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a package A depends on config option B and toolchain option C, then
the comment that is given when C is not fulfilled should also depend on B.
For example:
config BR2_PACKAGE_A
depends on BR2_B
depends on BR2_LARGEFILE
depends on BR2_WCHAR
comment "A needs a toolchain w/ largefile, wchar"
depends on !BR2_LARGEFILE || !BR2_WCHAR
This comment should actually be:
comment "A needs a toolchain w/ largefile, wchar"
depends on BR2_B
depends on !BR2_LARGEFILE || !BR2_WCHAR
or if possible (typically when B is a package config option declared in that
same Config.in file):
if BR2_B
comment "A needs a toolchain w/ largefile, wchar"
depends on !BR2_LARGEFILE || !BR2_WCHAR
[other config options depending on B]
endif
Otherwise, the comment would be visible even though the other dependencies
are not met.
This patch adds such missing dependencies, and changes existing such
dependencies from
depends on BR2_BASE_DEP && !BR2_TOOLCHAIN_USES_GLIBC
to
depends on BR2_BASE_DEP
depends on !BR2_TOOLCHAIN_USES_GLIBC
so that (positive) base dependencies are separate from the (negative)
toolchain dependencies. This strategy makes it easier to write such comments
(because one can simply copy the base dependency from the actual package
config option), but also avoids complex and long boolean expressions.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(untested)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch lines up the comments in Config.in files that clarify which
toolchain options the package depends on.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since 97c687000 (pkg-autotools.mk: default host AUTORECONF{,_OPT} to the
target values) we automatically enable autoreconf for host builds if it
is enabled for the target, so these can go.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
[Peter: leave change xz tarball format to not end up with circular deps]
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Also move smtpd.py removal to the global remove useless files define.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This finally removes the BR2_HAVE_DEVFILES option, that was used to
install/keep development files on target. With the recent migration of
the internal backend to the package infrastructure, we had anyway lost
the ability to build gcc for the target, and install the uClibc
development files on the target.
[Peter: also remove support/scripts/copy.sh]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
We already remove python2.7-config and the symbolic link
python-config, but we forgot to remove the python2-config symbolic
link.
Note that we can't use the <pkg>_CONFIG_SCRIPTS mechanism here because
python2.7-config is written in... Python, and doesn't follow the usual
syntax of <pkg>-config scripts. It takes the paths directly from
distutils.sysconfig.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
It's mostly sample code, normally not used, and has a bad shebang line.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Building host-python in parallel sometimes causes "Bus error" during
the installation step on our autobuilders, such as:
http://autobuild.buildroot.org/results/04bcc907c5e075fe1f39d4f49dcc50ec93708eb4/build-end.log
Extensive testing on one autobuilder has shown that building
host-python with MAKE1 work arounds this strange problem.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Pyhton 2.7.3 includes several security fixes.
See: http://www.python.org/download/releases/2.7.3/
Also fixes the patch making sqlite optional and remove the symlink patch
(which has been fixed upstream).
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Commit 3c90f75496 made Python use a
special ./configure command in order to avoid --enable-shared
--disable-static being passed, because it was causing issues when
building certain modules for a 64 bits system.
However, not having a shared libpython2.7 library for the host
prevents the libxml2 Python binding to get built.
So instead, we use the default configure command, but we add
--enable-static which is needed for Python to build correctly.
Note that we tested the build of Python on a 64 bits host as well as
the build of Python for a 64 bits target, and both went fine, with all
modules built properly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Added patch to disable buggy_getaddrinfo test during configure when
cross-compiling.
[Peter: Remove --enable-ipv6 which is now handled globally]
Signed-off-by: Vanya Sergeev <vsergeev at gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
distutils adds -L$LIBDIR (/usr/lib), breaking build of binary extensions.
Seen with netifaces, but other extensions may be affected as well.
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
zlib is needed for the host-setuptools package
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Thanks to the pkgparentdir and pkgname functions, we can rewrite the
AUTOTARGETS macro in a way that avoids the need for each package to
repeat its name and the directory in which it is present.
[Peter: pkgdir->pkgparentdir]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This is mostly a mechanical bump, with a refresh of all the patches to
accomodate the offsets, and some minor conflict resolution.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
The removal of -L flags from TARGET_LDFLAGS in
7e3e8ec040 has trigerred some more
issues with Python, requiring some more hacky fixes.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Commit 009d8fceab introduced
--enable-shared --disable-static options for the host autotools packages,
ultimately causing a regression on the host-python build, leading to
a number of critical modules not being built on the target python on
64 bits system. Introduce a quick fix for the release and before a deeper
fix.
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
When compiling Python on a host running Linux 3.0, the sys.platform
constant is set at "linux3". A lot of code (inside the interpreter
itself, its build system and third party apps and libraries) relies on
it to be linux2 on a Linux system.
This leads to the build of the target python package to break.
This behaviour has been reported on the Python bugtracker but is not
fixed yet.
http://bugs.python.org/issue12326
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Closes#3169
Typo in modules_lib_dirs section of patch specified modules_include_dirs
instead of modules_lib_dirs. This matters if PYTHON_MODULES_LIB
is not passed into the script.
Signed-off-by: Mark Wickham <markw@digi.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Some packages (like libxcb) need xml support in host-python in order to
build (.py file tries to import xml.etree.cElementTree).
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This commit does a number of changes and improvements to the Python
interpreter package :
* It converts the .mk file to the AUTOTARGETS infrastructure. Even
though Python uses only autoconf and not automake, the AUTOTARGETS
is a fairly good fit for the Python interpreter, so we make use of
it.
* It bumps the version to 2.7.1. As this is a minor release compared
to 2.7, there are no particular changes needed because of this
bump. All changes done to the package are cleanups and improvements
unrelated to the version bump.
* It uses the system libffi. Until now, Python was building its own
libffi (a library used by interprets to build code that makes
function call at runtime). Using the Python internal libffi was not
working as Python was not passing the appropriate arguments down to
libffi ./configure script. And it sounded better to use a
system-wide libffi, that could potentially be used by other
packages as well. This libffi is needed for the ctypes Python
module.
* Remove all "depends on BR2_PACKAGE_PYTHON" by moving all
Python-related options under a "if BR2_PACKAGE_PYTHON ... endif"
condition.
* Make the installation of pre-compiled Python modules (.pyc) the
default, since they are smaller and do not need to be compiled on
the target. It is still possible to install uncompiled modules, or
both the uncompiled and pre-compiled versions.
* The options to select the set of Python modules to compile has been
moved to a submenu.
* The codecscjk (Japanese, Korean and Chinese codecs) module is no
longer enabled by default.
* The commented options for gdbm and nis in Python have been
removed. Those were not supported, so let's get rid of unused code.
* The option for the tkinker module in Python has been removed, since
we don't have a package for Tk in Buildroot.
* Options for the bzip2, sqlite and zlib modules have been added,
since those modules have external dependencies.
* The set of patches has been completely reworked and extended, with
more fine-grained patches and newer functionalities. The patches
are split in two categories:
- Patches that make various modifications to the Python build
system to support cross-compilation or make some minor
modifications. Those patches are numbered from 0 to 100.
- Patches that add configuration options to the Python build
system in order to enable/disable the compilation of Python
extensions or modules (test modules, pydoc, lib2to3, sqlite, tk,
curses, expat, codecs-cjk, nis, unicodedata, database modules,
ssl, bzip2, zlib). These patches are numbered from 100 to 200.
All features of the previous four patches are preserved, but they
are organized differently and the patches have been renamed. This
makes it difficult to see the differences from the existing
patches.
* The host Python interpreter is now installed in $(HOST_DIR), since
it is used to build third party Python modules.
* The BR2_PACKAGE_PYTHON_DEV option is removed since
BR2_HAVE_DEVFILES already does the necessary work.
* The "make -i install" workaround introduced by Maxime Ripard is no
longer needed. It was caused by the compilation of the tests that
required the unicodedata module (which wasn't built in the host
Python interpreter). Since we no longer compile the Python tests,
the problem doesn't exist anymore and we can avoid this "-i"
option.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that TARGET_CC contains several space-separated words, it must be
used quoted everywhere.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
