Commit Graph

69 Commits

Author SHA1 Message Date
Yegor Yefremov
549bbba67f python/python3: globalize *.pyc files compilation
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.

To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.

The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.

The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.

A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
 - rework for python 3.5
 - integrate Samuel proposal that allows to detect compilation
   failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-26 22:07:35 +02:00
Thomas Petazzoni
f06f58f8fd python/python3: use --no-run-if-empty xargs option
As suggested by Samuel Martin, this commit adds the option
--no-run-if-empty xargs option to the "find ... | xargs ..." logic used
in the python and python3 target-finalize hooks to remove py/pyc/pyo
files. This ensures that the command doesn't fail if there are no files
matching the pattern.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-26 22:07:11 +02:00
Thomas Petazzoni
476f5fc8f6 python3: bump to 3.5.1
The major changes in terms of Buildroot packaging are:

 - Due to PEP488, Python no longer generates .pyc (unoptimized) and
   .pyo (optimized) byte-code files. Instead, it generates <foo>.pyc,
   <foo>.opt-1.pyc and <foo>.opt-2.pyc. Therefore, we removed the
   --disable-pyo-build option and kept only the --disable-pyc-build
   option, which completely disables building all .pyc files. In
   addition, since the optimized .opt-X.pyc files don't work if the
   corresponding un-optimized .pyc file is not present, we are for the
   moment unconditionally removing the optimized ones (keeping both
   the unoptimized and optimized ones doubles the required filesystem
   size!). So basically we preserve the behavior we had before this
   commit:

     BR2_PACKAGE_PYTHON3_PY_ONLY -> only *.py
     BR2_PACKAGE_PYTHON3_PYC_ONLY -> only non-optimized *.pyc
     BR2_PACKAGE_PYTHON3_PY_PYC -> both the *.py and non-optimized *.pyc

   To achieve this, the TARGET_FINALIZE_HOOKS are reworked:

    PYTHON3_REMOVE_PY_FILES is responsible for removing *.py files in
    the BR2_PACKAGE_PYTHON3_PYC_ONLY case.

    PYTHON3_REMOVE_PYC_FILES is responsible for removing *.pyc files
    in the BR2_PACKAGE_PYTHON3_PY_ONLY case.

    PYTHON3_REMOVE_OPTIMIZED_PYC_FILES is responsible for removing the
    optimized *.opt-1.pyc and *.opt-2.pyc files, which is done
    unconditionally.

 - The PEP3147 disabling patch had to be significantly reworked due to
   the code having changed heavily. The code was moved into a
   _bootstrap_external.py, which is a "frozen" Python module, i.e a
   module generated into a .h file at compile time using the
   _freeze_importlib program.

 - Due to the above, we now need to regenerate importlib.h at build
   time. Unfortunately, for the target Python _freeze_importlib is
   built for the target, so we can't run it on the build machine. To
   fix this, we copy the _freeze_importlib program from the
   host-python in $(HOST_DIR), and then patch the target python to use
   it. Since the same solution can be used for 'pgen', we do it, and
   avoid having to touch the graminit.{c,h} files.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-17 22:46:17 +02:00
Thomas Petazzoni
5c63280caf python3: switch to Git formatted patches
In preparation for the bump to Python 3.5.0, let's switch all the
patches to the Git format. This way, a Git repository of the Python
source code can be used to manage those patches, which makes it easier
to bump to newer Python versions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-01-05 00:14:15 +01:00
Christophe Vu-Brugier
c3d65a4dad python3: add patch to fix PEP 3147 issue with automake built packages
Packages built with automake use a `py-compile` helper to byte-compile
Python source files. This script uses the "py_compile" module from the
standard library. In turn, the compile() function in the "py_compile"
module invokes the cache_from_source() function provided by importlib.

This commit adds a new patch named "020-importlib-no-pep3147.patch"
that changes cache_from_source() and source_from_cache() in importlib
to get rid of the "__pycache__" directory.

This commit fixes the following import error in kmod when the module
is built for Python 3:

  >>> from kmod import Kmod
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ImportError: cannot import name 'Kmod'

Moreover, this commit removes two patches that are no longer necessary
since modifying cache_from_source() and source_from_cache() disables
PEP 3147 for the standard library and distutils / setuptools.

 * 004-old-stdlib-cache.patch
 * 016-distutils-no-pep3147.patch

Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-11-17 23:13:28 +01:00
Thomas Petazzoni
289b79b65f Revert "python3: add patch to fix PEP 3147 issue with automake built packages"
This reverts commit b586853559, which
causes some build failures of Python 3:

  http://autobuild.buildroot.org/results/923/923c4242dbd6d277c1d88f663b5916ceced985d4/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-11-05 10:10:59 +01:00
Christophe Vu-Brugier
b586853559 python3: add patch to fix PEP 3147 issue with automake built packages
Packages built with automake use a `py-compile` helper to byte-compile
Python source files. This script uses the "py_compile" module from the
standard library. In turn, the compile() function in the "py_compile"
module invokes the cache_from_source() function provided by importlib.

This commit adds a new patch named "020-importlib-no-pep3147.patch"
that changes cache_from_source() and source_from_cache() in importlib
to get rid of the "__pycache__" directory.

This commit fixes the following import error in kmod when the module
is built for Python 3:

  >>> from kmod import Kmod
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ImportError: cannot import name 'Kmod'

Moreover, this commit removes two patches that are no longer necessary
since modifying cache_from_source() and source_from_cache() disables
PEP 3147 for the standard library and distutils / setuptools.

 * 004-old-stdlib-cache.patch
 * 016-distutils-no-pep3147.patch

Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-11-04 22:55:28 +01:00
Thomas Petazzoni
4433c7d647 python3: add patch to fix logic to get gcc paths
The Python setup.py has a function called add_gcc_paths(), which
executes gcc -E -v to get the list of header paths searched by
gcc. However, the logic of setup.py is only valid with the normal
english output of gcc: it doesn't work if a non-english locale is
set. This causes setup.py to not find certain headers (such as zlib.h)
and therefore disabling the build of such extensions.

Reported-by: Bruno Coudoin <bruno.coudoin@gcompris.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-05-31 21:45:04 +02:00
Peter Korsgaard
54915af7c7 python3: needs dynamic library support
Fixes:
http://autobuild.buildroot.net/results/c65/c65c2b2c4c2ca328c4f26ba38798c40ab02c8772/
http://autobuild.buildroot.net/results/e0d/e0db38ca12dbe664d472c4f6f3f1ade4150119c3/
http://autobuild.buildroot.net/results/8e1/8e196d05dd65cdd782c8d41d937a7d9d2ef7dc1a/
http://autobuild.buildroot.net/results/e3a/e3a71f5d05a3e176a9e641e758b1ba64c9560991/

And many more.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-04-28 08:50:04 +02:00
Peter Korsgaard
7beb9ed0fd python3: move toolchain comment above main option
So suboptions are correctly indented.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-04-28 08:46:25 +02:00
Thomas Petazzoni
a692a251d4 python3: add hash file
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-04-28 00:26:47 +02:00
Thomas Petazzoni
6afba01e5d python3: bump to version 3.4.3
All patches are simply refreshed, except
002-properly-detect-if-python-build.patch which is removed because an
identical fix has been merged upstream:

  e128ea78e5

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-04-28 00:26:46 +02:00
Peter Korsgaard
7403ea730d Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-03-02 23:26:20 +01:00
Angelo Compagnucci
2df8cf5af7 package/python3: change BR2_PACKAGE_PYTHON_PYEXPAT description
This patch changes BR2_PACKAGE_PYTHON3_PYEXPAT description and
help text to underline that all the xml libraries will be
included in python.
It also reorders alphabetically the affected option.

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-02-21 17:09:31 +01:00
Yann E. MORIN
9863553fe8 packages: all salute the passing of avr32
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-02-14 17:43:11 +01:00
Thomas Petazzoni
414d7e97a7 python3: fix invalid ncursesw header path
This commit adds a patch to python3 that makes sure it does not use an
invalid header path (pointing to host headers) when including ncursesw
support.

Fixes:

  http://autobuild.buildroot.org/results/9bd/9bdaa392e8dd00c6ebee156b758e3c0cac480237/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Tested-by: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-02-12 22:28:17 +01:00
Thomas Petazzoni
be6798ace7 python3: make the ossaudiodev module optional
Like was done for the 'python' package, also make the ossaudiodev
module optional for 'python3'. ossaudiodev is always disabled for
host-python3, and a new option is added to enable it for the target
python3.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-01-10 16:32:48 +01:00
Thomas Petazzoni
c24c874810 python3: rename patches to the new convention
Note that we don't use completely sequential numbers, because patches
below 100 are used to address cross-compilation issues in Python 3,
while patches above 100 are used to make more Python 3 modules
configurable.

[Thomas: fixup commit log.]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2015-01-02 19:30:17 +01:00
Thomas Petazzoni
144e21f203 python3: bump to 3.4.2
This commit bumps python3 to Python 3.4.2. Two patches had to be
changed slightly to fix some minor conflicts.

PYTHON3_LIBTOOL_PATH = NO was added to prevent Buildroot from trying
to patch a version of libtool for which we don't have matching
patches, which isn't a problem since we're anyway not using the part
of the Python sources that uses libtool (it's the built-in copy of
libffi, and we use the external libffi).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
2015-01-02 19:29:53 +01:00
Peter Korsgaard
fe1b2ef1d3 Merge branch 'next'
Conflicts:
	Makefile
	package/flac/0001-fix-altivec-logic.patch
	package/grantlee/Config.in

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-12-01 11:16:42 +01:00
Peter Korsgaard
fbb8cbfdf4 python3: add patch to fix --includes / --ldflags output of python3-config
The build/real prefix handling using sed breaks if build != real and the
standard include / lib directories are used ($prefix/include and $prefix/lib).

E.G.

prefix_build="/usr", libdir="$prefix/lib", includedir="$prefix/include".

If this gets installed with make DESTDIR="/foo" install, then we end up with
prefix_real = prefix = "/foo/usr" as expected, but
includedir="/foo/foo/usr/include" and libdir="/foo/foo/usr/lib" because of
the double sed invocation (prefix is already expanded).  Work around it by
ensuring we only match the beginning of the string.

Submitted upstream: http://bugs.python.org/issue22907

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-11-22 19:14:11 +01:00
Thomas Petazzoni
5ad4ac0832 python3: don't use wcsftime() on uClibc
As investigated in bug #7646, wcsftime() doesn't work properly with
uClibc. Until it gets fixed in uClibc, let's tell Python 3 to not use
it. Python 3 will fall back to strftime(), which works properly.

[Peter: fix typo in comment]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Beyonlo <beyonlo@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-11-13 23:40:50 +01:00
Thomas De Schampheleire
f268f7131b .mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.

This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.

Alignment of line continuation characters (\) is kept as-is.

The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
    -e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'

Brief explanation of this command:
    ^\([A-Z0-9a-z_]\+\)     a regular variable at the beginning of the line
    \([?:+]\?=\)            any assignment character =, :=, ?=, +=
    \([^\\]\+\)             any string not containing a line continuation
    \([^\\ \t]\+\s*\\\)     string, optional whitespace, followed by a
                            line continuation character
    \(\s*\\\)               optional whitespace, followed by a line
                            continuation character

Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.

This expression was tested on following test text: (initial tab not
included)

	FOO     = spaces before
	FOO     =   spaces before and after
	FOO	= tab before
	FOO	  = tab and spaces before
	FOO =	tab after
	FOO =	   tab and spaces after
	FOO =   	spaces and tab after
	FOO =    \
	FOO = bar \
	FOO = bar space    \
	FOO   =		   \
	GENIMAGE_DEPENDENCIES   = host-pkgconf libconfuse
	FOO     += spaces before
	FOO     ?=   spaces before and after
	FOO     :=
	FOO     =
	FOO	=
	FOO	  =
	FOO =
	   $(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
	AT91BOOTSTRAP3_DEFCONFIG = \
	AXEL_DISABLE_I18N=--i18n=0

After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
  expression leaves the number of whitespace between the value and line
  continuation character intact, but the whitespace before that could have
  changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
  actually makes the code more readable.

Finally, the end result was manually reviewed.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 15:00:28 +02:00
Thomas Petazzoni
899d2af16d python, python3: convert py/pyc removal to TARGET_FINALIZE_HOOKS
Since the removal of py/pyc files is Python-specific, this commit
moves the logic removing those files to python.mk and python3.mk
respectively.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-05 14:32:41 +02:00
Thomas Petazzoni
ce0f19e5f6 python3: remove unneeded __pycache__ directories
The __pycache__ directories are only needed on PEP3147-enabled
installations, which is not the choice Buildroot has made. However,
during the build process of the target Python, the separation between
the target and host Python is not entirely complete, and starting the
host Python to byte-compile the target Python modules triggers the
compilation of a bunch of __pycache__ directories by the host Python,
which uses a PEP 3147 installation.

For now, simply get rid of those useless directories after the
installation of Python is completed. In the long term, it would be
good to achieve a better isolation between the host and target build
to avoid this issue.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-05 14:32:41 +02:00
Thomas Petazzoni
d02af2fd53 python3: add distutils fix for PEP 3147 issue
Python 3 has a new standard for installing .pyc file, called PEP
3147. Unfortunately, this standard requires both the .py and .pyc
files to be installed for a Python module to be found. This is quite
annoying on space-constrained embedded systems, since the .py file is
technically not required for execution.

For the Python standard library, our Python 3 package already contains
a patch named python3-004-old-stdlib-cache.patch, which allows to
disable the PEP 3147 installation.

But that leaves the distutils/setuptools package an unsolved
problem. This patch therefore adds a new patch to Python, which makes
distutils package use the traditional installation path when byte
compiling, rather than the PEP 3147 installation path. Since
setuptools relies on distutils internally, it also fixes setuptools
based packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
2014-10-05 14:32:35 +02:00
Thomas De Schampheleire
aaffd209fa packages: rename FOO_CONF_OPT into FOO_CONF_OPTS
To be consistent with the recent change of FOO_MAKE_OPT into FOO_MAKE_OPTS,
make the same change for FOO_CONF_OPT.

Sed command used:
   find * -type f | xargs sed -i 's#_CONF_OPT\>#&S#g'

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-10-04 18:54:16 +02:00
Christophe Vu-Brugier
edab68c1f1 python, python3: add patch to prevent distutils from adjusting the shebang
The copy_scripts() method in distutils copies the scripts listed
in the setup file and adjusts the first line to refer to the
current Python interpreter. When cross-compiling, this means that
the adjusted shebang refers to the host Python interpreter.

As a consequence, we add a patch for python and python3 that
force copy_scripts() to preserve the shebang when
cross-compilation is detected.

Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-09-21 19:46:13 +02:00
Jérôme Pouiller
77cf5b5a71 libffi and python: need threads support
libffi depends on pthreads.

Python depends on libffi (it can provide a builtin libffi, but also depends on
pthreads). Thus this patch also disable Python support if toolchain is compiled
w/o treads support.

Fixes http://sysmic.org/~jezz/results/204099dd:

../src/closures.c:119:21: fatal error: pthread.h: No such file or directory

[Peter: fix comment dependencies, add python3]
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-09-13 09:40:48 +02:00
Baruch Siach
ad4cbfe5f3 python3: fix termios build for xtensa
Forward port the patch from c821210b1b (python: fix termios build for
xtensa) to python3.

Fixes:
http://autobuild.buildroot.net/results/e7f/e7f71fbcb0796f315052947382fa39d85c5dc8fe/

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-08-06 22:17:30 +02:00
Thomas Petazzoni
d8ba42df87 python3: define license and license files
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-27 14:36:05 +02:00
Thomas Petazzoni
458bdf3f5f python3: optional decimal module support
This commit makes the decimal module support optional: since Python
3.3 it relies on an external library. The source code of this library
is available built-in as part of the Python sources, but in Buildroot
we generally prefer to use the external library when possible.

To achieve this, this commit adds a patch to Python that is similar to
the one we use for expat support, but this time for the
libmpdec/mpdecimal library.

As a consequence, since mpdecimal now builds properly even when
<fenv.h> is not available (on i386), this commit fixes:

  http://autobuild.buildroot.org/results/b64/b64d5c941a7cac00619da3a0696939f86a8eafc2/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-27 14:35:08 +02:00
Henry Margies
a37b214c43 python, python3: build with tzset support
This patch adds support for time.tzset() in Python 2.7 and Python 3.

The Python configure script sets ac_cv_working_tzset to 'no' when
cross compiling, although it should be available and work with all
toolchains supported by Buildroot (?).
The patch below pre-sets the variable so that time.tzset() works.

[Thomas: adjust commit log.]

Signed-off-by: Henry Margies <henry.margies@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-07-15 20:39:09 +02:00
Thomas Petazzoni
5364b22b6c python3: improve isolation of build environment and error handling
This commit mainly fixes bug #7268 by improving the isolation of
the build environment by making sure some host environment header
paths do not leak into the target build. The investigation was done by
David <buildroot-2014@inbox.com>. This is done by not calling the
add_multiarch_paths() function of setup.py when we're cross-compiling,
a change made in the newly introduced
python3-012-dont-add-multiarch-path.patch.

In addition to this, another patch is added to make sure the build is
aborted when one of the Python module fails to build. This is done in
python3-013-abort-on-failed-modules.patch. Without this, the Python
setup.py script simply logs which module failed to build, but doesn't
abort, so it's hard to notice when there is a problem.

[Peter: slightly reword commit message]
Cc: David <buildroot-2014@inbox.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-09 13:19:20 +02:00
Thomas Petazzoni
a5dcdf3706 python3: bump to version 3.4.1
A few minor updates on the patches were needed to resolve some
conflicts on the main Makefile.pre.in file, in the list of modules to
build/install.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-08 13:56:45 +02:00
Thomas Petazzoni
286ee8001c python3: do not rely only on LIBRARY_PATH for old compilers
The cross-compilation improvements integrated in Python rely on the
compiler exposing a line starting with LIBRARY_PATH when called with
-E -v. This is used by Python setup.py to find the installation
locations of libraries.

However, this LIBRARY_PATH line is not shown by very old compilers,
such as the gcc 4.2.x compiler used on the AVR32 architecture. This
causes libraries installed in the sysroot, such as libffi, to not be
detected by the setup.py script.

To fix this problem, this patch adds addtional logic to setup.py,
which consists in deriving the library paths from the sysroot
location, if no LIBRARY_PATH field was found.

Fixes:

  http://autobuild.buildroot.org/results/7a6/7a65e381cc04bf8f74fd63a6dcda502f3c26aeef/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-20 18:35:53 +02:00
Thomas Petazzoni
97fba4cd24 python3: correctly fix PYTHON3_PATH
Commit 6251ccf032 ('python3: fix the
value of PYTHON3_PATH') tried to fix the value of PYTHON3_PATH, but
did it incorrectly: it changed PYTHON_VERSION_MAJOR to
PYTHON_VERSION3_MAJOR, while it should have been
PYTHON3_VERSION_MAJOR.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-12 23:03:10 +02:00
Thomas Petazzoni
6251ccf032 python3: fix the value of PYTHON3_PATH
The PYTHON3_PATH was incorrectly referencing the site-packages of
Python 2 packages, due to the usage of PYTHON_VERSION_MAJOR, instead
of PYTHON3_VERSION_MAJOR. This commit fixes that by using the correct
variable, which in our testing fixed the build of python-pyasn against
python3.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-05 17:38:26 +02:00
Samuel Martin
a8127d3cf8 python3: rework python symlinks installation
This patch reworks the way python3 and python3-config symlink are
installed.

Buildroot wants to control these symlinks' installation:

* the python3 symlink should be unconditionally installed in the target
  tree, and the python3-config symlink in the staging tree, since it is
  the only python package built and installed in the target tree if the
  user selected it;

* the python3 and python3-config symlinks should only be installed in
  the host tree when python3 is the selection of the user for the
  target.

[Thomas: fix comment as suggested by Yann.]

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-04-05 15:38:07 +02:00
Jerzy Grzegorek
6b925f1e54 python3: drop PYTHON3_VERSION_MINOR variable
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-03-27 00:16:59 +01:00
Peter Korsgaard
b40eb00264 python3: bump version
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-03-17 22:10:12 +01:00
Arnout Vandecappelle
6c94ab3cfd host-python{, 3}: fix parallel install of libpython
During installation, host-python and host-python3 run the freshly built
python executable. This is done with a proper LD_LIBRARY_PATH to make
sure it picks up the libpython in the build directory. However, the
python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
Therefore, if libpython exists there, it will be used instead.

If the install step is run in parallel, it is possible that libpython
is already partially copied to $(HOST_DIR)/usr/lib when python is run.
This gives an error like:

python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short

The fix is simple: use RUNPATH instead of RPATH, which allows
LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
build directory is always used. RUNPATH is enabled by passing
--enable-new-dtags to the linker.

Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-03-05 19:45:24 +01:00
Thomas Petazzoni
bec21dcfa7 python, python3: fix unicodedata enabling
A confusion was made when applying commit
c3d539b53f ('host python/python3:
conditionally disable unicodedata') and then commit
73293e88c8 ('python, python3: enable
unicodedata for host-python, needed by setuptools').

The first commit added the enabling of unicodedata in the host Python
when the target Python is configured with unicodedata, which is
necessary because building the target unicodedata Python module
requires having unicodedata support in the host Python.

The second commit enabled the unicodedata unconditionally in the host
Python, because it is needed by host-setuptools.

However, when the second commit was applied, the logic that consisted
in disabling unicodedate in the host Python when not needed was kept,
which resulted in breaking the build of host-setuptools due to the
lack of unicodedata support in the host python.

This commit fixes that by only leaving the unconditional
--enable-unicodedata in the host Python configuration options, and
therefore removing the problematic conditional --disable-unicodedata.

Fixes:

  http://autobuild.buildroot.org/results/b62/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-03-01 13:06:18 +01:00
Peter Korsgaard
b108fdcb83 Merge branch 'next'
Conflicts:
	Makefile
	package/dmraid/Config.in
	package/gdb/Config.in.host
	package/linux-headers/linux-headers.mk
	package/python/python.mk
	package/python3/python3.mk
	package/rt-tests/Config.in
	package/sdl/sdl.mk
	package/systemd/systemd-01-fix-getty-unit.patch
	package/systemd/systemd-02-fix-page-size.patch
	package/systemd/systemd-03-uclibc-fix.patch
	package/udev/Config.in
	package/udisks/Config.in
	package/vlc/vlc.mk
	system/Config.in

Quite some merge conflicts, hopefully I didn't screw up anything.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-28 14:30:23 +01:00
Przemyslaw Wrzos
99d930e65d python, python3: fix to ensure libpython is stripped
The python and python3 builds mark libpython as read-only which
prevents it from being stripped out correctly for the target.

Signed-off-by: Przemyslaw Wrzos <przemyslaw.wrzos@calyptech.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-19 08:53:59 +01:00
Thomas Petazzoni
73293e88c8 python, python3: enable unicodedata for host-python, needed by setuptools
As we are going to bump setuptools to a much newer version, the host
python needs to be built with support for unicodedata.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-19 08:53:12 +01:00
Thomas Petazzoni
eb7667bf41 python3: provide a PYTHON3_PATH
The Python package infrastructure will need the Python 3 package to
provide a PYTHON3_PATH environment variable in order to build
third-party Python modules.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-18 23:29:34 +01:00
Thomas Petazzoni
0d327c267a python3: bump to 3.4.0rc1
This commit bumps the Python3 package to use Python 3.4.0rc1.

About the patches:

 * The patches below 100 are significantly changed, because like for
   Python 2.x, a good number of improvements have been made in the
   upstream Python for cross-compilation. Therefore, almost all of
   these patches have been modified.

 * All the patches above 100 are simply updated for Python 3.4.0, with
   a small refactoring for the handling of test modules.

The details of the python3.mk changes are:

 * --without-ensurepip to tell Python to not use PIP at build time.

 * Many environment variables are no longer passed, they were specific
   to our cross-compilation patches

 * The fixup of the LIBDIR in the Python Makefile is no longer needed
   since Python has switched to _sysconfigdata.py for distutils
   configuration instead of parsing the Makefile.

 * A new post patch hooks touches the two files generated by pgen to
   make sure they are newer than the pgen sources, which ensures pgen
   is not built/executed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-18 23:22:18 +01:00
Thomas Petazzoni
249778541b python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3
Some parts of python3.mk were hardcoding the 3.3 version as the major
version, which does not work for Python 3.4 and other future
versions. Instead, use the existing PYTHON3_VERSION_MAJOR.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-18 23:21:34 +01:00
Thomas De Schampheleire
c3d539b53f host python/python3: conditionally disable unicodedata
The host python always had --disable-unicodedata, regardless of the
corresponding configuration option BR2_PACKAGE_PYTHON_UNICODEDATA.
Since the host python is used to byte-compile python modules, this meant
that such modules could not contain unicode strings. For example, following
statement in a python module:
    print u"\N{SOLIDUS}"

would cause the byte-compilation to fail with message:
    SyntaxError: ("(unicode error) \\N escapes not supported (can't load
    unicodedata module)",

Instead, conditionally disable unicodedata based on
BR2_PACKAGE_PYTHON_UNICODEDATA, also for the host python.

This fixes bug #6542 (https://bugs.busybox.net/show_bug.cgi?id=6542)

Reported-by: Gernot Vormayr <gvormayr@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-02-18 23:09:05 +01:00