When using an external toolchain that was built with Buildroot and a
merged /usr, the dynamic linker is actually in /usr/lib.
But the check_glibc macro limits the depth it is looking for the dynamic
linker, and misses it when it is in /usr/lib because it is too deep.
We could fix that in two ways: increase the depth in which we look
for it, or follow symlinks. We choose the second solution.
Signed-off-by: Cam Hutchison <camh@xdna.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In commit 32bec8ee2f
("toolchain-external: copy ld*.so* for all C libraries") we changed
how the musl dynamic linker symbolic link was being created. Instead
of having specific logic in Buildroot, we switched to simply copying
the ld*.so.* symbolic link from staging to target, as well as the
target of this symbolic link.
However, it turns out that by default, musl creates its dynamic linker
symbolic link with an absolute path as the target of the link:
/lib/libc.so.
Therefore, external Musl toolchains built with Buildroot look like
this:
lrwxrwxrwx 1 thomas thomas 12 Jul 4 19:46 ld-musl-armhf.so.1 -> /lib/libc.so
The principle of the copy_toolchain_lib_root function, which is used
to copy libraries from staging to target, is to copy symbolic links
and follow their targets. In this case, it means we end up copying
/lib/libc.so (from the host machine) into the target folder. From
there on, there are two cases:
1. /lib/libc.so exists in your host system. It gets copied to the
target. But later on, Buildroot also copies /lib/libc.so from
staging to target, overwriting the bogus libc.so. So everything
works fine, even though it's admittedly ugly.
2. /lib/libc.so doesn't exist in your host system. In this case, the
build fails with no clear error message.
This problem does not happen with Musl toolchains built by
Crosstool-NG, because Crosstool-NG replaces the absolute target of the
dynamic linker symbolic link by a relative path.
However, since we want to support existing Buildroot Musl toolchains
and generally work with the fact that Musl by default installs an
absolute symlink, the following commit improves the
copy_toolchain_sysroot function to replace symbolic links with an
absolute destination to use a relative destination. I.e, in staging,
the ld-musl-armhf.so.1 symbolic link looks like this:
lrwxrwxrwx 1 thomas thomas 14 Jul 5 22:59 output/staging/lib/ld-musl-armhf.so.1 -> ../lib/libc.so
Fixes:
http://autobuild.buildroot.net/results/ce80264575918a8f71d9eab1091c21df85b65b1a/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In commit b3cc7e65ee, the definition of the DESTDIR variable was moved
down into the loop that follows symlinks in the libraries that are
copied to target. However, the corresponding mkdir was not moved down,
so that no directories are ever created.
In practice, this mkdir is normally redundant since the directories
should already have been created as part of creating STAGING_DIR.
Still, the current situation is clearly wrong, so fix it by moving the
mkdir down to after the assignment to DESTDIR.
While we're at it, also remove a redundant empty line. It's a leftover
from when a lot of variables were declared above.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In copy_toolchain_sysroot, if no ld.so has been found in the
STAGING_DIR after the sysroot copy, we look at ARCH_SYSROOT_DIR if a
ld.so is available there. We do this for both ld*.so and ld*.so.*.
However, when copying thing from staging to target (as listed in
TOOLCHAIN_EXTERNAL_LIBS), we only match on ld*.so.*. This would mean
that even if a dynamic linker matching ld*.so but not ld*.so.* was
copied into staging by copy_toolchain_sysroot, it would anyway not be
copied to the target filesystem, making the system unusable.
Therefore, we can remove the special case on ld*.so, and keep only
ld*.so.*.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root copies libraries from staging to target,
resolving and copying symbolic links along the way.
The most inner loop, a "while" loop, starts from an initial name, and
if it's a symbolic link, gets resolved to the target, and the loop
iterates until we reach a real file. However, the destination folder
where the symbolic link or real file is created is computed in DESTDIR
only once, before this loop starts. Therefore, this loop works fine
when all symbolic links in the chain, and the real file all belong to
the same directory. But it doesn't do the correct thing when the
symbolic link and/or real file are in different folder.
An example is Crosstool-NG musl toolchains, where the dynamic loader
is in /lib/ld-musl*.so but points to ../usr/lib/libc.so. With the
current logic, we copy /lib/ld-musl*.so to /lib, but we also copy
libc.so to /lib instead of the expected /usr/lib.
This currently doesn't cause any problem because the musl dynamic
linker is manually created by the TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
hook. However, this logic has a number of problems, so in a followup
commit, we are going to put the musl dynamic linker in
TOOLCHAIN_EXTERNAL_LIBS, which will cause it to be copied by
copy_toolchain_lib_root. But we obviously want the link and its target
to be copied to the right place, hence this fix.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit ba6bac1383 made a change in copying of
the dynamic loader, with the goal of reducing toolchain-specific fixups.
Any ld*.so file found in the toolchain's lib directory would be copied to
the staging/lib directory.
For the toolchains that previously needed fixup, this new behavior is fine.
The reason they needed fixup was that the normal copy action did not include
any dynamic loader.
However, for certain other toolchains this new behavior actually breaks
things: regardless of ARCH_LIB_DIR, which may be lib64 instead of lib, the
dynamic loader from lib is copied _over_ any previously correct dynamic
loader.
This has been witnessed with the CodeSourcery x86_64 and
CodeSourcery MIPS64 toolchains. In both cases, a 32-bit dynamic loader was
copied to staging/lib, while a 64-bit version was expected.
Fix the problem by only performing this explicit dynamic loader copy if no
dynamic loader is found in staging/lib.
Fixes http://autobuild.buildroot.net/results/8bf/8bffe54032aad9cc710a22411ef3bff4a2c93e55/
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root handles symlinks by recreating them, disregarding
the original destination and assuming the destination is in the same
directory as the link itself.
When a library link points to the real library file in another directory,
for example:
usr/lib/octeon2/libcrypt.so -> ../../../lib32/octeon2/libcrypt.so.1
then the link created by copy_toolchain_lib_root is broken.
It is more robust to copy the symlink to keep the destination intact. The
destination path should be present, possibly through other symbolic links.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The input to copy_toolchain_lib_root is not one library, not a list of
libraries, but a library name pattern with glob wildcards.
This pattern is then passed to 'find' to get the actual list of libraries
matching the pattern. Reflect this using an appropriate variable name.
Note: if the root of the buildroot tree contains a file matching one of
these library patterns, the copying of libraries from staging to target will
not be correct. It is not impossible to fix that, e.g. using 'set -f', but
maybe it's not worth it.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root has slightly different logic depending on the type
of library object: file or link. All actions related to links are not
relevant in case you are working with a file. Hence, try to increase clarity
by not executing unnecessary lines in the 'file' case.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A previous commit rewrote broken symbolic links in staging, caused by a
non-singular ARCH_LIB_DIR. In this case, the symbolic links are typically
using one or more intermediate directory symlinks, which can be simplified
using the newly introduced simplify_symlink helper.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The external toolchain logic flattens the directory layout in the staging
directory. Regardless of the number of levels present in the extracted
toolchain, libraries are always copied to lib/ and usr/lib/, and directory
symbolic links are provided to make the original paths available as well.
Due to this, the same library may be reachable through a number of paths:
one path without any symbolic link, and one or more paths using directory
symlinks.
Using a direct path in a symlink destination is generally preferred because
it is clearer, but it is also more robust against accidental removal of an
intermediate directory symlink.
Introduce a helper function to simplify a symlink's destination to such a
direct path. This function will be used in a subsequent patch.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain from the Cavium Octeon SDK has a sysroot layout as follows:
./lib32
./lib32/octeon2
./lib32-fp
./lib64
./lib64/octeon2
./lib64-fp
./usr
./usr/lib
./usr/lib32
./usr/lib32/octeon2
./usr/lib32-fp
./usr/lib64
./usr/lib64/octeon2
./usr/lib64-fp
./usr/bin
./usr/bin32
./usr/bin32-fp
./usr/bin64-fp
./usr/libexec
./usr/libexec32
./usr/libexec32-fp
./usr/libexec64-fp
./usr/sbin
./usr/sbin32
./usr/sbin32-fp
./usr/sbin64-fp
./usr/include
./usr/share
./sbin
./sbin32
./sbin32-fp
./sbin64-fp
./etc
./var
with the following selections:
- lib64 : default
- lib64/octeon2 : -march=octeon2
- lib64-fp : -march=octeon3
- lib32 : -mabi=n32
- lib32/octeon2 : -mabi=n32 -march=octeon2
- lib32-fp : -mabi=n32 -march=octeon3
In case of '-mabi=n32 -march=octeon2' (but same is true for n64+octeon2)the
original Buildroot toolchain logic would copy both the libraries in
lib32 as the subdirectory lib32/octeon2, which means that every library is
installed twice (but only one of each is really needed).
While ARCH_LIB_DIR is determined by the location of libc.a, which in this
case is effectively:
<sysroot>/usr/lib32/octeon2/libc.a
the variable only retains 'lib32' and not 'lib32/octeon2' as expected.
To make Buildroot cope with this style of toolchain layout, we need to adapt
the calculation of ARCH_LIB_DIR to also include the second part.
This, in turn, means that ARCH_LIB_DIR is no longer guaranteed to be a
singular path component, resulting in some additional changes.
Certain older Linaro toolchains actually had the same layout. Libraries were
located in lib/<tuple> rather than lib directly. Previously, this was
handled by adding a toolchain-specific fixup that creates a symlink
lib/<tuple> -> lib, but with this patch this would no longer be needed.
Note that one difference with the Octeon case is that these Linaro
toolchains are not actually multilib, i.e. there is just one location with
the libraries and thus there is no problem with duplicated libraries.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The helper function copy_toolchain_sysroot has some logic to transform a
path into a number of '../' components based on the depth of that path.
As this same logic will be needed in another place in a subsequent patch,
extract it into a separate helper relpath_prefix.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Normally, the Buildroot toolchain logic copies all required libraries from
the external toolchain to the staging directory, including the dynamic
loader ld-*.so.
There are cases, however, where the dynamic loader is _not_ automatically
copied to staging. This happens when the dynamic loader is not inside
ARCH_LIB_DIR itself (e.g. lib64), but instead resides in 'lib' (assume, of
course, that ARCH_LIB_DIR != 'lib').
Currently, this is fixed in a toolchain-specific fixup, e.g. by recreating a
missing symlink or copying over a missing file. Such toolchain specific
fixups are not very nice.
Moreover, in a subsequent patch, the value of ARCH_LIB_DIR changes for some
toolchains, causing them to have the same problem of a missing dynamic
loader. This used to be the case for older Linaro toolchains with libraries
in 'lib/<tuple>': Buildroot used to set ARCH_LIB_DIR=lib but the mentioned
patch changes it to 'lib/<tuple>' instead. As a result, the files directly
under 'lib/' will no longer be copied. There should be none, but the dynamic
loader is a notable exception.
[Note: support for these older Linaro toolchain has been removed in 2016.11]
Instead, copy over the ld.so file(s)/link(s) from the extracted toolchain
into staging, in the central copy_toolchain_sysroot function. The existing
toolchain logic will then handle the copy of these files from staging to
target.
This means the toolchain-specific fixups can be removed.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The copy_toolchain_sysroot helper features a complex rsync loop that copies
various directories from the extracted toolchain to the staging directory.
The complexity mainly stems from the fact that we support multilib toolchain
tarballs but only copy one of the multilib variants into staging.
Increase understandability of this logic by explicitly restricting the
rsync excludes to the iteration of the for loop they are relevant for.
Additionally, update the function comment.
Note: all attempts to reduce duplication between both rsync while keeping
things nice and readable failed. One has to be extremely careful regarding
line continuation, indentation, and single vs double quoting. In the end, a
split up rsync seemed most clean.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check_arm_abi function takes as second argument the path to the
cross-readelf, but does not use it. Therefore, this commit gets rid of
this unnecessary argument.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The current test to verify if the toolchain uses musl or not is based on
checking if /lib/libc.so or /lib/libm.so exist in the sysroot. However,
some toolchains (notably Crosstool-NG ones) put these libraries in
/usr/lib/.
To fix this, build a minimal C program and check if the program
interpreter contains /lib/ld-musl.
Signed-off-by: Ilya Kuzmich <ilya.kuzmich@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The somewhat complicated sed expression has been removed in commit
06cd604ec6 ("toolchain/external: use
-dumpversion to check gcc version"), so let's remove the comment that
was explaining this sed expression.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, `--version` option is used and later matched with a regex to get
the actual gcc version. There's a dedicated gcc option to do exactly that:
`-dumpversion`.
Also `--version` may return a string customised by a vendor that provides
the toolchain, which makes the current regex approach error prone. In
fact, this situation has been seen with a real customised toolchain.
Signed-off-by: Krzysztof Konopko <kris@youview.com>
Signed-off-by: Tomasz Szkutkowski <tomasz.szkutkowski@youview.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Function copy_toolchain_sysroot, which is in charge of copying the relevant
bits from the external toolchain to the staging directory, performs an rsync
loop of various directories and excludes the pattern 'usr/lib/locale' with
the intention of skipping the directory <toolchain>/usr/lib/locale.
However, while this worked in the original commit, commit
5628776c4a broke it inadvertently. The
relevant part of the diff:
- rsync -au --chmod=Du+w --exclude 'usr/lib/locale' \
- $${ARCH_SYSROOT_DIR}/$$i $(STAGING_DIR)/ ; \
+ rsync -au --chmod=Du+w --exclude 'usr/lib/locale' \
+ --exclude lib --exclude lib32 --exclude lib64 \
+ $${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
Notice how the source directory now contains a trailing slash, which impacts
the way the exclude rules are interpreted. Previously, when 'i' was 'usr',
the exclude of 'usr/lib/locale' would find a match. With the trailing slash,
there will never be a match, unless for a directory 'usr/usr/lib/locale'.
The right rule would have been '--exclude lib/locale'.
However, just that fix does not solve the problem in all cases, in
particular in the (common) case where ARCH_LIB_DIR is 'lib'. This is due
another change in that commit, changing the iterated values of the above
rsync:
- for i in etc $${ARCH_LIB_DIR} sbin usr ; do \
+ for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
Due to the fact that we rsync both 'usr' as 'usr/lib' (assuming ARCH_LIB_DIR
is 'lib') we need to add the correct exclude in both cases. But the exclude
is different for both. When i == 'usr', the correct exclude rule would be
'--exclude lib/locale' while when i == 'usr/lib' the correct rule would be
'--exclude locale'.
Since we would like to avoid separate cases for this, use the following
exclude: '--exclude locale/'. The trailing slash will make sure only
directories called 'locale' will match. The targeted directories are then
usr/lib/locale and usr/share/locale. The latter directory was not matched
originally, but it should not hurt changing that.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As discussed with Thomas Petazzoni, we can reduce the nesting level by early
returning on an invalid iteration.
I did not move the 'else' case (the common case) outside the if-else because
it would make the code less symmetrical and IMO makes it _less_ clear.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently support gcc as old as 4.3. However, Buildroot works
perfectly well with even older gcc versions (tested with 4.1). So we
can add an option BR2_TOOLCHAIN_EXTERNAL_GCC_OLD to support that. The
help text of this option is written with plenty of discouragement.
We use _OLD and not something like _PRE_4_3, because at some point we
will likely remove the 4.3 option and what would then require a name
change.
We don't set any _AT_LEAST option in this case because it's no use -
there is no lower bound on the version in this case. We therefore leave
BR2_TOOLCHAIN_GCC_AT_LEAST empty (the implicit default). When it is
empty, we don't do a version check at all in check_gcc_version
(previously we errored out when it was empty).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
[Thomas: remove extension for the generated temporary file, since it's
really an executable, not an object file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This reverts commit 9a1e9efe26.
Currently Codescape toolchains cannot be used to generate a big endian
root file system because the support for side by side sysroots is not
complete.
There is a patch [1] waiting in the queue which fixes the issue for the
current version of Codescape toolchains we have, but it will not work
for the next one that is coming. So, instead of messing more with the
toolchain infra I think it's better to handle these specific Codescape
toolchain's weirdness in hooks which won't affect others.
[1]: http://patchwork.ozlabs.org/patch/571708/
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In 2a87b64 (toolchain-external: align library locations in target and
staging dir), copying the libraries from the sysroot to the target was
changed to a simple find-based solution.
To be sure that the staging directory was entered to find the libraries,
in case the variable was pointing to a symlink, the -L clause to find
was used.
However, that causes extraneous libraries to be copied over.
For example, a ct-ng toolchain would have this sysroot (e.g for an arm
32-bit toolchain):
.../sysroot/lib/
.../sysroot/lib32 -> lib
.../sysroot/lib64 -> lib
.../sysroot/usr/lib/
.../sysroot/usr/lib32 -> lib
.../sysroot/usr/lib64 -> lib
Which we would carry as-is to our own sysroot.
But then, in target, our skeleton creates the /lib/ and /usr/lib
directories, with the necessary lib32 or lib64 symlink pointing to it.
In this case, a lib32->lib symlink is created, but no lib64 symlink
since this is a 32-bit architecture.
To copy the required libraries from staging into target, we scan the
staging directory for all occurences of the required libraries, and copy
them over to target, keeping the same directory layout relative to the
sysroot.
For example:
.../sysroot/usr/lib/libfoo.so --> .../target/usr/lib/libfoo.so
.../sysroot/usr/lib32/libbar.so --> .../target/usr/lib32/libbar.so
.../sysroot/usr/lib64/libbuz.so --> .../target/usr/lib64/libbuz.so
So, when we copy over the libraries from our staging to the target
directory, the "find -L .../sysroot -name libblabla.so.*" would find
multiple instances of libblabla, each in the /usr/lib /usr/lib32 and
/usr/lib64 locations (they are all the exact same file, though).
Since we do have the /usr/lib32->lib symlink, all is OK (but there are
two copies going on, which could be avoided). However, since we do not
have the /usr/lib64->lib symlink, the /usr/lib64/ directory is created.
This was very difficult to observe, as no /lib64/ directory is created,
only the /usr/lib64/ one was. To top it off, this only happens with a
merged /usr, which does not seem like not a common case without systemd.
Since the reason to use -L was to be sure to enter our staging
directory, we just need to ensure that the path ends up with a slash, as
was already talked about in this thread:
http://lists.busybox.net/pipermail/buildroot/2016-April/159737.html
After further discussion, it turns out that the original patch came along
because of the confusion between output/staging (which is a symlink) and
$(STAGING_DIR) which expands to output/host/usr/<tupple>/sysroot (which is
never a symlink), so the symlink handling isn't really needed at all.
[Peter: drop description comment, extend description]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Don't enable SSP support on external toolchains just because they use
glibc or musl. Instead of that, make the external toolchains explictily
declare if they support SSP or not. And also add a check to detect SSP
support when using custom external toolchains.
For internal toolchains we always enable SSP support for glibc and musl.
Fixes:
http://autobuild.buildroot.net/results/ac7c9b3ad2e52abfe6b79a80045e4218eeb87175/
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
[Thomas:
- remove uClibc-specific SSP check, since there is now a generic
check being done.
- send potential compilation errors caused by the SSP check to
oblivion, in order to avoid causing confusion for the user.
- add autobuilder reference.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The sysroot toolchain support check is duplicated at two locations in
the external toolchain infra. So move it inside the
check_unusable_toolchain helper that is called when the toolchain
package is configured (TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS).
The check in TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS can be safely
removed since it's already done in check_unusable_toolchain helper.
The check in TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS was removed by
2a87b64f8e.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some toolchain can't be used by Buildroot due to sysroot location
issue, so the $(ARCH)-linux-gnu-gcc -print-file-name=libc.a command
return only "libc.a"
This lead to an error during the header check version helper,
so these toolchains can't be imported into Buildroot.
cc1: fatal error: $PWD/libc.a/usr/include/linux/version.h: No such file or directory
compilation terminated.
support/scripts/check-kernel-headers.sh: line 38: /tmp/check-headers.4V5PPF: Permission denied
This issue happen with the first linaro 2015.11 [1] release and
CodeSourcery standard edition [2].
Here is the sysroot directory tree for linaro 2015.11:
$ ls libc/arm-linux-gnueabihf
etc lib sbin usr var
Here is the sysroot directory tree for CodeSourcery standard:
$ ls libc/sgxx-glibc
etc lib lib64 sbin usr var
Add a check to error out with an explicit error message
The check don't use toolchain_find_libc_a function directly since
"realpath -f" is used internally and return an absolute path.
[1] https://bugs.linaro.org/show_bug.cgi?id=1995#c7
[2] http://lists.busybox.net/pipermail/buildroot/2014-October/110696.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain-external logic is roughly:
- populate the staging dir by rsyncing the entire ${ARCH_LIB_DIR} and
usr/${ARCH_LIB_DIR} from sysroot.
- populate the target dir by explictly copying some libraries from sysroot
into target/lib and some other libraries in target/usr/lib, the split
being hardcoded into buildroot regardless of the location in the sysroot.
This means that a library libfoo could be located in:
staging/lib/libfoo.so
target/usr/lib/libfoo.so
When debugging an application that links against this library, gdb will
fruitlessly search for 'usr/lib/libfoo.so' in staging, and then suggest to
use 'set solib-search-path' which is a hack, really.
To solve the problem, we need to make sure that libraries from the toolchain
are installed in the same relative location in staging and target.
Achieve this by:
- replacing the convoluted search for libraries using for+find in sysroot
with a simple find in staging.
- determining DESTDIR for each library individually based on the location in
staging.
- treating LIB_EXTERNAL_LIBS and USR_LIB_EXTERNAL_LIBS equivalently
These changes also allow for the removal of most arguments to
copy_toolchain_lib_root in the method itself and their callers.
Test procedure:
- set configuration for a given toolchain
- make clean toolchain
- find output/target | sort > /tmp/out-before
- apply patch
- make clean toolchain
- find output/target | sort > /tmp/out-after
- diff -u /tmp/out-before /tmp/out-after
The only changes should be some libraries moving from lib to usr/lib or vice
versa. Notable examples being libstdc++ and libatomic.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- use -L instead of -follow in the find invocation, as suggested by
Arnout.
- move the BR2_STATIC_LIBS condition as a make condition rather than
a shell condition, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
LIBSPATH is populated based on a find with a pattern that can look like:
libfoo*.so
and thus the output of the find will contain all file paths that match this
pattern.
Unfortunately, the name LIBSPATH suggests that only one entry is returned,
rather than possibly multiple.
As this code is quite complex, use the more accurate name LIBPATHS iso
LIBSPATH.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an
rsync of various directories from the extracted external toolchain to the
corresponding directory in staging.
The relevant (simplified) snippet is:
for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
--exclude '/lib/' --exclude '/lib32/' \
--exclude '/lib64/' \
$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
done ; \
The exclusion logic of lib/lib32/lib64 has originally been added by commit
5628776c4a with the purpose of only copying
the relevant usr/lib* directory from the toolchain to staging, instead of
all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be
copied and usr/lib and usr/lib32 are ignored. It works by ignoring any
lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately
copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have
impact on the files beneath the main source directory.)
However, ARCH_LIB_DIR can take other values than (lib, lib32, lib64), for
example lib32-fp or lib64-fp (Octeon III toolchain with -march=octeon3). In
the existing code, the rsync for 'usr' would then already copy these lib
directories, and the next rsync for 'usr/$${ARCH_LIB_DIR}' does nothing.
By itself, this is not a very big problem: the staging directory simply has
some extra directories. However, a subsequent patch will create a staging
symlink from $${ARCH_LIB_DIR} to lib. The first rsync would then overwrite
that symlink with the real directory usr/$${ARCH_LIB_DIR} from the
toolchain, which is not correct.
Assuming the patch that creates the symlink ARCH_LIB_DIR->lib is applied,
the original situation after 'make clean toolchain' with an
ARCH_LIB_DIR=lib32-fp is:
$ ls -ld output/staging/{,usr/}lib* output/target/{usr/,}lib*
drwxr-xr-x 2 4096 May 26 2015 output/staging/lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/staging/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/staging/lib32-fp -> lib
drwxr-xr-x 2 4096 Jan 20 13:47 output/staging/usr/lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/staging/usr/lib32 -> lib
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/lib32-fp
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/lib64-fp
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/libexec
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec32
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec64-fp
drwxr-xr-x 2 4096 Jan 20 13:48 output/target/lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/target/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/target/lib32-fp -> lib
drwxr-xr-x 2 4096 Jan 20 13:48 output/target/usr/lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/target/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 13:47 output/target/usr/lib32-fp -> lib
Notice how usr/lib32-fp is not a symlink but a directory, and the presence
of an unnecessary directory usr/lib64-fp.
This patch improves the rsync exclusion rules by excluding any lib*
directory on the first rsync. As this would also exclude any
libexec/libexec32/... directory, explicitly include them first (first match
takes precedence). This (as is already the case today) results in more
usr/libexec* directories than needed, but it is not touched by this patch.
With the fix applied, the situation becomes:
drwxr-xr-x 2 4096 May 26 2015 output/staging/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32-fp -> lib
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32-fp -> lib
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/libexec
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec32
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3 4096 May 26 2015 output/staging/usr/libexec64-fp
drwxr-xr-x 2 4096 Jan 20 14:27 output/target/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32-fp -> lib
drwxr-xr-x 2 4096 Jan 20 14:27 output/target/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32-fp -> lib
For cases where ARCH_LIB_DIR is one of lib, lib32 or lib64 this fix
makes no difference, and likewise for internal toolchains.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an
rsync of various directories from the extracted external toolchain to the
corresponding directory in staging.
The relevant (simplified) snippet is:
for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
--exclude lib --exclude lib32 --exclude lib64 \
$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
done ; \
The exclusion logic of lib/lib32/lib64 has been added by commit
5628776c4a with the purpose of only copying
the relevant usr/lib* directory from the toolchain to staging, instead of
all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be
copied and usr/lib and usr/lib32 are ignored. It works by ignoring any
lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately
copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have
impact on the files beneath the main source directory.)
However, on the rsync of 'usr', ANY of the following directories AND files
would be excluded:
lib/
lib
lib32/
foobar/something/lib/
something-else/lib64/
while it is only the intention to skip directories directly under usr.
Therefore, add a leading (to restrict the scope to first-level) and trailing
(to restrict to directories) slash to the exclude pattern. From 'man rsync':
- if the pattern starts with a / then it is anchored to [..] the root of
the transfer.
- if the pattern ends with a / then it will only match a directory, not
a regular file, symlink, or device.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently our toolchain infrastructure assumes that every toolchain has
nested sysroot directories. However that's not true for all of them. The
Codescape toolchains from Imagination Technologies use a side by side
sysroot structure, for instance.
This patch allows our toolchain infrastructure to detect what kind of
sysroot structure we have (nested or side by side) and performs the
appropriate actions.
[Thomas: update the comment above the function, to explain what's
going on with nested sysroots and side-by-side sysroots.]
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Introduced by previous patch 0f75b2635e,
this printf would break the build of glibc, because there is no format
to printf:
printf: usage: printf [-v var] format [arguments]
Signed-off-by Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reported-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
'echo -n' is not a POSIX construct (no flag support), we shoud use
'printf', especially in init script.
This patch was generated by the following command line:
git grep -l 'echo -n' -- `git ls-files | grep -v 'patch'` | xargs sed -i 's/echo -n/printf/'
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, when a preconfigured prebuilt toolchain forgets to specify
its gcc version, the error message is a bit misleading, like:
Incorrect selection of gcc version: expected .x, got 4.9.2
Add a an explicit check for the gcc version being set, that reports a
better error message.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
gcc will always report a three-digit version sting, like 4.9.3 or 5.1.0.
For gcc before 5, we want to check the first two digits, while starting
with gcc 5, we are only concerned about the first digit.
So, change our matching code to test for the leading part of the version
string, up to the first dot after as-many version digit we're interested
in.
Note: we're adding the dot in the .mk code rather than in the Kconfig
symbol, because it seemed cleaner to do so.
Reported-by: Jörg Krause <joerg.krause@embedded.rocks>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit wires up the gcc version dependency mechanism in the
external toolchain backend. To do so, it:
* Changes the definition of all pre-defined external toolchain
profiles to select the appropriate BR2_TOOLCHAIN_GCC_AT_LEAST_*
option.
* For custom external toolchains, provides a visible Config.in
"choice" to select the gcc version used in the external toolchain.
* Adds a new check_gcc_version function, that verifies that the real
gcc version found in the external toolchain matches the one
declared in the Buildroot configuration.
[Thomas: use better sed expression proposed by Yann E. Morin, which
works with more cases.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This makes sure we don't have any weird permissions on the staging dir,
which could affect the target.
Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
check_arm_abi builds a test C file to check that the toolchain is
working correctly, with the output redirected to /dev/null.
However, some toolchains (OSELAS 2014.12.0, for instance) foolishly
append ".gdb" to the output filename for an intermediate file, causing
an attempt to write to /dev/null.gdb, which obviously fails.
Fix this by adding changing the output to a temporary file, which is
later removed along with any other "suffixed" files.
Suggested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we instruct users to enable/disable BR2_TOOLCHAIN_HAS_NATIVE_RPC
but that is a blind option. The only option users can set/unset is
BR2_TOOLCHAIN_EXTERNAL_INET_RPC.
Use that in the error message.
Notes: the only way for this message to appear is for a custom external
toolchain, either downloaded or pre-installed, so even though we check
the validity of the toolchain with BR2_TOOLCHAIN_HAS_NATIVE_RPC, we do
report on BR2_TOOLCHAIN_EXTERNAL_INET_RPC.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Remove BR2_INET_IPV6 select for predefined external toolchains.
Remove the (non)IPv6 option prompt since it's now mandatory.
And force the toolchain check now that internal uclibc is always built
with IPv6 support and external non-IPv6 toolchains are disallowed.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Distro toolchains, i.ie. toolchains coing with distributions, will
almost invariably be unsuitable for use with Buildroot:
- they are mostly non-relocatable;
- their sysroot is tainted with a lot of extra libraries.
Especially, the toolchains coming with Ubuntu (really, all the Debian
familly of distros) are configured with --sysroot=/ which makes them
non-relocatable, and they already contain quite some libraries that
conflict (in any combination of version, API or ABI) with what Buildroot
wants to build (i.e. extra libraries, some not even present in
Buildroot...) but also their mere preence when Buildroot does not expect
them to be already built (so that a package would enable features when
it should not).
So, try to detect those toolchains and black-list them; inform the user
that the toolchain is unusable for the reasons mentioned above.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As discussed on the mailing list drop the non-largefile option for
toolchains.
The size delta is minimal and it just complicates package dependencies.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently only check that the Buildroot configuration matches what is
available in the toolchain.
Since we're going to remove the check for LFS and make it a mandatory
feature, we will lose the corresponding buildroot option, so we won't be
able to use check_uclibc_feature as-is.
Introduce a magic value passed as the buildroot option name to recognise
checks for mandatory uclibc options that do not have a corresponding
option in buildroot.
If the buildroot option name is empty then the check is against a
mandatory uclibc option.
If a mandatory uclibc option is missing we reject the toolchain as being
unusable by buildroot.
[Thomas: minor tweaks in comment, remove space instead of tab.]
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since commit b2e88073db (toolchain: check ARM EABI vs. EABIhf for external
toolchains), check_arm_abi accepts a second parameter. Update the comment
accordingly.
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
