Currently, when verifying the configuration of a uClibc toolchain for
the presence of locale support, we check __UCLIBC_HAS_LOCALE__. It
turns out that we in fact also expect __UCLIBC_HAS_XLOCALE__ to be
defined, as without it locale_t is not defined, causing build failure
in some packages, such as libcpprestsdk:
In file included from /home/thomas/autobuild/instance-0/output-1/build/libcpprestsdk-2.10.18/Release/include/cpprest/json.h:18,
from /home/thomas/autobuild/instance-0/output-1/build/libcpprestsdk-2.10.18/Release/src/pch/stdafx.h:88,
from /home/thomas/autobuild/instance-0/output-1/build/libcpprestsdk-2.10.18/Release/src/http/client/http_client_msg.cpp:13:
/home/thomas/autobuild/instance-0/output-1/build/libcpprestsdk-2.10.18/Release/include/cpprest/asyncrt_utils.h:317:13: error: 'locale_t' does not name a type
317 | typedef locale_t xplat_locale;
| ^~~~~~~~
As essentially our requirement for uClibc in external toolchains is
"it should match the uClibc configuration used by Buildroot for
internal toolchains", it makes sense to verify
__UCLIBC_HAS_XLOCALE__. Note that of course checking
__UCLIBC_HAS_XLOCALE__ is sufficient, as it cannot be enabled if
__UCLIBC_HAS_LOCALE isn't.
This addresses an issue with the Synopsys ARC external toolchain,
which is built with __UCLIBC_HAS_LOCALE__, but without
__UCLIBC_HAS_XLOCALE__ causing a build failure with some
packages (such as libcpprestsdk).
Therefore, this patch also changes how the Synospys ARC external
toolchain is exposed in Buildroot: it no longer advertise locale
support.
Fixes:
http://autobuild.buildroot.org/results/e6778e60cc1ea455f5b4511d5824f04d8040f67b
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
check_fortran will display an error message if there is no Fortran
compiler in the toolchain. In the past, running into this error message
would be unlikely, since the Fortran test was only execuded when
BR2_TOOLCHAIN_HAS_FORTRAN was set.
However, since commit c7f641cbaa (toolchain/toolchain-external: always
call checks with dependencies), the Fortran check is unconditional. The
error itself benign, and the macro will handle the situation correctly,
since that is what it is designed to do. However, the error message
looks ugly and can be confusing.
[...]
>>> toolchain-external-custom Extracting
>>> toolchain-external-custom Patching
>>> toolchain-external-custom Configuring
/bin/bash: line 1: .../bin/aarch64-linux-gfortran: No such file or directory
>>> toolchain-external-custom Building
... everything continues normally ...
Let's suppress the error message, since triggerig the error is an
integral part of how the test works and doesn't mean that anything is
wrong.
Signed-off-by: Markus Mayer <mmayer@broadcom.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Previously, it was possible for external toolchains to be used that had
support for languages or libraries that Buildroot was not aware of.
If Buildroot is not made aware of this support, it will not know to copy
the requisite libraries into the filesystem.
This is problematic as packages may perform their own checks [0] to find
out what the toolchain supports and builds will link against libraries
from the toolchain but will be missing dependencies in the filesystem.
Now, the support helpers alert the user when a toolchain supports a
language or library that has not been set in the Buildroot configuration.
Also, while we're here, add `-ffree-form` to the Fortran check to
suppress a meaningless warning.
[0]: https://bugs.busybox.net/show_bug.cgi?id=15634
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since the commit [1], the utils/genrandconfig script improved the
configuration randomization used by autobuilders. Since then it can
generate a configuration that is not suitable for an external toolchain
such the "Codescape IMG GNU Linux Toolchain".
Indeed this toolchain can be selected for mips32r5 or mips64r5 while only
mips32r2 or mips64r2 are really supported. The toolchain issue will be
fixed in a followup change.
We want to catch such issue in check_unusable_toolchain function otherwise
it is detected late during the sysroot import into staging and trigger
a weird error message:
ln: failed to create symbolic link 'output/host/mips64el-buildroot-linux-gnu/sysroot//nvmedata/autobuild/instance-25/buildroot/libc.a': No such file or directory
ln: failed to create symbolic link 'output/host/mips64el-buildroot-linux-gnu/sysroot/usr//nvmedata/autobuild/instance-25/buildroot/libc.a': No such file or directory
This is similar test than for the main sysroot check but this time we have
to use the toolchain cflags to check the architecture sysroot.
If the architecture sysroot doesn't exist, the toolchain will reply with
"libc.a".
Either the toolchain is really broken or we used a wrong target
architecture variant. In the later case, the toolchain infrastructure will
print a meaningful error message.
Note: We also may get a similar issue using the toolchain-external-custom package
if a toolchain is used with a wrong target architecture variant.
Fixes:
http://autobuild.buildroot.org/results/701/701e8a5f713f7bdd1f32a4c549cdaac580e2522a/
[1] aeee90ec10
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
copy_toolchain_lib_root was not handling the case of "readlink"
returning nothing, which will happen if the symlink it is trying to
resolve does not point to a valid file on the build host. This
shouldn't happen, but it can.
The end result of this situation would be an endless loop of error
messages that would only end if aborted manually.
[...]
cp: missing destination file operand after
'/local/users/mmayer/buildroot/output/arm64/target//'
Try 'cp --help' for more information.
readlink: missing operand
Try 'readlink --help' for more information.
basename: missing operand
Try 'basename --help' for more information.
dirname: missing operand
Try 'dirname --help' for more information.
^C
make[1]: *** [package/pkg-generic.mk:384:
Instead of looping endlessly without explanation, let's abort and
inform the user that something seems amiss with their setup.
Signed-off-by: Markus Mayer <mmayer@broadcom.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When changing permissions on all directories in the staging directory,
after copying sysroot, paths that contain spaces break the call to
chmod.
With -print0 for find and -0 for xargs white spaces are correctly
interpreted.
Signed-off-by: Daniel Lang <d.lang@abatec.at>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The gdbinit supplied by Buildroot does two things:
A. specify the sysroot where gdb can find shared libraries
B. mark the sysroot as a 'safe path' for its auto-load feature, to make sure
that pretty printers for libstdc++.so are added automatically (see commit
6fb3216a80)
When debugging a core file, and the gdbinit file is specified via '-x'
rather than '-ix', then the order of these settings matters: If you first
set the sysroot, then gdb will immediately start finding the shared
libraries it needs for the core file, detect libstdc++ and its associated
libstdc++-gdb.py file, then give a big warning about safe paths:
warning: File ".../i686-buildroot-linux-gnu/sysroot/lib/libstdc++.so.6.0.24-gdb.py"
auto-loading has been declined by your `auto-load safe-path' set
to "$debugdir:$datadir/auto-load".
To enable execution of this file add
add-auto-load-safe-path .../i686-buildroot-linux-gnu/sysroot/lib/libstdc++.so.6.0.24-gdb.py
line to your configuration file "/home/me/.gdbinit".
To completely disable this security protection add
set auto-load safe-path /
line to your configuration file "/home/me/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
info "(gdb)Auto-loading safe path"
and the pretty printing code is not loaded. This is because the second
line from the gdbinit file was not yet parsed at this point.
By changing the order (first configuring the safe path, then setting the
sysroot), this issue does not appear and everything is as expected.
Note that when '-ix' were used instead of '-x' to pass the gdbinit file to
gdb, then the order would not matter, because the entire gdbinit file would
be parsed before considering the core file.
However, even though the Buildroot manual now suggests '-ix', users may not
have noticed this change and continue to use '-x'.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Some 3rd party vendor toolchains have multiple files which match
these glob patterns. In this case, the shell script failed.
Switching to use find and xargs solves the issue.
Signed-off-by: Jonah Petri <jonah@petri.us>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
gdb can automatically load certain files as described in [1]. Such files
could install pretty-printers for complex data structures.
libstdcxx (C++ standard library) provided by gcc, is one example of a
library for which such auto-load file is available. But there are other
examples too, like libglib2.
However, gdb will only auto-load files if the file is located in one of the
locations treated as 'safe'. The Buildroot sysroot is not by default in that
list.
Provide a better debugging experience by adding the sysroot to the 'safe'
list, via the gdbinit file prepared by Buildroot.
[1] https://sourceware.org/gdb/onlinedocs/gdb/objfile_002dgdbdotext-file.html
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
A gcc compiler, which was configured with
--with-gcc-major-version-only, will only return a single
number. (debian does this for example).
A simple modification allows the check to work with both
single numbers (eg. '9') and full versions (eg. '9.2.1').
Signed-off-by: Norbert Lange <nolange79@gmail.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
make-4.3 shipped with a backward incompatible change in how sharp signs
are handled in macros. Previously, up to make 4.2, the sharp sign would
always start a comment, unless backslash-escaped, even in a macro or a
fucntion call.
Now, the sharp sign is no longer starting a comment when it appears
inside such a macro or function call. This behaviour was supposed to be
in force since 3.81, but was not; 4.3 fixed the code to match the doc.
As such, use of external toolchains is broken, as we use the sharp sign
in the copy_toolchain_sysroot macro, in shell variable expansion to
strip off any leading /: ${target\#/}.
Fix that by applying the workaround suggested in the release annoucement
[0], by using a variable to hold a sharp sign.
[0] https://lists.gnu.org/archive/html/info-gnu/2020-01/msg00004.html
Signed-off-by: Yaroslav Syrytsia <me@ys.lc>
[yann.morin.1998@free.fr:
- move the SHARP_SIGN definition out of Makefile and into support/
- expand the commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When Buildroot is released, it knows up to a certain kernel header
version, and no later. However, it is possible that an external
toolchain will be used, that uses headers newer than the latest version
Buildroot knows about.
This may also happen when testing a development, an rc-class, or a newly
released kernel, either in an external toolchain, or with an internal
toolchain with custom headers (same-as-kernel, custom version, custom
git, custom tarball).
In the current state, Buildroot would refuse to use such toolchains,
because the test is for strict equality.
We'd like to make that situation possible, but we also want the user not
to be lenient at the same time, and select the right headers version
when it is known.
So, we add a new Kconfig blind option that the latest kernel headers
version selects. This options is then used to decide whether we do a
strict or loose check of the kernel headers.
Suggested-by: Aaron Sierra <asierra@xes-inc.com>
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- only do a loose check for the latest version
- expand commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In commit 6136765b23 ("toolchain:
generate check-headers program under $(BUILD_DIR)"), the
check_kernel_headers_version function was simplified to not check the
return value of the check-kernel-headers.sh script, assuming that
"make" does bail out on the first failing command.
However, check_kernel_headers_version when used in $(2)_CONFIGURE_CMDS
from pkg-toolchain-external.mk, is called in a sequence of commands,
where the return value of each command is not checked. Therefore, a
failure of check-kernel-headers.sh no longer aborts the build.
Since all other macros are using this principle of calling "exit 1",
we revert back to the same for check_kernel_headers_version, as it was
done prior to 6136765b23.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Carlos Santos <unixmania@gmail.com>
Reviewed-by: Carlos Santos <unixmania@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Some installations mount /tmp with the 'noexec' option, which prevents
running the program generated there to check the kernel headers.
Avoid the problem by generating the program under $(BUILD_DIR), passed
as the first argument to check-kernel-headers.sh.
We could globally export a TMPDIR environment variable with some path
under $(BUILD_DIR) but such solution would be too intrusive, depriving
the user from the freedom to set TMPDIR at his will (or needs).
Fixes: https://bugs.busybox.net/show_bug.cgi?id=12241
Signed-off-by: Carlos Santos <unixmania@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some toolchain vendors may have backported those options to older gcc
versions, and we have no way to know, so we have to check that the
user's selection is acceptable.
Extend the macro that currently checks for SSP in the toolchain, with
a new test that the actual SSP option is recognised and accepted.
Note that the SSP option is either totaly empty, or an already-quoted
string, so we can safely and easily assign it to a shell variable to
test and use it.
Note that we do not introduce BR2_TOOLCHAIN_HAS_SSP_STRONG, because:
- our internal toolchain infra only supports gcc >= 4.9, so it has
SSP strong;
- of the external pre-built toolchains, only the codesourcery-arm
one has a gcc-4.8 which lacks SSP strong, all the others have a
gcc >= 4.9;
- we'd still have to do the actual check for custom external
toolchains anyway.
So, we're not adding BR2_TOOLCHAIN_HAS_SSP_STRONG just for a single
case.
Signed-off-by: "Yann E. MORIN" <yann.morin@orange.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The check_musl function currently builds a program and verifies if the
program interpreter starts with /lib/ld-musl. While this works fine
for dynamically linked programs, this obviously doesn't work for a
purely static musl toolchain such as [1].
There is no easy way to identify a toolchain as using the musl C
library. For glibc, dynamic linking is always supported, so we look at
the dynamic linker name. For uClibc, there is a distinctive
uClibc_config.h header file. There is no such distinctive feature in
musl.
We end up resorting to looking for the string MUSL_LOCPATH, which is
used by musl locale_map.c source file. This string has been present in
musl since 2014. It certainly isn't a very stable or convincing
solution to identify the C library as being musl, but it's the best we
could find.
Note that we are sure there is a libc.a file, because the
check_unusable_toolchain function checks that there is a such a file.
[1] http://autobuild.buildroot.net/toolchains/tarballs/br-arm-musl-static-2018.05.tar.bz2
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When using an external toolchain that was built with Buildroot and a
merged /usr, the dynamic linker is actually in /usr/lib.
But the check_glibc macro limits the depth it is looking for the dynamic
linker, and misses it when it is in /usr/lib because it is too deep.
We could fix that in two ways: increase the depth in which we look
for it, or follow symlinks. We choose the second solution.
Signed-off-by: Cam Hutchison <camh@xdna.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In commit 32bec8ee2f
("toolchain-external: copy ld*.so* for all C libraries") we changed
how the musl dynamic linker symbolic link was being created. Instead
of having specific logic in Buildroot, we switched to simply copying
the ld*.so.* symbolic link from staging to target, as well as the
target of this symbolic link.
However, it turns out that by default, musl creates its dynamic linker
symbolic link with an absolute path as the target of the link:
/lib/libc.so.
Therefore, external Musl toolchains built with Buildroot look like
this:
lrwxrwxrwx 1 thomas thomas 12 Jul 4 19:46 ld-musl-armhf.so.1 -> /lib/libc.so
The principle of the copy_toolchain_lib_root function, which is used
to copy libraries from staging to target, is to copy symbolic links
and follow their targets. In this case, it means we end up copying
/lib/libc.so (from the host machine) into the target folder. From
there on, there are two cases:
1. /lib/libc.so exists in your host system. It gets copied to the
target. But later on, Buildroot also copies /lib/libc.so from
staging to target, overwriting the bogus libc.so. So everything
works fine, even though it's admittedly ugly.
2. /lib/libc.so doesn't exist in your host system. In this case, the
build fails with no clear error message.
This problem does not happen with Musl toolchains built by
Crosstool-NG, because Crosstool-NG replaces the absolute target of the
dynamic linker symbolic link by a relative path.
However, since we want to support existing Buildroot Musl toolchains
and generally work with the fact that Musl by default installs an
absolute symlink, the following commit improves the
copy_toolchain_sysroot function to replace symbolic links with an
absolute destination to use a relative destination. I.e, in staging,
the ld-musl-armhf.so.1 symbolic link looks like this:
lrwxrwxrwx 1 thomas thomas 14 Jul 5 22:59 output/staging/lib/ld-musl-armhf.so.1 -> ../lib/libc.so
Fixes:
http://autobuild.buildroot.net/results/ce80264575918a8f71d9eab1091c21df85b65b1a/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In commit b3cc7e65ee, the definition of the DESTDIR variable was moved
down into the loop that follows symlinks in the libraries that are
copied to target. However, the corresponding mkdir was not moved down,
so that no directories are ever created.
In practice, this mkdir is normally redundant since the directories
should already have been created as part of creating STAGING_DIR.
Still, the current situation is clearly wrong, so fix it by moving the
mkdir down to after the assignment to DESTDIR.
While we're at it, also remove a redundant empty line. It's a leftover
from when a lot of variables were declared above.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In copy_toolchain_sysroot, if no ld.so has been found in the
STAGING_DIR after the sysroot copy, we look at ARCH_SYSROOT_DIR if a
ld.so is available there. We do this for both ld*.so and ld*.so.*.
However, when copying thing from staging to target (as listed in
TOOLCHAIN_EXTERNAL_LIBS), we only match on ld*.so.*. This would mean
that even if a dynamic linker matching ld*.so but not ld*.so.* was
copied into staging by copy_toolchain_sysroot, it would anyway not be
copied to the target filesystem, making the system unusable.
Therefore, we can remove the special case on ld*.so, and keep only
ld*.so.*.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root copies libraries from staging to target,
resolving and copying symbolic links along the way.
The most inner loop, a "while" loop, starts from an initial name, and
if it's a symbolic link, gets resolved to the target, and the loop
iterates until we reach a real file. However, the destination folder
where the symbolic link or real file is created is computed in DESTDIR
only once, before this loop starts. Therefore, this loop works fine
when all symbolic links in the chain, and the real file all belong to
the same directory. But it doesn't do the correct thing when the
symbolic link and/or real file are in different folder.
An example is Crosstool-NG musl toolchains, where the dynamic loader
is in /lib/ld-musl*.so but points to ../usr/lib/libc.so. With the
current logic, we copy /lib/ld-musl*.so to /lib, but we also copy
libc.so to /lib instead of the expected /usr/lib.
This currently doesn't cause any problem because the musl dynamic
linker is manually created by the TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
hook. However, this logic has a number of problems, so in a followup
commit, we are going to put the musl dynamic linker in
TOOLCHAIN_EXTERNAL_LIBS, which will cause it to be copied by
copy_toolchain_lib_root. But we obviously want the link and its target
to be copied to the right place, hence this fix.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit ba6bac1383 made a change in copying of
the dynamic loader, with the goal of reducing toolchain-specific fixups.
Any ld*.so file found in the toolchain's lib directory would be copied to
the staging/lib directory.
For the toolchains that previously needed fixup, this new behavior is fine.
The reason they needed fixup was that the normal copy action did not include
any dynamic loader.
However, for certain other toolchains this new behavior actually breaks
things: regardless of ARCH_LIB_DIR, which may be lib64 instead of lib, the
dynamic loader from lib is copied _over_ any previously correct dynamic
loader.
This has been witnessed with the CodeSourcery x86_64 and
CodeSourcery MIPS64 toolchains. In both cases, a 32-bit dynamic loader was
copied to staging/lib, while a 64-bit version was expected.
Fix the problem by only performing this explicit dynamic loader copy if no
dynamic loader is found in staging/lib.
Fixes http://autobuild.buildroot.net/results/8bf/8bffe54032aad9cc710a22411ef3bff4a2c93e55/
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root handles symlinks by recreating them, disregarding
the original destination and assuming the destination is in the same
directory as the link itself.
When a library link points to the real library file in another directory,
for example:
usr/lib/octeon2/libcrypt.so -> ../../../lib32/octeon2/libcrypt.so.1
then the link created by copy_toolchain_lib_root is broken.
It is more robust to copy the symlink to keep the destination intact. The
destination path should be present, possibly through other symbolic links.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The input to copy_toolchain_lib_root is not one library, not a list of
libraries, but a library name pattern with glob wildcards.
This pattern is then passed to 'find' to get the actual list of libraries
matching the pattern. Reflect this using an appropriate variable name.
Note: if the root of the buildroot tree contains a file matching one of
these library patterns, the copying of libraries from staging to target will
not be correct. It is not impossible to fix that, e.g. using 'set -f', but
maybe it's not worth it.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
copy_toolchain_lib_root has slightly different logic depending on the type
of library object: file or link. All actions related to links are not
relevant in case you are working with a file. Hence, try to increase clarity
by not executing unnecessary lines in the 'file' case.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A previous commit rewrote broken symbolic links in staging, caused by a
non-singular ARCH_LIB_DIR. In this case, the symbolic links are typically
using one or more intermediate directory symlinks, which can be simplified
using the newly introduced simplify_symlink helper.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The external toolchain logic flattens the directory layout in the staging
directory. Regardless of the number of levels present in the extracted
toolchain, libraries are always copied to lib/ and usr/lib/, and directory
symbolic links are provided to make the original paths available as well.
Due to this, the same library may be reachable through a number of paths:
one path without any symbolic link, and one or more paths using directory
symlinks.
Using a direct path in a symlink destination is generally preferred because
it is clearer, but it is also more robust against accidental removal of an
intermediate directory symlink.
Introduce a helper function to simplify a symlink's destination to such a
direct path. This function will be used in a subsequent patch.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain from the Cavium Octeon SDK has a sysroot layout as follows:
./lib32
./lib32/octeon2
./lib32-fp
./lib64
./lib64/octeon2
./lib64-fp
./usr
./usr/lib
./usr/lib32
./usr/lib32/octeon2
./usr/lib32-fp
./usr/lib64
./usr/lib64/octeon2
./usr/lib64-fp
./usr/bin
./usr/bin32
./usr/bin32-fp
./usr/bin64-fp
./usr/libexec
./usr/libexec32
./usr/libexec32-fp
./usr/libexec64-fp
./usr/sbin
./usr/sbin32
./usr/sbin32-fp
./usr/sbin64-fp
./usr/include
./usr/share
./sbin
./sbin32
./sbin32-fp
./sbin64-fp
./etc
./var
with the following selections:
- lib64 : default
- lib64/octeon2 : -march=octeon2
- lib64-fp : -march=octeon3
- lib32 : -mabi=n32
- lib32/octeon2 : -mabi=n32 -march=octeon2
- lib32-fp : -mabi=n32 -march=octeon3
In case of '-mabi=n32 -march=octeon2' (but same is true for n64+octeon2)the
original Buildroot toolchain logic would copy both the libraries in
lib32 as the subdirectory lib32/octeon2, which means that every library is
installed twice (but only one of each is really needed).
While ARCH_LIB_DIR is determined by the location of libc.a, which in this
case is effectively:
<sysroot>/usr/lib32/octeon2/libc.a
the variable only retains 'lib32' and not 'lib32/octeon2' as expected.
To make Buildroot cope with this style of toolchain layout, we need to adapt
the calculation of ARCH_LIB_DIR to also include the second part.
This, in turn, means that ARCH_LIB_DIR is no longer guaranteed to be a
singular path component, resulting in some additional changes.
Certain older Linaro toolchains actually had the same layout. Libraries were
located in lib/<tuple> rather than lib directly. Previously, this was
handled by adding a toolchain-specific fixup that creates a symlink
lib/<tuple> -> lib, but with this patch this would no longer be needed.
Note that one difference with the Octeon case is that these Linaro
toolchains are not actually multilib, i.e. there is just one location with
the libraries and thus there is no problem with duplicated libraries.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The helper function copy_toolchain_sysroot has some logic to transform a
path into a number of '../' components based on the depth of that path.
As this same logic will be needed in another place in a subsequent patch,
extract it into a separate helper relpath_prefix.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Normally, the Buildroot toolchain logic copies all required libraries from
the external toolchain to the staging directory, including the dynamic
loader ld-*.so.
There are cases, however, where the dynamic loader is _not_ automatically
copied to staging. This happens when the dynamic loader is not inside
ARCH_LIB_DIR itself (e.g. lib64), but instead resides in 'lib' (assume, of
course, that ARCH_LIB_DIR != 'lib').
Currently, this is fixed in a toolchain-specific fixup, e.g. by recreating a
missing symlink or copying over a missing file. Such toolchain specific
fixups are not very nice.
Moreover, in a subsequent patch, the value of ARCH_LIB_DIR changes for some
toolchains, causing them to have the same problem of a missing dynamic
loader. This used to be the case for older Linaro toolchains with libraries
in 'lib/<tuple>': Buildroot used to set ARCH_LIB_DIR=lib but the mentioned
patch changes it to 'lib/<tuple>' instead. As a result, the files directly
under 'lib/' will no longer be copied. There should be none, but the dynamic
loader is a notable exception.
[Note: support for these older Linaro toolchain has been removed in 2016.11]
Instead, copy over the ld.so file(s)/link(s) from the extracted toolchain
into staging, in the central copy_toolchain_sysroot function. The existing
toolchain logic will then handle the copy of these files from staging to
target.
This means the toolchain-specific fixups can be removed.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The copy_toolchain_sysroot helper features a complex rsync loop that copies
various directories from the extracted toolchain to the staging directory.
The complexity mainly stems from the fact that we support multilib toolchain
tarballs but only copy one of the multilib variants into staging.
Increase understandability of this logic by explicitly restricting the
rsync excludes to the iteration of the for loop they are relevant for.
Additionally, update the function comment.
Note: all attempts to reduce duplication between both rsync while keeping
things nice and readable failed. One has to be extremely careful regarding
line continuation, indentation, and single vs double quoting. In the end, a
split up rsync seemed most clean.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check_arm_abi function takes as second argument the path to the
cross-readelf, but does not use it. Therefore, this commit gets rid of
this unnecessary argument.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The current test to verify if the toolchain uses musl or not is based on
checking if /lib/libc.so or /lib/libm.so exist in the sysroot. However,
some toolchains (notably Crosstool-NG ones) put these libraries in
/usr/lib/.
To fix this, build a minimal C program and check if the program
interpreter contains /lib/ld-musl.
Signed-off-by: Ilya Kuzmich <ilya.kuzmich@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The somewhat complicated sed expression has been removed in commit
06cd604ec6 ("toolchain/external: use
-dumpversion to check gcc version"), so let's remove the comment that
was explaining this sed expression.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, `--version` option is used and later matched with a regex to get
the actual gcc version. There's a dedicated gcc option to do exactly that:
`-dumpversion`.
Also `--version` may return a string customised by a vendor that provides
the toolchain, which makes the current regex approach error prone. In
fact, this situation has been seen with a real customised toolchain.
Signed-off-by: Krzysztof Konopko <kris@youview.com>
Signed-off-by: Tomasz Szkutkowski <tomasz.szkutkowski@youview.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Function copy_toolchain_sysroot, which is in charge of copying the relevant
bits from the external toolchain to the staging directory, performs an rsync
loop of various directories and excludes the pattern 'usr/lib/locale' with
the intention of skipping the directory <toolchain>/usr/lib/locale.
However, while this worked in the original commit, commit
5628776c4a broke it inadvertently. The
relevant part of the diff:
- rsync -au --chmod=Du+w --exclude 'usr/lib/locale' \
- $${ARCH_SYSROOT_DIR}/$$i $(STAGING_DIR)/ ; \
+ rsync -au --chmod=Du+w --exclude 'usr/lib/locale' \
+ --exclude lib --exclude lib32 --exclude lib64 \
+ $${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
Notice how the source directory now contains a trailing slash, which impacts
the way the exclude rules are interpreted. Previously, when 'i' was 'usr',
the exclude of 'usr/lib/locale' would find a match. With the trailing slash,
there will never be a match, unless for a directory 'usr/usr/lib/locale'.
The right rule would have been '--exclude lib/locale'.
However, just that fix does not solve the problem in all cases, in
particular in the (common) case where ARCH_LIB_DIR is 'lib'. This is due
another change in that commit, changing the iterated values of the above
rsync:
- for i in etc $${ARCH_LIB_DIR} sbin usr ; do \
+ for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
Due to the fact that we rsync both 'usr' as 'usr/lib' (assuming ARCH_LIB_DIR
is 'lib') we need to add the correct exclude in both cases. But the exclude
is different for both. When i == 'usr', the correct exclude rule would be
'--exclude lib/locale' while when i == 'usr/lib' the correct rule would be
'--exclude locale'.
Since we would like to avoid separate cases for this, use the following
exclude: '--exclude locale/'. The trailing slash will make sure only
directories called 'locale' will match. The targeted directories are then
usr/lib/locale and usr/share/locale. The latter directory was not matched
originally, but it should not hurt changing that.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As discussed with Thomas Petazzoni, we can reduce the nesting level by early
returning on an invalid iteration.
I did not move the 'else' case (the common case) outside the if-else because
it would make the code less symmetrical and IMO makes it _less_ clear.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
We currently support gcc as old as 4.3. However, Buildroot works
perfectly well with even older gcc versions (tested with 4.1). So we
can add an option BR2_TOOLCHAIN_EXTERNAL_GCC_OLD to support that. The
help text of this option is written with plenty of discouragement.
We use _OLD and not something like _PRE_4_3, because at some point we
will likely remove the 4.3 option and what would then require a name
change.
We don't set any _AT_LEAST option in this case because it's no use -
there is no lower bound on the version in this case. We therefore leave
BR2_TOOLCHAIN_GCC_AT_LEAST empty (the implicit default). When it is
empty, we don't do a version check at all in check_gcc_version
(previously we errored out when it was empty).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
[Thomas: remove extension for the generated temporary file, since it's
really an executable, not an object file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This reverts commit 9a1e9efe26.
Currently Codescape toolchains cannot be used to generate a big endian
root file system because the support for side by side sysroots is not
complete.
There is a patch [1] waiting in the queue which fixes the issue for the
current version of Codescape toolchains we have, but it will not work
for the next one that is coming. So, instead of messing more with the
toolchain infra I think it's better to handle these specific Codescape
toolchain's weirdness in hooks which won't affect others.
[1]: http://patchwork.ozlabs.org/patch/571708/
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In 2a87b64 (toolchain-external: align library locations in target and
staging dir), copying the libraries from the sysroot to the target was
changed to a simple find-based solution.
To be sure that the staging directory was entered to find the libraries,
in case the variable was pointing to a symlink, the -L clause to find
was used.
However, that causes extraneous libraries to be copied over.
For example, a ct-ng toolchain would have this sysroot (e.g for an arm
32-bit toolchain):
.../sysroot/lib/
.../sysroot/lib32 -> lib
.../sysroot/lib64 -> lib
.../sysroot/usr/lib/
.../sysroot/usr/lib32 -> lib
.../sysroot/usr/lib64 -> lib
Which we would carry as-is to our own sysroot.
But then, in target, our skeleton creates the /lib/ and /usr/lib
directories, with the necessary lib32 or lib64 symlink pointing to it.
In this case, a lib32->lib symlink is created, but no lib64 symlink
since this is a 32-bit architecture.
To copy the required libraries from staging into target, we scan the
staging directory for all occurences of the required libraries, and copy
them over to target, keeping the same directory layout relative to the
sysroot.
For example:
.../sysroot/usr/lib/libfoo.so --> .../target/usr/lib/libfoo.so
.../sysroot/usr/lib32/libbar.so --> .../target/usr/lib32/libbar.so
.../sysroot/usr/lib64/libbuz.so --> .../target/usr/lib64/libbuz.so
So, when we copy over the libraries from our staging to the target
directory, the "find -L .../sysroot -name libblabla.so.*" would find
multiple instances of libblabla, each in the /usr/lib /usr/lib32 and
/usr/lib64 locations (they are all the exact same file, though).
Since we do have the /usr/lib32->lib symlink, all is OK (but there are
two copies going on, which could be avoided). However, since we do not
have the /usr/lib64->lib symlink, the /usr/lib64/ directory is created.
This was very difficult to observe, as no /lib64/ directory is created,
only the /usr/lib64/ one was. To top it off, this only happens with a
merged /usr, which does not seem like not a common case without systemd.
Since the reason to use -L was to be sure to enter our staging
directory, we just need to ensure that the path ends up with a slash, as
was already talked about in this thread:
http://lists.busybox.net/pipermail/buildroot/2016-April/159737.html
After further discussion, it turns out that the original patch came along
because of the confusion between output/staging (which is a symlink) and
$(STAGING_DIR) which expands to output/host/usr/<tupple>/sysroot (which is
never a symlink), so the symlink handling isn't really needed at all.
[Peter: drop description comment, extend description]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Don't enable SSP support on external toolchains just because they use
glibc or musl. Instead of that, make the external toolchains explictily
declare if they support SSP or not. And also add a check to detect SSP
support when using custom external toolchains.
For internal toolchains we always enable SSP support for glibc and musl.
Fixes:
http://autobuild.buildroot.net/results/ac7c9b3ad2e52abfe6b79a80045e4218eeb87175/
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
[Thomas:
- remove uClibc-specific SSP check, since there is now a generic
check being done.
- send potential compilation errors caused by the SSP check to
oblivion, in order to avoid causing confusion for the user.
- add autobuilder reference.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The sysroot toolchain support check is duplicated at two locations in
the external toolchain infra. So move it inside the
check_unusable_toolchain helper that is called when the toolchain
package is configured (TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS).
The check in TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS can be safely
removed since it's already done in check_unusable_toolchain helper.
The check in TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS was removed by
2a87b64f8e.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some toolchain can't be used by Buildroot due to sysroot location
issue, so the $(ARCH)-linux-gnu-gcc -print-file-name=libc.a command
return only "libc.a"
This lead to an error during the header check version helper,
so these toolchains can't be imported into Buildroot.
cc1: fatal error: $PWD/libc.a/usr/include/linux/version.h: No such file or directory
compilation terminated.
support/scripts/check-kernel-headers.sh: line 38: /tmp/check-headers.4V5PPF: Permission denied
This issue happen with the first linaro 2015.11 [1] release and
CodeSourcery standard edition [2].
Here is the sysroot directory tree for linaro 2015.11:
$ ls libc/arm-linux-gnueabihf
etc lib sbin usr var
Here is the sysroot directory tree for CodeSourcery standard:
$ ls libc/sgxx-glibc
etc lib lib64 sbin usr var
Add a check to error out with an explicit error message
The check don't use toolchain_find_libc_a function directly since
"realpath -f" is used internally and return an absolute path.
[1] https://bugs.linaro.org/show_bug.cgi?id=1995#c7
[2] http://lists.busybox.net/pipermail/buildroot/2014-October/110696.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The toolchain-external logic is roughly:
- populate the staging dir by rsyncing the entire ${ARCH_LIB_DIR} and
usr/${ARCH_LIB_DIR} from sysroot.
- populate the target dir by explictly copying some libraries from sysroot
into target/lib and some other libraries in target/usr/lib, the split
being hardcoded into buildroot regardless of the location in the sysroot.
This means that a library libfoo could be located in:
staging/lib/libfoo.so
target/usr/lib/libfoo.so
When debugging an application that links against this library, gdb will
fruitlessly search for 'usr/lib/libfoo.so' in staging, and then suggest to
use 'set solib-search-path' which is a hack, really.
To solve the problem, we need to make sure that libraries from the toolchain
are installed in the same relative location in staging and target.
Achieve this by:
- replacing the convoluted search for libraries using for+find in sysroot
with a simple find in staging.
- determining DESTDIR for each library individually based on the location in
staging.
- treating LIB_EXTERNAL_LIBS and USR_LIB_EXTERNAL_LIBS equivalently
These changes also allow for the removal of most arguments to
copy_toolchain_lib_root in the method itself and their callers.
Test procedure:
- set configuration for a given toolchain
- make clean toolchain
- find output/target | sort > /tmp/out-before
- apply patch
- make clean toolchain
- find output/target | sort > /tmp/out-after
- diff -u /tmp/out-before /tmp/out-after
The only changes should be some libraries moving from lib to usr/lib or vice
versa. Notable examples being libstdc++ and libatomic.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas:
- use -L instead of -follow in the find invocation, as suggested by
Arnout.
- move the BR2_STATIC_LIBS condition as a make condition rather than
a shell condition, as suggested by Arnout.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
