Commit Graph

1010 Commits

Author SHA1 Message Date
Carlos Santos
dc7c6487cf Makefile: check rootfs overlays with BR2_ROOTFS_MERGED_USR enabled
Add a step to target-finalize that checks each rootfs overlay, following
the criteria established for custom skeletons and using the same script
uesd by skeleton-custom.mk.

Add a paragraph to the documentation clarifying that rootfs overlays
don't need to contain /bin, /lib or /sbin and must not contain them when
BR2_ROOTFS_MERGED_USR is enabled.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 17:09:28 +01:00
Joel Carlson
8fed162987 core/sdk: don't mangle symlinks with '.' or '..' at start
The current transform changes any '.' at the start of a filename to
$(BR2_SDK_PREFIX). This also applies to the target of a symlink, when
it is relative.

We thus might end up with something like:
    $(BR2_SDK_PREFIX)/bin/aarch64-linux-gnu-ar ->
    $(BR2_SDK_PREFIX)./opt/ext-toolchain/bin/aarch64-linux-gnu-ar

when it should be:
    $(BR2_SDK_PREFIX)/bin/aarch64-linux-gnu-ar ->
    ../opt/ext-toolchain/bin/aarch64-linux-gnu-ar

We fix that by making sure we always remove a known prefix, i.e. we
remove the path to host dir. The obvious solution would be to cd into
$(HOST_DIR)/.. , then tar ./host/ and finally use a --transfrom pattern
as 's,^\./$(notdir $(HOST_DIR)),$(BR2_SDK_PREFIX)'.

Since $(HOST_DIR) can point to a user-supplied location, we don't know
very well how the pattern may patch.

Instead, we cd into / and tar the full path to $(HOST_DIR).

Since tar removes any leading '/', it would spurr a warning message,
which is annoying. So we explicitly remove the leading '/' from
$(HOST_DIR) when we tar it.

Finally, we transform all filenames to replace a leading $(HOST_DIR)
(without a leading /) to the prefix to use.

Signed-off-by: Joel Carlson <JoelsonCarl@gmail.com>
[yann.morin.1998@free.fr:
  - use a single transform pattern
  - use full HOST_DIR path as pattern to replace
  - update commit log accordingly
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-02-06 16:30:14 +01:00
Thomas De Schampheleire
d3e535a839 Makefile: unexport 'PLATFORM' and 'OS' environment variables
Some package builds may fail when environment variables are present with the
same names as make variables in a package. This is a bigger problem for
environment variables with generic names, like 'PLATFORM' and 'OS'.

'PLATFORM' is for example a problem for host-acl.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-05 15:22:30 +01:00
John Keeping
e5e0f637ab Makefile: respect strip exclusions for special libraries
ld-*.so and libpthread*.so* are not stripped in the same way as other
binaries because some applications need symbols in these libraries in
order to operate correctly.

However, the special handling for these binaries ignores the usual
BR2_STRIP_EXCLUDE_* rules so it is not possible to build an image which
has debugging symbols in these binaries.

Pull out the common find functionality so that we can build two find
commands that re-use the common exclusion rules.

Fix-suggested-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: John Keeping <john@metanate.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-02-04 16:56:01 +01:00
Vivien Didelot
45f0395971 Makefile: add update-defconfig target
For symmetry with the Kconfig-based packages offering comprehensive
targets like linux-update-defconfig, barebox-update-defconfig and so
on, add a new top level update-defconfig target to run savedefconfig.

Signed-off-by: Vivien Didelot <vivien.didelot@gmail.com>
Reviewed-by: Titouan Christophe <titouan.christophe@railnova.eu>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-04 14:35:17 +01:00
Thomas Petazzoni
a86b626b5b Makefile: move definition of TARGET_DIR inside .config condition
In a follow-up commit introducing per-package directory support, we
will need to define TARGET_DIR in a different way depending on the
value of a Config.in option. To make this possible, the definition of
TARGET_DIR should be moved inside the BR2_HAVE_DOT_CONFIG condition.

We have verified that $(TARGET_DIR) is only used within the
BR2_HAVE_DOT_CONFIG condition. Outside of this condition, such as in
the "clean" target, $(BASE_TARGET_DIR) is used.

Suggested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-17 22:38:52 +01:00
Peter Korsgaard
8e928a8389 Makefile, manual, website: Bump copyright year
Happy 2019!

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-01-06 21:30:34 +01:00
Adam Duskett
bbf32a77ec utils/test-pkg: force checking dependencies
Currently, if a user runs "make" while specifying a specific package
(IE: make -p foo),  the Makefile logic skips checking to see if all the
dependencies are selected in the specified packages config file. This behavior
is useful to test simple packages which do not have "complex" dependencies.

However; if a developer uses test-pkg -p ${package_name} to check their package,
the package may pass all the checks, but would have otherwise failed with a
simple "make" because the developer may have failed to add a select line in
packages config file, even if there is a new dependency in the packages
Makefile.

Pass the environment variable "BR_FORCE_CHECK_DEPENDENCIES"  to the Makefile in
the test-pkg script,  and check it's value in the Makefile. If the value is
"YES" force checking for dependency issues.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-01-03 11:41:26 +01:00
Ricardo Martincoski
e7e30455ef Makefile: offload .gitlab-ci.yml generation
GitLab has severe limitations imposed to triggers.
Using a variable in a regexp is not allowed:
|    only:
|        - /-$CI_JOB_NAME$/
|        - /-\$CI_JOB_NAME$/
|        - /-%CI_JOB_NAME%$/
Using the key 'variables' always lead to an AND with 'refs', so:
|    only:
|        refs:
|            - branches
|            - tags
|        variables:
|            - $CI_JOB_NAME == $CI_COMMIT_REF_NAME
would make the push of a tag not to trigger all jobs anymore.
Inheritance is used only for the second level of keys, so:
|.runtime_test: &runtime_test
|    only:
|        - tags
|tests.package.test_python_txaio.TestPythonPy2Txaio:
|    <<: *runtime_test
|    only:
|        - /-TestPythonPy2Txaio$/
would override the entire key 'only', making the push of a tag not to
trigger all jobs anymore.

So, in order to have a trigger per job and still allow the push of a tag
to trigger all jobs (all this in a follow up patch), the regexp for each
job must be hardcoded in the .gitlab-ci.yml and also the inherited
values for key 'only' must be repeated for every job.
This is not a big issue, .gitlab-ci.yml is already automatically
generated from a template and there will be no need to hand-editing it
when jobs are added or removed.

Since the logic to generate the yaml file from the template will become
more complex, move the commands from the main Makefile to a script.

Using Python or other advanced scripting language for that script would
be the most versatile solution, but that would bring another dependency
on the host machine, pyyaml if Python is used. So every developer that
needs to run 'make .gitlab-ci.yml' and also the docker image used in the
GitLab pipelines would need to have pyyaml pre-installed.
Instead of adding the mentioned dependency, keep using a bash script.

While moving the commands to the script:
 - mimic the behavior of the previous make target and fail on any
   command that fails, by using 'set -e';
 - break the original lines in one command per line, making the diff for
   any patch to be applied to this file to look nicer;
 - keep the script as simple as possible, without functions, just a
   script that executes from the top to bottom;
 - do not perform validations on the input parameters, any command that
   fails already makes the script to fail;
 - do not add an usage message, the script is not intended to be called
   directly.

This patch does not change functionality.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Thomas: make the script output on stdout rather than take the output
file name as second argument.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-09 21:30:24 +01:00
Peter Korsgaard
13c43455a0 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-02 08:16:10 +01:00
Peter Korsgaard
9019189bd1 Kickoff 2019.02 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-01 23:36:34 +01:00
Peter Korsgaard
9089a9ff30 Update for 2018.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-12-01 23:06:49 +01:00
Peter Korsgaard
0031f52190 Update for 2018.11-rc3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-30 13:27:09 +01:00
Thomas Petazzoni
beef2b4ab8 Makefile: define TARGET_DIR_WARNING_FILE relative to TARGET_DIR
In commit 7e9870ce32 ("core: introduce
intermediate BASE_TARGET_DIR variable"), the definition of
TARGET_DIR_WARNING_FILE was changed to use $(BASE_TARGET_DIR) instead
of $(TARGET_DIR).

However, this change is incompatible with per-package directories, and
is in fact not needed.

With per-package directories, using $(BASE_TARGET_DIR) means that
TARGET_DIR_WARNING_FILE is
output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM. Due to this, when
skeleton-init-common or skeleton-custom attempt to install it, it
fails, because it should be installed to their package per-package
target directory, and not the global output/target directory that doesn't
exist yet. The failure looks like this:

/usr/bin/install -m 0644 support/misc/target-dir-warning.txt /home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
/usr/bin/install: cannot create regular file '/home/thomas/projets/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM': No such file or directory
make[1]: *** [package/pkg-generic.mk:336: /home/thomas/projets/buildroot/output/build/skeleton-init-common/.stamp_target_installed] Error 1

TARGET_DIR_WARNING_FILE is used in three places:

 - In skeleton-custom.mk and skeleton-init-common.mk, where as
   explained above, using $(TARGET_DIR) fixes the use of
   $(TARGET_DIR_WARNING_FILE) in the context of per-package target
   directories.

 - In fs/common.mk, where it is used as argument to $(notdir ...) to
   retrieve just the name of the warning file. So in this case, we
   really don't care about the path of the file, just its name.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:11:19 +01:00
Thomas Petazzoni
052fec0c08 Makefile: move .NOTPARALLEL statement after including .config file
In a follow-up commit, we will make the .NOTPARALLEL statement
conditional on a Config.in option, so we need to move it further down.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:10:57 +01:00
Thomas Petazzoni
d0f4f95e39 Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.

For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.

This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.

Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.

Thanks to those changes, the global "dirs" target is entirely removed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:09:46 +01:00
Thomas Petazzoni
11e8c900ff Makefile: evaluate CCACHE and HOST{CC, CXX} at time of use
As we are going to move to per-package SDK, the location of CCACHE and
therefore the definitions of HOSTCC and HOSTCXX need to be evaluated
at the time of use and not at the time of assignment. Indeed, the
value of HOST_DIR changes from one package to the other.

Therefore, we need to change from := to =.

In addition, while doing A := $(something) $(A) is possible, doing A =
$(something) $(A) is not legal. So, instead of defining HOSTCC in
terms of the current HOSTCC variable, we re-use HOSTCC_NOCCACHE
instead.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-26 19:08:13 +01:00
Peter Korsgaard
bc89c1a834 Update for 2018.11-rc2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-21 08:44:25 +01:00
Peter Korsgaard
419fc6abca Update for 2018.11-rc1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-09 22:56:48 +01:00
Yann E. MORIN
3950e69dad core: support host gcc of the future
When we do a release, we know only of a set of gcc versions that the
host may have. But in the future, distributions with newer gcc versions
may show up.

Currently, we do not recognise those versions, and thus we do as if they
were older than the oldest we know of. This means that a set of packages
become unselectable, when they should be.

We fix that by capping the detected version to the highest we know of.

Reported-by: gargar_ on IRC
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-23 11:43:35 +02:00
Arnout Vandecappelle (Essensium/Mind)
7099476882 Makefile: .gitlab-ci.yml: fail when listing tests fail
To update the .gitlab-ci.yml file, we run run-tests -l to list all the
tests and post-process the output in a format suitable for
.gitlab-ci.yml. However, in a pipeline, it is the last command that
gives the return value. In addition, we have to redirect stderr of
run-tests -l because nose2 prints the tests on stderr, not stdout. Thus,
when run-tests -l fails, the update of .gitlab-ci.yml silently succeeds
but no tests are included in the .gitlab-ci.yml.

To fix this, set the pipefail option. This is bash-specific, but our
Makefile ascertains that we are running with bash as the shell (if bash
is available, but if it is not, dependencies.sh will error out). The
error message is still invisible, but at least make will fail.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-21 23:33:04 +02:00
Michal Sojka
e33ea1c9a2 core/legal-info: Add package dependencies with licenses to the manifest
This adds one column to the legal-info manifest table. It contains the
dependencies of the given package and their licenses. This information
is useful when assessing license compatibility of the packages and
their libraries.

An example of the content of the new column for the MPD package is
shown below:

    "alsa-lib [LGPL-2.1+ (library), GPL-2.0+ (aserver)] boost
    [BSL-1.0] libid3tag [GPL-2.0+] libmad [GPL-2.0+] libogg
    [BSD-3-Clause] libvorbis [BSD-3-Clause] libzlib [Zlib]
    skeleton-init-common [unknown] skeleton-init-sysv [unknown] sqlite
    [Public domain] toolchain-external-linaro-arm [unknown]"

[Credits to Yann E. MORIN <yann.morin.1998@free.fr> for suggesting a
few simplifications.]

Signed-off-by: Michal Sojka <sojka@merica.cz>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-21 19:13:20 +02:00
Michal Sojka
8f14901043 core/legal-info: Change order of legal-manifest parameters
The last parameter {HOST|TARGET} is now first. With this change,
adding new columns to the legal manifest file (as in the next commit)
will be slightly easier to review.

Signed-off-by: Michal Sojka <sojka@merica.cz>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-21 19:12:31 +02:00
Yann E. MORIN
7007dc2bc9 core: detect and reject build paths which contain an '@'
gcc does not build when the srcdir path contains a '@', because that
path is then substitued in a texi file as argument to an @include
directive. But then, the '@' in the path will start a command evaluation
of its own, thus breaking the build. For example, with a $(O) path set
to /home/ymorin/dev/buildroot/O/to@ti :

    perl ../../gcc/../contrib/texi2pod.pl ../../gcc/doc/invoke.texi > gcc.pod
    ../../gcc/doc/invoke.texi:1678: unknown command `ti'
    ../../gcc/doc/invoke.texi:1678: @include: could not find /home/ymorin/dev/buildroot/O/to/build/host-gcc-initial-7.3.0/build/gcc/../../gcc/../libiberty/at-file.texi

[Peter: use findstring instead of subst/compare]
Reported-by: c32 on IRC
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-10-20 20:50:48 +02:00
Yann E. MORIN
0c45649c12 legal-info: use the per-package variable to get the hash file
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-20 20:04:06 +02:00
Mark Corbin
9b3d52b400 arch: add support for RISC-V 64-bit (riscv64) architecture
This enables a riscv64 system to be built with a Buildroot generated
toolchain (gcc >= 7.x, binutils >= 2.30, glibc only).

This configuration has been used to successfully build a qemu-bootable
riscv-linux-4.15 kernel (https://github.com/riscv/riscv-linux.git).

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
[Thomas:
 - simplify arch.mk.riscv by directly setting GCC_TARGET_ARCH
 - simplify glibc.mk changes by using GLIBC_CONF_ENV.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-23 23:42:41 +02:00
Mark Corbin
bd0640a213 arch: allow GCC target options to be optionally overwritten
The BR2_GCC_TARGET_* configuration variables are copied to
corresponding GCC_TARGET_* variables which may then be optionally
modified or overwritten by architecture specific makefiles.

All makefiles must use the new GCC_TARGET_* variables instead
of the BR2_GCC_TARGET_* versions.

Signed-off-by: Mark Corbin <mark.corbin@embecosm.com>
[Thomas: simplify include of arch/arch.mk]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-23 22:17:57 +02:00
Petr Vorel
6eacea5ae0 support/kconfig: bump to kconfig from Linux 4.17-rc2
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-20 23:14:38 +02:00
Trent Piepho
b8d0aadc6d Makefile: fix issue with printvars executing giant shell command
The underlying problem is that $(foreach V,1 2 3,) does not evaluate to
an empty string.  It evaluates to "  ", three empty strings separated by
whitespace.

A construct of this format, with a giant list in the foreach, is part of
the printvars command.  This means that "@:$(foreach ....)", which is
intended to expand to a null command, in fact expands to "@:       "
with a great deal of whitespace.  Make chooses to execute this command
with:
    execve("/bin/sh", ["/bin/sh", "-c", ":       "]

But with far more whitespace.  So much that it can exceed shell command
line length limits.

This solution is to move the foreach to another step in the recipe.  The
"@:" is retained as the first line so the recipe is not Empty, which
would cause a change in make behavior when make builds the target.  The
2nd line, all whitespace, will be skipped by make.

Signed-off-by: Trent Piepho <tpiepho@impinj.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-09-18 22:03:23 +02:00
Peter Korsgaard
721e4cbb52 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-07 13:13:17 +02:00
Peter Korsgaard
89920e9735 Kickoff 2018.11 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-06 22:52:43 +02:00
Peter Korsgaard
339d550e92 Update for 2018.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-06 22:11:06 +02:00
Peter Korsgaard
24b5ff16ae Update for 2018.08-rc3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-09-01 00:28:13 +02:00
Peter Korsgaard
a907ab7db5 Update for 2018.08-rc2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-08-20 10:55:03 +02:00
Yann E. MORIN
c32ad51cbf core/sdk: generate the SDK tarball ourselves
Currently, the wording in the manual instructs the user to generate a
tarball from "the contents of the +output/host+ directory".

This is pretty confusing, because taken literally, this would amount to
running a command like:

    tar cf my-sdk.tar -C output/host/ .

This creates a tarbomb [0], which is very bad practice, because when
extracted, it creates multiple files in the current directory.

What one really wants to do, is create a tarball of the host/ directory,
with something like:

    tar cf my-sdk.tar -C output host/

However, this is not much better, because the top-most directory would
have a very common name, host/, which is pretty easy to get conflict
with when it gets extracted.

So, we fix that mess by giving the top-most directory a recognisable
name, based on the target tuple, which we also use as the name of the
archive (suffixed with the usual +.tar.gz+.) We offer the user the
possibility to override that default by specifying the +BR2_SDK_PREFIX+
variable on the command line.

Since this is an output file, we place it in the images/ directory.

As some users expressed a very strong feeling that they do not want to
generate a tarball at all, and that doing so would badly hurt their
workflows [1], we actually prepare the SDK as was previously done, but
under the new, intermediate rule 'prepare-sdk'. The existing 'sdk' rule
obviously depend on that before generating the tarball.

We choose to make the existing rule to generate the tarball, and
introduce a new rule to just prepare the SDK, rather than keep the
existing rule as-is and introduce a new one to generate the tarball,
because it makes sense to have the simplest rule do the correct thing,
leaving advanced, power users use the longest command. If someone
already had a wrapper that called 'sdk' and expected just the host
directory to be prepared, then this is not broken; it just takes a bit
longer (gzip is pretty fast).

Update the manual accordingly.

[0] https://en.wikipedia.org/wiki/Tar_(computing)#Tarbomb
[1] http://lists.busybox.net/pipermail/buildroot/2018-June/thread.html#223377
    and some messages in the ensuing thread...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Wolfgang Grandegger <wg@grandegger.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Stefan Becker <chemobejk@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br>
Reviewed-by: Stefan Becker <chemobejk@gmail.com>
Signed-off-by: &quot;Yann E. MORIN&quot; &lt;<a href="mailto:yann.morin.1998@free.fr" target="_blank">yann.morin.1998@free.fr</a>&gt;<br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-14 16:03:48 +02:00
Thomas Petazzoni
1290241dc6 Makefile: introduce check-package target
The snippet of code that runs a check-package on all
.mk/.hash/Config.in files is currently only available within
.gitlab-ci.yml, and isn't immediately and easily usable by Buildroot
users. In order to simplify this, this commit introduces a top-level
"check-package" make target that implements the same logic. The
.gitlab-ci.yml file is changed to use "make check-package".

Since this target is oriented towards Buildroot developers, we
intentionally do not clutter the already noisy "make help" text with
this additional make target.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-12 14:39:32 +02:00
Thomas Petazzoni
71d8148e59 Update for 2018.08-rc1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-05 15:40:05 +02:00
Arnout Vandecappelle (Essensium/Mind)
27aa7ae618 Makefile: help: BR2_DEFCONFIG for defconfig must be on command line
The help text says that BR2_DEFCONFIG will be used as input, but a
BR2_DEFCONFIG specified in the existing .config file will *not* be
used. So say explicitly that it must be specified on the command line.
Note that both "BR2_DEFCONFIG=... make defconfig" and
"make defconfig BR2_DEFCONFIG=..." will work.

While we're at it, add a semicolon to separate the two statements.

Note that this overflows the help text beyond 80 characters, but that
is already the case in many other lines.

Reported-by: Anisse Astier <anisse@astier.eu>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-07-28 23:21:14 +02:00
Peter Korsgaard
ef01260b3d Kickoff 2018.08 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-06-02 11:11:56 +02:00
Peter Korsgaard
f3d114a1ef Update for 2018.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-06-01 22:22:57 +02:00
Peter Korsgaard
bea6b866ef Update for 2018.05-rc3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-05-28 23:02:21 +02:00
Peter Korsgaard
c11ed3a4d9 Update for 2018.05-rc2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-05-22 23:26:26 +02:00
Thomas Petazzoni
b19e24b9f7 Update for 2018.05-rc1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-09 23:00:18 +02:00
Chris Lesiak
bbe5c6dad4 Makefile: Update mtime of $(TARGET_DIR)/usr in target-finalize
The systemd ConditionNeedsUpdate option is useful when offline updates
of the vendor operating system resources in /usr require updating of
/etc or /var on the next following boot.

Two examples of services making use of this option are
systemd-hwdb-update.service and systemd-sysusers.service.

ConditionNeedsUpdate=/etc will be true if the mtime of /etc/.updated
is older than the mtime of /usr.  After services conditional on
ConditionNeedsUpdate have run, systemd-update-done.service will
synch the mtime of /usr to /etc/.updated so that the condition will
be false on subsequent boots.

For systems with writable /usr partitions where updates are done to
the running system, the update program will touch /usr as a final step.
But with Buildroot, where updates are often done by dumping a new
image onto the device, and where /usr is on a filesystem mounted
read-only, touching /usr as part of the update process is not practical.
Instead, it should be done a build time.

For testers, please note that systemd-update-done in v234 added a
regression where the mtime of /etc/.updated is set to the current time
instead of the mtime or /usr.  This will be fixed in v239.

For more details, see:
http://0pointer.de/public/systemd-man/systemd.unit.html
http://0pointer.de/public/systemd-man/systemd-update-done.service.html

Signed-off-by: Chris Lesiak <chris.lesiak@licor.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-03 22:12:21 +02:00
Thomas Petazzoni
325bb37942 arch: remove Blackfin architecture
The Blackfin architecture has for a long time been complicated to
maintain, with poor support in upstream binutils/gcc. As of April
2018, the Blackfin architecture has been dropped from the upstream
Linux kernel. Also, the Analog Device engineer who used to be in touch
with the Buildroot community also privately said we should drop the
support for this architecture, which Analog Devices is no longer
using, promoting and maintaining.

The BR2_BINFMT_FLAT_SEP_DATA option becomes unselectable, it will be
removed in a future commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-15 22:03:41 +02:00
James Byrne
49395dc0e5 Makefile: take default SOURCE_DATE_EPOCH from repo containing Makefile
For reproducible builds, SOURCE_DATE_EPOCH will be set to the git commit
date if it is not defined in the environment, but this was done by
explicitly using $(TOPDIR)/.git as the git repository, which would not
give the expected result if Buildroot had been put into a subdirectory
of another repository.

This commit removes that restriction, meaning that the default date will
now be the date of the git commit that contains Makefile, regardless of
what level above Makefile the repository is at. This works because the
current directory when the 'git log' command is executed will always be
the directory containing Makefile (it must be, since TOPDIR is set from
CURDIR).

In general this should be a sensible default, and in cases where a
different date is required SOURCE_DATE_EPOCH can be defined in the
environment before invoking make.

Signed-off-by: James Byrne <james.byrne@origamienergy.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-12 23:32:32 +02:00
James Byrne
9b47146eb2 Makefile: avoid executing 'git log' each time SOURCE_DATE_EPOCH is used
If SOURCE_DATE_EPOCH is not defined it was given a definition that
caused 'git log' to be executed each time the variable is referenced,
which is not very efficient given that the answer cannot change.

This commit moves the definition of BR2_VERSION_GIT_EPOCH after the
inclusion of Makefile.in (so that GIT is defined) and makes it a
simply expanded variable so that it is only evaluated once.

Signed-off-by: James Byrne <james.byrne@origamienergy.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-09 20:59:49 +02:00
George Redivo
ca9b17a263 package/pkg-generic: add <pkg>-show-recursive-(r)depends targets
This commit adds the support for <pkg>-show-recursive-depends and
<pkg>-show-recursive-rdepends which respectively show the list of all
dependencies or reverse dependencies for a given package. The existing
show-depends and show-rdepends only show the first-level dependencies,
while show-recursive-depends and show-recursive-rdepends show
recursively the dependencies.

It is worth mentioning that while show-recursive-depends really shows
all dependencies, show-recursive-rdepends is a bit limited because the
reverse dependencies of host packages are not properly accounted
for. But that's a limitation that already exists in show-rdepends, and
that cannot easily be solved.

Signed-off-by: George Redivo <george.redivo@datacom.ind.br>
[Thomas:
 - split from the patch that was also changing graph-depends
 - rename show-rrdepends to show-recursive-rdepends
 - add show-recursive-depends
 - don't create GRAPHS_DIR.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-01 22:25:57 +02:00
Peter Korsgaard
3d02062787 Makefile: Ensure BASE_TARGET_DIR exists, not TARGET_DIR
This was present in Yann's original patch, but got dropped when I rebased
commit 7e9870ce32 (core: introduce intermediate BASE_TARGET_DIR variable) to
fix the Makefile conflicts.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 21:08:33 +02:00
Yann E. MORIN
543107d390 fs: remove intermediate artefacts
Each of the intermediate, per-rootfs target directories, as well as the
intermediate tarball, can take quite some place, and is mostly a
duplication of what's already in target/. The only delta, if any, would
be the tweaks made by the filesystem image generations, but those tweaks
are most probably only meaningful when seen as root.

We normally do not remove intermediate files, but those can be quite
large, and are not directly usable by, nor accessible to the user.
So, get rid of them once the filesystem has been generated.

This does not need to be done in fakeroot.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
c6e425729e fs: introduce per-rootfs TARGET_DIR variable
... which for now still points to the base target directory, but this is
a step forward.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:53:06 +02:00
Yann E. MORIN
2765973e01 fs: set per-rootfs variable name
Like we do for packages with the PKG variable, set ROOTFS to contain the
upper-case name of the rootfs currently being generated.

This will be useful in later patches, when we need more per-rootfs
variables, like a per-rootfs TARGET_DIR for example.

In Makefiles, per-rule variables trickle down the dependency chain, to
all dependencies of that rule, so we have to stop ROOTFS as soon as
we're not in a rootfs. This means we have to stop it at target-finalize
(which is a dependency of all filesystems), and for each package
individually, since some packages (host or target) can be direct
dependencies of filesystems as well.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:52:52 +02:00
Yann E. MORIN
7e9870ce32 core: introduce intermediate BASE_TARGET_DIR variable
This new BASE_TARGET_DIR variable is set in stone to point to the real
location where packages will be installed. Its name is modelled after
its definition: it is located in $(BASE_DIR), and it is named 'target/',
hence BASE_TARGET_DIR.

The already-existing TARGET_DIR variable now simply points to the same
location, except that it is recursively expanded, so that we can later
change it depending on the context.

All locations that really need to reference the existing target/
directory, are changed to use BASE_TARGET_DIR; surprinsigly enough, they
all seem to be located in the main Makefile. :-) The rest is left with
using good-old TARGET_DIR.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-31 20:47:25 +02:00
Arnout Vandecappelle (Essensium/Mind)
91b4a4525b Makefile: create symlink to non-default HOST_DIR
If BR2_HOST_DIR is not the default, it can be difficult to find the
host directory (i.e., HOST_DIR always has to be passed explicitly in
addition to the output directory). For example, the Eclipse plugin
assumes that HOST_DIR=BASE_DIR/host.

Create a symlink from $(BASE_DIR)/host to $(HOST_DIR) if it is not the
default. Also remove it in the clean target.

When BR2_HOST_DIR is the default, HOST_DIR_SYMLINK will be empty so
there will be no additional dependency to dirs and nothing to remove
in clean.

Fixes https://bugs.busybox.net/show_bug.cgi?id=10151

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-31 18:57:21 +02:00
Arnout Vandecappelle (Essensium/Mind)
ca9a0b2515 Makefile: move mkdir rule to after HOST_DIR is defined
HOST_DIR is defined twice: once to its default value before .config is
included, and once more to BR2_HOST_DIR after .config is included.
However, the rule that defines the mkdir for HOST_DIR comes between
these two, so it will always use the default definition. Therefore,
if a non-default BR2_HOST_DIR is used, there will be no rule to create
that directory, while the dirs target depends on it.

This happens to work at the moment, because in the dirs target,
$(STAGING_DIR) comes before $(HOST_DIR), so $(HOST_DIR) will be created
implicitly. However, this will fail in top-level parallel builds where
both will be created in parallel.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-03-31 18:57:21 +02:00
Stefan Becker
6697f3bcdb Makefile: fix build break in sdk target
After commit 6729050f3a nothing creates
$(HOST_DIR)/share/buildroot anymore, causing sdk to fail with:

 /bin/bash: .../output/host/share/buildroot/sdk-location: No such file or directory

Add creation of that directory to the "sdk" build steps itself.

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-26 10:48:19 +02:00
Thomas Petazzoni
ea9669fffa core: kill DEPENDENCIES_HOST_PREREQ
Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:52:06 +02:00
Thomas Petazzoni
38d1c4aae3 package/pkg-generic: handle host-fakedate as a regular dependency
This commit moves the host-fakedate dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper regular dependency handled by the
package infrastructure.

host-fakedate is added as dependency to all packages, except
host-skeleton, because we depend on it.

In addition, we make sure that host-fakedate does not grow a
dependency on host-{tar,xz,lzip,ccache} to avoid circular
dependencies. host-fakedate does not need any extraction tool and does
not need to build C/C++ code (the source code is just a shell script
available in Buildroot).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:50:29 +02:00
Thomas Petazzoni
3ff90c8888 Makefile, skeleton: move the host skeleton logic to host-skeleton package
As part of the per-package SDK work, we want to avoid having logic
that installs files to the global HOST_DIR, and instead do it inside
packages. One thing that gets installed to the global HOST_DIR is the
minimal "skeleton" that we create in host:

 - the "usr" symbolic link for backward compatibility

 - the "lib" directory, and its lib64 or lib32 symbolic links

This commit moves this logic to a new host-skeleton package, and makes
all packages (except itself) depend on it.

While at it, use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS.

[Peter: drop host-patchelf reference in commit message]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-25 17:34:54 +02:00
Peter Korsgaard
528f165476 Kickoff 2018.05 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-05 19:32:12 +01:00
Peter Korsgaard
8a94ff12d2 Update for 2018.02
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 22:28:34 +01:00
Yann E. MORIN
c2a9358b6e core: find a host UTF-8 locale
Some packages really want to use an UTF-8 locale, or they break.

However, there is no guarantee that any given locale is available on a
system. For example,, while most mainstream distros (Debian and
derivatives, Fedora...) do have the generic, language-agnostic C.UTF-8
locale, Gentoo does not provide it.

So, find the first UTF-8 locale available on the system, and take any
that is available. We however do favour using the user-set current
locale, then using the language-agnostic C.UTF-8, and eventually any
random UTF-8 locale.

Note: we only need to enforce LC_ALL, because setting it implies
everything else:
    http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02

    """
    1. If the LC_ALL environment variable is defined and is not null,
    the value of LC_ALL shall be used.
    """

[Peter: use same regexp as in dependencies.sh]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-04 11:59:03 +01:00
Peter Korsgaard
d5a63f48d7 Update for 2018.02-rc3
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-27 22:58:57 +01:00
Peter Korsgaard
44cec6431c Update for 2018.02-rc2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-15 23:03:43 +01:00
Peter Korsgaard
30bffce8d5 Update for 2018.02-rc1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-05 16:34:00 +01:00
Peter Korsgaard
676400379a Makefile, manual, website: Bump copyright year
Happy 2018!

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-01 16:49:41 +01:00
Chris Lesiak
5f2dc44590 Makefile: Store OS release in /usr/lib/os-release
It is recommended that vendor trees store OS release information
in /usr/lib/os-release and that /etc/os-release should be a relative
symlink to /usr/lib/os-release.

For more details, see:

http://0pointer.de/public/systemd-man/os-release.html

[Peter: don't hide command, simplify ln invocation]
Signed-off-by: Chris Lesiak <chris.lesiak@licor.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-28 21:45:19 +01:00
Maxime Hadjinlian
bf28a165d9 pkg-{download, generic}: remove source-check
This feature is not used by anyone in the core developpers and makes a
drastic simplification of the pkg-download infrastructure harder.

The future patch will move much of what's in the current pkg-download.mk
file into the dl-wrapper which is a shell script.

Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-08 21:25:46 +01:00
Thomas Petazzoni
1c8dda3e43 Merge branch 'next'
This merges the next branch accumulated during the 2017.11 release
cycle back into the master branch.

A few conflicts had to be resolved:

 - In the DEVELOPERS file, because Fabrice Fontaine was added as a
   developer for libupnp in master, and for libupnp18 in
   next. Resolution is simple: add him for both.

 - linux/Config.in, because we updated the 4.13.x release used by
   default in master, while we moved to 4.14 in next. Resolution: use
   4.14.

 - package/libupnp/libupnp.hash: a hash for the license file was added
   in master, while the package was bumped into next. Resolution: keep
   the hash for the license file, and keep the hash for the newest
   version of libupnp.

 - package/linux-headers/Config.in.host: default version of the kernel
   headers for 4.13 was bumped to the latest 4.13.x in master, but was
   changed to 4.14 in next. Resolution: use 4.14.

 - package/samba4/: samba was bumped to 4.6.11 in master for security
   reasons, but was bumped to 4.7.3 in next. Resolution: keep 4.7.3.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-01 21:56:44 +01:00
Peter Korsgaard
57dcad243e Kickoff 2018.02 cycle
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-12-01 10:31:43 +01:00
Peter Korsgaard
9dd76697cc Update for 2017.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-30 22:35:17 +01:00
Yann E. MORIN
203219ca93 core: sort packages and eliminate duplicates in show-targets
Currently, enabling more than one filesystem image will make
'show-targets' list a few host packages more than once.

This is because all filesystem images add the same set of
host-packages to their dependencies, which are then added as-is
to the package list.

Thus, host-fakeroot, host-makedevs and, if needed, host-mkpasswd will
appear as many times as there are filesystem images enabled.

Fix that by sorting the package list, thus eliminating duplicates from
that list. Also sort the rootfs list for good measure. Sort the two
separately, so that rootfses are last.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-30 11:40:43 +01:00
Yann E. MORIN
1b8ad2d08e core: check files are not touched by more than one package
Currently, we do nothing about packages that touch the same file: given
a specific configuration, the result is reproducible (even though it
might not be what the user expected) because the build order is
guaranteed.

However, when we later introduce top-level parallel build, we will no
longer be able to guarantee a build order, by the mere way of it being
parallel. Reconciliating all those modified files will be impossible to
do automatically. The only way will be to refuse such situations.

As a preliminary step, introduce a helper script that detects files that
are being moified by two or more packages, and reports them and the
impacted packages, at the end of the build.

The list being reported at the end of the build will make it prominently
visible in autobuilder results, so we can assess the problem, if any.

Later on, calling that helper script can be done right after the package
installation step, to bail out early.

Thanks Arnout for the pythonist way to write default dictionaries! ;-)

Note: doing it in python rather than a shell script is impressively
faster: where the shell script takes ~1.2s on a minimalist build, the
python script only takes ~0.015s, that is about 80 times faster.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Peter Seiderer <ps.report@gmx.net>
[Thomas: rename script without .py extension.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-11-27 23:02:19 +01:00
Peter Seiderer
b9d2d4cb4e Fix makefile include order by using sort/wildcard.
The 'include' directive in GNU make supports wildcards, but their
expansion has no defined sort order (GLOB_NOSORT is passed to glob()).
Usually this doesn't matter. However, there is at least one case where
it does make a difference: toolchain/*/*.mk includes both the
definitions of the external toolchain packages and
pkg-toolchain-external.mk, but pkg-toolchain-external.mk must be
included first.

For predictability, use ordered 'include $(sort $(wildcard ...))'
instead of unordered direct 'include */*.mk' everywhere.

Fixes [1] reported by Petr Vorel:

  make: *** No rule to make target 'toolchain-external-custom', needed by '.../build/toolchain-external/.stamp_configured'.  Stop.

[1] http://lists.busybox.net/pipermail/buildroot/2017-November/206969.html

Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Tested-by: Petr Vorel <petr.vorel@gmail.com>
[Arnout: also sort the one remaining include, of the external docs]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-11-24 00:08:23 +01:00
Peter Korsgaard
071cc43892 Update for 2017.11-rc2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-13 22:28:15 +01:00
Peter Korsgaard
a7e1971cc6 Update for 2017.11-rc1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-06 22:03:32 +01:00
Yann E. MORIN
0437d2f8f6 core/reproducible: do not override SOURCE_DATE_EPOCH
SOURCE_DATE_EPOCH is currently forcibly set (to either the git commit
date, or the last release date).

However, the spec mandates that it should not be modified if already
set: https://reproducible-builds.org/specs/source-date-epoch/

    Build systems MUST NOT overwrite this variable for child
    processes to consume if it is already present.

Abide by the rule, and only set it if not already set.

This will allow users to pass it from an upper-layer buildsystem (e.g. a
jenkins or gitlab-ci job, for example), when they have a reson to do so.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Reported-by: Einar Jón Gunnarsson <tolvupostur@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Einar Jón Gunnarsson <tolvupostur@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-05 11:43:19 +01:00
Alfredo Alvarez Fernandez
862b76cfef Add DEPENDENCIES_HOST_PREREQ to the list of packages
That way packages included in that list like ccache will also be
regarded as a normal packages for targets like external-deps,
show-targets or legal-info

Signed-off-by: Alfredo Alvarez Fernandez <alfredo.alvarez_fernandez@nokia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-10-22 15:42:26 +02:00
Matt Weber
5ad679c8a8 HOST_DIR/lib: symlink respectively to lib32/64
Discovered the issue on a RHEL7.4 machine where
the cmake build dynamically selected HOST_DIR/lib64
as the installation path for the lzo2 library.

Fixes failures like the following:
host-mtd
http://autobuild.buildroot.net/results/d31/d31581d2e60f35cf70312683df99c768e2ea8516/

host-squashfs
http://autobuild.buildroot.net/results/d9c/d9c95231ac774ed71580754a15ebb3b121764310/

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2017-10-05 20:32:14 +02:00
Thomas Petazzoni
d02b7bfab1 Kickoff 2017.11 cycle
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-09-02 15:14:27 +02:00
Peter Korsgaard
8ce27bb9fe Update for 2017.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-09-02 01:17:43 +02:00
Thomas Petazzoni
ef19f376c7 Update for 2017.08-rc3
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-23 23:38:05 +02:00
Thomas Petazzoni
0ba4a139f5 Update for 2017.08-rc2
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-11 22:36:39 +02:00
Thomas Petazzoni
b63fdf4714 Update for 2017.08-rc1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-02 23:00:13 +02:00
Arnout Vandecappelle
dab80981d1 Makefile: add alldefconfig target
It is used by Kconfig's merge_config.sh.

No alldefpackageconfig is added, since it's rather pointless: it would
only enable busybox.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-25 23:06:58 +02:00
Arnout Vandecappelle
2607051e1b Makefile: refactor *config targets
The rules for the *config targets are all very similar, so factor them
together using $@.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-25 23:06:49 +02:00
Yann E. MORIN
2de968f03a system: provide package-wide system variables and macros
Some macros, soon some variables, currently defined in the skeleton are
going to be used by other packages.

Some of those variables will be used as Makefile conditions (e.g. in
ifeq() conditions), so they *must* be defined before being used.

Since the skeleton package, starting with an 's', is included quite
late, those variables would not be available to most packages.

Offload the existing macros into the new system/system.mk file, that is
included early, before any package is. Rename the macros to appropriate
names.

Future commits will add new macros and variables in that file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 21:51:17 +02:00
Wolfgang Grandegger
994301a26f core: introduce "sdk" target to make a relocatable SDK
We use a separate make target to build a relocatable SDK. We first
sanitize the RPATH in host tree. Next we also sanitize the
staging tree. Therefore "sdk" must depend on world.

Sanitizing staging is not really needed, in the sense that any rpath
in there is simply not going to be used. We want to sanitize staging
for the following reasons:

- To avoid leaking references to the original output directory. This
  way, we can validate that the SDK is relocatable by running a simple
  "grep -r ${BASE_DIR} ${HOST_DIR}". Obviously RPATH sanitization is
  not sufficient (e.g. also the references to source files have to be
  stripped), but it's a step in the right direction. This reason is
  obviously only relevant for the SDK.

- To make sure that when an executable is copied to target that it
  actually executes correctly. Since within Buildroot we never copy
  stuff from staging to target, this is clearly only relevant for
  the SDK.

Finally we install the script "relocate-sdk.sh" into the top directory
of the SDK (HOST_DIR) and the SDK location path is stored in the file
"HOST_DIR/share/buildroot/sdk-location"-

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-22 14:35:13 +02:00
Wolfgang Grandegger
5f4ca51809 core: sanitize RPATH in target tree before copying the overlay
We sanitize the RPATH of ELF files in the target tree to deal
with stupid packages that don't correctly use --prefix/DESTDIR
and that end up putting the full absolute build-time directory
in the RPATH.

We do it before copying the overlay and calling the post-build
script. The user is completely responsible for what happens
in the last two steps, and it should never be touched by us.

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-20 22:47:04 +02:00
Arnout Vandecappelle
ce58db7232 Makefile: properly create $(HOST_DIR)/usr compatibility symlink
Up to now we created the $(HOST_DIR)/usr compatibility symlink as part
of the creation of $(HOST_DIR) itself. However, when the user specifies
a custom BR2_HOST_DIR, it is possible that the directory already exists
so this rule will never trigger.

Therefore, add an explicit rule for creating $(HOST_DIR)/usr and add
this rule to the dependencies of the dirs target. HOST_DIR itself goes
back to the standard rule for directories. The order-only dependency of
STAGING_DIR isn't needed any more either: HOST_DIR is implicitly
created if needed by mkdir -p, and we don't need to trigger the
HOST_DIR rule any more if the directory already exists.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-10 17:45:57 +02:00
Yann E. MORIN
719726af2e legal-info: add hash for our own license file
This silences the annoying warning that there is no hash file for our
own COPYING file.

Also change the message so that it is more obvious what we're doing.

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-07-05 16:31:40 +02:00
Arnout Vandecappelle
0f9c0bf3d5 Globally replace $(HOST_DIR)/usr/bin with $(HOST_DIR)/bin
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.

This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 15:19:29 +02:00
Arnout Vandecappelle
4c8872d8e7 Makefile: remove $(HOST_DIR)/usr from BR_PATH
Now $(HOST_DIR)/usr is a symlink to $(HOST_DIR), it makes no sense to
still have it in BR_PATH.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:46:15 +02:00
Arnout Vandecappelle
14151d77af Eliminate $(HOST_DIR)/usr
We currently use $(HOST_DIR)/usr as the prefix for host packages. That
has a few disadvantages:

- There are some things installed in $(HOST_DIR)/etc and
  $(HOST_DIR)/sbin, which is inconsistent.

- To pack a buildroot-built toolchain into a tarball for use as an
  external toolchain, you have to pack output/host/usr instead of the
  more obvious output/host.

- Because of the above, the internal toolchain wrapper breaks which
  forces us to work around it (call the actual toolchain executable
  directly). This is OK for us, but when used in another build system,
  that's a problem.

- Paths are four characters longer.

To allow us to gradually eliminate $(HOST_DIR)/usr while building
packages, replace it with a symlink to .

The symlinks from $(HOST_DIR)/usr/$(GNU_TARGET_NAME) and
$(HOST_DIR)/usr/lib that were added previously are removed again.

Note that the symlink creation will break when $(HOST_DIR)/usr
already exists as a directory, i.e. when rebuilding in an existing
output directory. This is necessary: if we don't break it now, the
following commits (which remove the usr part from various variables)
_will_ break it.

At the same time as creating this symlink, we have to update the
external toolchain wrapper and the external toolchain symlinks to go
one directory less up. Indeed, $(HOST_DIR) is one level less up than
it was before.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:45:35 +02:00
Arnout Vandecappelle
4c790b8864 Move $(HOST_DIR)/usr/lib to $(HOST_DIR)/lib
This is a step towards eliminating $(HOST_DIR)/usr. It allows us to
convert all packages installing things into $(HOST_DIR)/usr/lib without
affecting the rest.

To allow compatibility with packages that still use $(HOST_DIR)/usr as
the prefix, create a symlink from usr/lib to ../lib.

Note that the symlink creation will break when $(HOST_DIR)/usr/lib
already exists as a directory, i.e. when rebuilding in an existing
output directory. This is necessary: if we don't break it now, the
following commits (which remove the usr part from various variables)
_will_ break it.

At the same time as creating this symlink, we also have to update the
check-host-rpath script to accept both $(HOST_DIR)/usr/lib and
$(HOST_DIR)/lib, because depending on how the package derives the
path, it may be different.

Since there are some dependency chains that involve $(STAGING_DIR),
$(STAGING_DIR) may in fact be created before $(HOST_DIR). Since
$(STAGING_DIR) is a subdirectory of $(HOST_DIR), it is possible that the
newly added rule for $(HOST_DIR) never triggers. To make sure that the
rule does trigger, add an order-only dependency from $(STAGING_DIR) to
$(HOST_DIR).

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:39:58 +02:00
Arnout Vandecappelle
82ec49787d Move $(HOST_DIR)/usr/$(GNU_TARGET_NAME) one level up.
This is a step towards eliminating $(HOST_DIR)/usr. It allows us to
convert all packages installing things into
$(HOST_DIR)/usr/$(GNU_TARGET_NAME) (i.e., binutils and gcc) without
affecting the rest.

To allow compatibility with packages that still use $(HOST_DIR)/usr as
the prefix, create a symlink from usr/$(GNU_TARGET_NAME) to
../$(GNU_TARGET_NAME).

Note that the symlink creation will break when $(HOST_DIR)/usr/lib
already exists as a directory, i.e. when rebuilding in an existing
output directory. This is necessary: if we don't break it now, the
following commits (which remove the usr part from various variables)
_will_ break it.

Effectively, the usr/ part is removed from $(STAGING_SUBDIR) (and
therefore from $(STAGING_DIR)), so update the definition of that
variable right away.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-05 11:38:23 +02:00
Yann E. MORIN
ea5695b5df core/pkg-util: pass package directory and name when saving license files
This will be useful when checking the hashes of the license files.

[Peter: use '.' as buildroot directory so /buildroot.hash isn't checked]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-07-03 18:03:20 +02:00
Peter Korsgaard
51825df3a1 Makefile: generate wrapper makefile when running make without a .config
The recent change to error out instead of running menuconfig when no .config
is available broke an existing use case:

make O=output-foo; cd output-foo; br-init-conf (or similar to get a sensible .config); make

As there is no wrapper makefile in output-foo.

Fix it by ensuring the wrapper gets created if needed.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-07-03 15:02:10 +02:00
Thomas Petazzoni
b287ea6fc5 .gitlab-ci.yml: run our runtime tests
This commit improves our .gitlab-ci.yml logic to execute our runtime
tests located in support/testing/. To do so, this commit:

 - Adds more Debian packages to be installed, namely the nose2 and
   pexpect packages needed by the runtime testing infrastructure, as
   well as the necessary Qemu emulators

 - The description of how to run the runtime tests. Each test is
   executed as a separate Gitlab CI job, so that the status of each
   test is easily visible in the Gitlab CI web interface.

 - The Makefile is improved to auto-generate .gitlab-ci.yml from
   .gitlab-ci.yml.in, like we're doing for defconfigs. Since the
   dependencies of .gitlab-ci.yml are no longer correct, we abandon
   them and instead make it a PHONY target.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Arnout: simplify .gitlab-ci.in a little, removing redundant stuff;
         make .gitlab-ci.yml a PHONY target]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-02 23:45:07 +02:00
Arnout Vandecappelle
c3493069f2 Makefile: don't run "menuconfig" automatically
Since forever, we run 'menuconfig' automatically on an unconfigured
tree. However, this does not help users that much:
- If they read the documentation, they should already know to run
  make menuconfig first.
- If they haven't read the documentation, dropping them in menuconfig
  isn't very helpful.
- It's a likely that the user didn't intend to be in an unconfigured
  tree (e.g. wrong O= specified), so starting menuconfig (and polluting
  this wrong O= directory) is not very helpful.
- It's possible that the user really doesn't want menuconfig, but
  instead needs xconfig, or some defconfig, or ...

So, instead of trying to guess what the user needs, print an error and
let the user decide what to do next.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 10:26:06 +02:00