2013-06-06 01:53:30 +02:00
|
|
|
################################################################################
|
2012-04-17 16:45:20 +02:00
|
|
|
#
|
|
|
|
|
# This file contains the download helpers for the various package
|
|
|
|
|
# infrastructures. It is used to handle downloads from HTTP servers,
|
|
|
|
|
# FTP servers, Git repositories, Subversion repositories, Mercurial
|
|
|
|
|
# repositories, Bazaar repositories, and SCP servers.
|
|
|
|
|
#
|
2013-06-06 01:53:30 +02:00
|
|
|
################################################################################
|
2012-04-17 16:45:20 +02:00
|
|
|
|
2012-04-17 16:45:19 +02:00
|
|
|
# Download method commands
|
2015-01-02 16:53:41 +01:00
|
|
|
export WGET := $(call qstrip,$(BR2_WGET))
|
2014-07-02 23:11:20 +02:00
|
|
|
export SVN := $(call qstrip,$(BR2_SVN))
|
2014-07-02 23:11:21 +02:00
|
|
|
export CVS := $(call qstrip,$(BR2_CVS))
|
2014-07-02 23:11:24 +02:00
|
|
|
export BZR := $(call qstrip,$(BR2_BZR))
|
2014-07-02 23:11:19 +02:00
|
|
|
export GIT := $(call qstrip,$(BR2_GIT))
|
2015-01-02 16:53:41 +01:00
|
|
|
export HG := $(call qstrip,$(BR2_HG))
|
|
|
|
|
export SCP := $(call qstrip,$(BR2_SCP))
|
2020-04-15 18:48:45 +02:00
|
|
|
export SFTP := $(call qstrip,$(BR2_SFTP))
|
2014-07-02 23:11:26 +02:00
|
|
|
export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
|
2012-04-17 16:45:19 +02:00
|
|
|
|
support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.
However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.
To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.
So, we bump the git-specific format-version to -br1.
The %ci date has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.
As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.
Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.
For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
---8<------8<------8<------8<---
#!/bin/sh
# Find and download all packages using git as backend.
# Manually fix hashes for affected packages.
# Packages that only have a host variant
HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'
# Packages that have a non-git main _SOURCE, and/or which
# have BR_NO_CHECK_HASH_FOR for the git _SOURCE
NOT_GIT='aufs|aufs-util|xenomai|linux-headers'
export BR2_DL_DIR=$(pwd)/temp-dl-dir
make defconfig
make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
boot/vexpress-firmware/ package/ \
|sed -r -e 's,.*/([^/]+)\.mk,\1,' \
|sed -r -e '/^('"${NOT_GIT}"')$/d;' \
-e 's/^('"${HOST_ONLY}"')/host-\1/;' \
-e 's/$/-legal-info/;'
)
---8<------8<------8<------8<---
2019-03-25 22:48:12 +01:00
|
|
|
# Version of the format of the archives we generate in the corresponding
|
|
|
|
|
# download backend:
|
|
|
|
|
BR_FMT_VERSION_git = -br1
|
support/download: make the svn backend more reproducible
Since c043ecb20ce6 (support/download: change format of archives
generated from svn), the svn backend uses the generic helper to
create reproducible archives.
That helper really does its job as expected, but the svn backend
is flawed in two ways:
- the first, most obvious breakage happens with versions older
than 1.9, as they do not support the '--show-item' option
for the 'info' action;
- the second is more involved, in that svn will by default
expand the old, legacy, deprecated, cumbersome CVS-style
keywords, in the form of revision marks like '$Date$' in a
C-style comment in a source file. These replacements are
done on checkout as well as on export, and they use local
settings, like the local locale and timezone.
This means that two people with different settings, will get
different sources when the svn-checkout or svn-export the same
revision from the same tree...
Needless to say that this is not very reproducible...
While the first is easily solved, the second is more involved.
We need to ensure that what source is used initially to compute
the hash, will also be the source that are used to check the hash.
There are basically two solutions:
1. we ensure the same environment, by forcing the timezone and
the locale to arbitrary values
2. we disable keyword expansion
For the first solution, this still leaves the possibility that we
miss some environment settings that have an impact on the keyword
expansion. It would mean that Yann's settings be used, as he did
introduce the hash for the only svn-downloaded package we have,
avrdude, settings which are:
TZ=Europe/Paris
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="fr_FR.utf8"
LC_NUMERIC="fr_FR.utf8"
The second option means that the generated archives change. That
means we'd have to bump the archive version for svn downloads, and
that we update the hashes for all the svn-downloaded packages.
We chose to go with the second option, because this is what really
makes more sense, rather than hard-coding arbitrary values in the
environment. And we also have only one svn-downloaded package,
avrdude.
And thus, we're reaching the trigger for this change: avrdude is
impacted by the CVS-keyword expansion issue:
https://svn.savannah.gnu.org/viewvc/avrdude/trunk/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js?revision=1396&view=markup
which would give two different files when checked out on different
machines:
diff -durN foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js
--- foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
+++ bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
@@ -1,6 +1,6 @@
/**
* @preserve jquery.layout 1.3.0 - Release Candidate 30.51
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
* $Rev: 303005 $
*
* Copyright (c) 2012
@@ -4718,7 +4718,7 @@
/**
* jquery.layout.state 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5074,7 +5074,7 @@
/**
* jquery.layout.buttons 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5356,7 +5356,7 @@
/**
* jquery.layout.browserZoom 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2012
* Kevin Dalman (http://allpro.net)
So we also update the hash for avrdude.
Fixes:
http://autobuild.buildroot.org/results/e3b/e3b0508047f32008ebfa83c5255ec5994b6af120/ (time issue)
http://autobuild.buildroot.org/results/48e/48e78e84b425e79cdb98c16ab40247a0fa7e9676/ (keyword expansion issue)
Reported-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Reviewed-by: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-03-10 23:59:37 +01:00
|
|
|
BR_FMT_VERSION_svn = -br2
|
support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.
However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.
To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.
So, we bump the git-specific format-version to -br1.
The %ci date has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.
As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.
Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.
For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
---8<------8<------8<------8<---
#!/bin/sh
# Find and download all packages using git as backend.
# Manually fix hashes for affected packages.
# Packages that only have a host variant
HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'
# Packages that have a non-git main _SOURCE, and/or which
# have BR_NO_CHECK_HASH_FOR for the git _SOURCE
NOT_GIT='aufs|aufs-util|xenomai|linux-headers'
export BR2_DL_DIR=$(pwd)/temp-dl-dir
make defconfig
make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
boot/vexpress-firmware/ package/ \
|sed -r -e 's,.*/([^/]+)\.mk,\1,' \
|sed -r -e '/^('"${NOT_GIT}"')$/d;' \
-e 's/^('"${HOST_ONLY}"')/host-\1/;' \
-e 's/$/-legal-info/;'
)
---8<------8<------8<------8<---
2019-03-25 22:48:12 +01:00
|
|
|
|
2014-12-11 23:52:05 +01:00
|
|
|
DL_WRAPPER = support/download/dl-wrapper
|
|
|
|
|
|
2014-02-04 16:18:51 +01:00
|
|
|
# DL_DIR may have been set already from the environment
|
2014-02-10 22:48:55 +01:00
|
|
|
ifeq ($(origin DL_DIR),undefined)
|
2014-02-04 16:18:51 +01:00
|
|
|
DL_DIR ?= $(call qstrip,$(BR2_DL_DIR))
|
2012-04-17 16:45:19 +02:00
|
|
|
ifeq ($(DL_DIR),)
|
2013-07-20 08:52:43 +02:00
|
|
|
DL_DIR := $(TOPDIR)/dl
|
2012-04-17 16:45:19 +02:00
|
|
|
endif
|
2014-02-10 22:48:55 +01:00
|
|
|
else
|
|
|
|
|
# Restore the BR2_DL_DIR that was overridden by the .config file
|
|
|
|
|
BR2_DL_DIR = $(DL_DIR)
|
|
|
|
|
endif
|
2012-04-17 16:45:19 +02:00
|
|
|
|
core: ensure we use the realpath(3) of DL_DIR
When $(TOPDIR)/dl is a symlink, checking out git submodules can fail,
as reported by Michael in #11086.
To reproduce a similarly-related mis-behaviour:
$ mkdir -p foo/bar foo/buz
$ cd foo/bar
$ ln -s ../buz meh
$ cd meh
$ cd ../../foo
The last command should not succeed, because, relative to meh, there is
no ../../foo directory; we would expect it to be ../../../foo, instead.
But since meh is a symlink to a directory, then a relative path from that
symlink is interpreted as relative to the derefrenced directory, i.e.
from buz in this case.
But where this gets even weirder, is that, if the last command is
replaced by:
$ cd ../../../foo
then it still works, too.
And that is the root of Michael's issue: the dl directory in Buildroot's
TOPDIR is a symlink to a similarly-named directory one directory higher,
which then confuses relative paths, which gets especially and noticeably
bad for git submodules.
Avoid this strangeness, and just use so-called "physical" path, i.e. a
path where all symlinks to directories have been dereferenced.
Fixes: #11086
Reported-by: Michael Nosthoff <posted@heine.so>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Michael Nosthoff <posted@heine.so>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-15 18:03:48 +01:00
|
|
|
# ensure it exists and a absolute path, derefrecing symlinks
|
|
|
|
|
DL_DIR := $(shell mkdir -p $(DL_DIR) && cd $(DL_DIR) >/dev/null && pwd -P)
|
2012-09-10 12:38:29 +02:00
|
|
|
|
2012-04-17 16:45:19 +02:00
|
|
|
#
|
|
|
|
|
# URI scheme helper functions
|
|
|
|
|
# Example URIs:
|
|
|
|
|
# * http://www.example.com/dir/file
|
|
|
|
|
# * scp://www.example.com:dir/file (with domainseparator :)
|
|
|
|
|
#
|
|
|
|
|
# geturischeme: http
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 09:06:03 +02:00
|
|
|
geturischeme = $(firstword $(subst ://, ,$(call qstrip,$(1))))
|
2018-04-02 10:14:23 +02:00
|
|
|
# getschemeplusuri: git|parameter+http://example.com
|
|
|
|
|
getschemeplusuri = $(call geturischeme,$(1))$(if $(2),\|$(2))+$(1)
|
2012-04-17 16:45:19 +02:00
|
|
|
# stripurischeme: www.example.com/dir/file
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 09:06:03 +02:00
|
|
|
stripurischeme = $(lastword $(subst ://, ,$(call qstrip,$(1))))
|
2012-04-17 16:45:19 +02:00
|
|
|
# domain: www.example.com
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 09:06:03 +02:00
|
|
|
domain = $(firstword $(subst $(call domainseparator,$(2)), ,$(call stripurischeme,$(1))))
|
2012-04-17 16:45:19 +02:00
|
|
|
# notdomain: dir/file
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 09:06:03 +02:00
|
|
|
notdomain = $(patsubst $(call domain,$(1),$(2))$(call domainseparator,$(2))%,%,$(call stripurischeme,$(1)))
|
2012-04-17 16:45:19 +02:00
|
|
|
#
|
|
|
|
|
# default domainseparator is /, specify alternative value as first argument
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 09:06:03 +02:00
|
|
|
domainseparator = $(if $(1),$(1),/)
|
2012-04-17 16:45:19 +02:00
|
|
|
|
Revert "pkg-download: remove explicit PKG_VERSION from github helper"
This reverts commit 1e5a8916b2ab4c9c99548fa6fbd4855eee323881.
The idea was that the version string can be derived because we know the
package name.
However, this patch does not account for the fact that $(pkgname) always
points to the latest pacakge scanned, which in all other situation we're
using it, is the current package, because it is used inside one ot he
xxx-inner macros that are $(eval)ualed. So $(pkgname) is evaluated
"early" and gets the expected value.
However, the github value is not in one of those macros, so it gets
evaluated "late", when doing the actual download. So, by that time,
$(pkgname) will expand to the last package scanned, which is actuall the
manual (without a br2-external tree).
That would require that the _SITE variable be assigned with the :=
assignment operator. This is weird, because that would make it the only
variable to require that, but only when using the github helper, which
is even less obvious and would cause a lot of trouble...
The obvious fixup would seem to be to use $(PKG) instead, because that
already contains the upper-case package name that vcan be used as a
prefix to variables.
However, that does not work either, because we have a check that forbids
a trsailing slash in _SITE, check that is done in pacakge/pkg-generic,
inside the xxx-inner macro, during the $(eval) call.
And at that time, PKG is not yet defined, because it is only defined for
an actual recipe.
So we can't seem to have a workable solution. So, just revert the patch.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Marcin Nowakowski <marcin.nowakowski@imgtec.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-13 23:00:09 +01:00
|
|
|
# github(user,package,version): returns site of GitHub repository
|
|
|
|
|
github = https://github.com/$(1)/$(2)/archive/$(3)
|
2013-12-05 18:20:44 +01:00
|
|
|
|
2021-03-28 21:13:49 +02:00
|
|
|
# gitlab(user,package,version): returns site of Gitlab-generated tarball
|
|
|
|
|
gitlab = https://gitlab.com/$(1)/$(2)/-/archive/$(3)
|
|
|
|
|
|
support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-23 00:08:38 +02:00
|
|
|
# Expressly do not check hashes for those files
|
2015-05-02 11:05:00 +02:00
|
|
|
# Exported variables default to immediately expanded in some versions of
|
|
|
|
|
# make, but we need it to be recursively-epxanded, so explicitly assign it.
|
|
|
|
|
export BR_NO_CHECK_HASH_FOR =
|
support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-23 00:08:38 +02:00
|
|
|
|
2012-04-17 16:45:19 +02:00
|
|
|
################################################################################
|
2019-04-15 21:47:24 +02:00
|
|
|
# DOWNLOAD_URIS - List the candidates URIs where to get the package from:
|
2018-04-02 10:14:23 +02:00
|
|
|
# 1) BR2_PRIMARY_SITE if enabled
|
|
|
|
|
# 2) Download site, unless BR2_PRIMARY_SITE_ONLY is set
|
|
|
|
|
# 3) BR2_BACKUP_SITE if enabled, unless BR2_PRIMARY_SITE_ONLY is set
|
|
|
|
|
#
|
|
|
|
|
# Argument 1 is the source location
|
2019-04-15 21:47:24 +02:00
|
|
|
# Argument 2 is the upper-case package name
|
2012-04-17 16:45:19 +02:00
|
|
|
#
|
|
|
|
|
################################################################################
|
|
|
|
|
|
2018-04-02 10:14:23 +02:00
|
|
|
ifneq ($(call qstrip,$(BR2_PRIMARY_SITE)),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 21:47:24 +02:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_PRIMARY_SITE)/$($(2)_DL_SUBDIR)),urlencode) \
|
2019-04-15 21:47:23 +02:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_PRIMARY_SITE)),urlencode)
|
2018-04-02 10:14:23 +02:00
|
|
|
endif
|
2012-04-17 16:45:19 +02:00
|
|
|
|
2018-04-02 10:14:23 +02:00
|
|
|
ifeq ($(BR2_PRIMARY_SITE_ONLY),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 21:47:23 +02:00
|
|
|
$(patsubst %/,%,$(dir $(call qstrip,$(1))))
|
2018-04-02 10:14:23 +02:00
|
|
|
ifneq ($(call qstrip,$(BR2_BACKUP_SITE)),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 21:47:24 +02:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_BACKUP_SITE)/$($(2)_DL_SUBDIR)),urlencode) \
|
2019-04-15 21:47:23 +02:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_BACKUP_SITE)),urlencode)
|
2018-04-02 10:14:23 +02:00
|
|
|
endif
|
|
|
|
|
endif
|
2013-09-11 14:12:04 +02:00
|
|
|
|
2019-04-15 21:47:24 +02:00
|
|
|
################################################################################
|
|
|
|
|
# DOWNLOAD -- Download helper. Will call DL_WRAPPER which will try to download
|
|
|
|
|
# source from the list returned by DOWNLOAD_URIS.
|
|
|
|
|
#
|
|
|
|
|
# Argument 1 is the source location
|
2019-04-15 21:47:26 +02:00
|
|
|
# Argument 2 is the upper-case package name
|
package/pkg-download: do not try to vendor _EXTRA_DOWNLOADS
For golang- or cargo-based packages, we apply a vendoring pass after the
package's "main" download is done. Whether to vendor or not is based on
the heuristic that a specific directory exists or not; for golang
packages, we look for '/vendor', while for cargo, we look for '/VENDOR'.
This is fine for the "main" (by lack of a better term) download, but
this falls flat on its face for extra downloads. Indeed, some packages
may need to download data sets, or assets, as _EXTRA_DOWNLOADS. Those
are usually just data blobs, and are not actual golang or cargo packages;
as such they do not need to be vendored, but worse, if we try to
actually vendor them, this fails because the required files for
vendoring are missing from the archives in such data sets.
We fix that by decoupling the download for the extra download, from the
download for the main archive. We pass the post-processing option only
to the main download.
This makes the hard assumption that extra downloads will never need to
be post-processed for vendoring, of course; we hope this will always be
correct in practice. If not, we can fix it later.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout:
- no loop needed for MAIN_DOWNLOAD, it can have only one;
- remove superfluous backslash in the definition of MAIN_DOWNLOAD;
- introduce _ADDITIONAL_DOWNLOADS to avoid filter-out.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-05 19:19:39 +02:00
|
|
|
# Argument 3 is a space-separated list of optional arguments
|
2019-04-15 21:47:24 +02:00
|
|
|
#
|
|
|
|
|
################################################################################
|
|
|
|
|
|
2018-04-02 10:14:23 +02:00
|
|
|
define DOWNLOAD
|
2019-04-15 21:47:26 +02:00
|
|
|
$(Q)mkdir -p $($(2)_DL_DIR)
|
2020-12-19 16:35:16 +01:00
|
|
|
$(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \
|
|
|
|
|
flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
|
2019-04-15 21:47:26 +02:00
|
|
|
-c '$($(2)_DL_VERSION)' \
|
|
|
|
|
-d '$($(2)_DL_DIR)' \
|
2018-04-02 17:13:58 +02:00
|
|
|
-D '$(DL_DIR)' \
|
2018-04-02 15:33:53 +02:00
|
|
|
-f '$(notdir $(1))' \
|
2019-04-15 21:47:26 +02:00
|
|
|
-H '$($(2)_HASH_FILE)' \
|
|
|
|
|
-n '$($(2)_BASENAME_RAW)' \
|
|
|
|
|
-N '$($(2)_RAWNAME)' \
|
|
|
|
|
-o '$($(2)_DL_DIR)/$(notdir $(1))' \
|
|
|
|
|
$(if $($(2)_GIT_SUBMODULES),-r) \
|
2020-01-21 21:33:37 +01:00
|
|
|
$(if $($(2)_GIT_LFS),-l) \
|
2019-04-15 21:47:26 +02:00
|
|
|
$(foreach uri,$(call DOWNLOAD_URIS,$(1),$(2)),-u $(uri)) \
|
package/pkg-download: do not try to vendor _EXTRA_DOWNLOADS
For golang- or cargo-based packages, we apply a vendoring pass after the
package's "main" download is done. Whether to vendor or not is based on
the heuristic that a specific directory exists or not; for golang
packages, we look for '/vendor', while for cargo, we look for '/VENDOR'.
This is fine for the "main" (by lack of a better term) download, but
this falls flat on its face for extra downloads. Indeed, some packages
may need to download data sets, or assets, as _EXTRA_DOWNLOADS. Those
are usually just data blobs, and are not actual golang or cargo packages;
as such they do not need to be vendored, but worse, if we try to
actually vendor them, this fails because the required files for
vendoring are missing from the archives in such data sets.
We fix that by decoupling the download for the extra download, from the
download for the main archive. We pass the post-processing option only
to the main download.
This makes the hard assumption that extra downloads will never need to
be post-processed for vendoring, of course; we hope this will always be
correct in practice. If not, we can fix it later.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout:
- no loop needed for MAIN_DOWNLOAD, it can have only one;
- remove superfluous backslash in the definition of MAIN_DOWNLOAD;
- introduce _ADDITIONAL_DOWNLOADS to avoid filter-out.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-05 19:19:39 +02:00
|
|
|
$(3) \
|
2015-01-02 16:53:40 +01:00
|
|
|
$(QUIET) \
|
2014-12-11 23:52:05 +01:00
|
|
|
-- \
|
2019-04-15 21:47:26 +02:00
|
|
|
$($(2)_DL_OPTS)
|
2012-04-17 16:45:19 +02:00
|
|
|
endef
|
