Some packages download files (especially patches) with a ? in the
URL. The ? marks the query part of the URL. However, the downloaded
file still contains the ? but from then on it doesn't designate a
query part anymore. Therefore, when fetching from PRIMARY or
BACKUP site over http, the server will report a 404 Not Found.
To fix, we need to replace the ? with %3F. Obviously, this should
be done only when fetching from PRIMARY or BACKUP. For fetching
from the real upstream, the ? really does designate the query part.
Fixes#9371.
Reported-by: Johan Derycke <johanderycke@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Thomas: replace 'SECONDARY' by 'BACKUP'.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
For packages that use a version control repository rather than a pre-made
tarball, the directory prefix used inside the tarball is currently
FOO_BASE_NAME, which can be 'foo' or 'host-foo'.
This means that the hash of such tarball will be different for target and
host packages, even though the contents are exactly the same. Hence, if the
hash file is created based on 'foo', and later a fresh build is made where
'host-foo' happens to be built before 'foo' (with a different config, for
example), the hash will be detected as incorrect and a new download is
started.
This problem does not affect many packages/users, due to the number of
conditions to be met:
- the package should be available for target _and_ host
- the package needs to use a VCS download method, e.g. git, hg, svn, ...
This does not include standard github downloads, which download a pre-made
archive.
- there should be a hash file containing the hash of the downloaded archive.
Since normally there is no hash file for packages with sources coming from
a version control system, this restricts even further. Some examples of
packages in this category that do have a hash file (but not necessarily
match the earlier conditions): expedite, vexpress-firmware, squashfs, ...
- the archive needs to be stored in a 'primary site' after initial archiving
and thus be downloaded later using a non-version-controlled method, like
wget or scp. This is because the version control download methods do not
receive a '-H' parameter pointing to the hash file and thus no hashes are
checked at all even if the file is present.
While packages matching the third condition could be considered to be 'wrong'
and need to be fixed, it does actually makes sense to have a hash file for
packages from version control, in particular if they are stored in a
primary site as mentioned in the last condition.
Regardless of any different opinions on the previous paragraph, it is also
not conceptually correct that a tarball of a package source can contain a
Buildroot-specific directory prefix 'host-'. Therefore, use
FOO_RAW_BASE_NAME instead of FOO_BASE_NAME when calling the dl-wrapper.
Example test scenario that exhibits the problem:
$ rm -rf /tmp/dl dl/squashfs-9c1db6d13a51a2e009f0027ef336ce03624eac0d.tar.gz
$ make qemu_x86_64_defconfig
$ make host-squashfs-dirclean host-squashfs-source
$ mkdir /tmp/dl
$ mv dl/squashfs-9c1db6d13a51a2e009f0027ef336ce03624eac0d.tar.gz /tmp/dl/
$ sed -i -e 's,BR2_PRIMARY_SITE=.*,BR2_PRIMARY_SITE="file:///tmp/dl",' \
-e '/BR2_PRIMARY_SITE/aBR2_PRIMARY_SITE_ONLY=y' .config
$ make host-squashfs-dirclean host-squashfs-source
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Introduce a new package variable $(PKG)_DL_OPTS. When this variable
is defined, its value is passed to the downloader as options to
the underlying command. Packages can now retrieve archives from server
expecting logins and passwords, use referer url, proxy or specific
options for cloning a repository.
Signed-off-by: Romain Perier <romain.perier@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a new package variable that packages can set to specify that they
need git submodules.
Only accept this option if the download method is git, as we can not get
submodules via an http download (via wget).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Aleksandar Simeonov <aleksandar@barix.com>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a download via a VCS method fails, Buildroot attempts the download
from the backup mirror (if any is set). Such a download is done with the
wget helper.
Given a package that has a .hash file for relase tarballs, which also
allows the user to use a random revision from a VCS, the normal download
wrapper will not check for hashes, on the assumption that maybe they are
not reproducible [*] (or user configurable).
However, when the download fails (for any reason: network error,
upstream removed the revision, old machine with incomplete set of CA
certificates...), the backup download now proceeds over with http (or
https) with the wget wrapper. The wget wrapper *always* checks for
hashes, and when a .hash file is present but no hash is found for the
download, this is considered an error.
However, when we download from the backup mirror for a main download that
should have been done with git/hq/svn/.., we in this situation have no hash
for the download. Thus, we should not fail on a missing hash for that
download. Same situation if a primary site is used.
Add a test for the site-method in the DOWNLOAD_INNER macro; if it was either
one of the VCS method, pass a one-off BR_NO_CHECK_HASH_FOR variable set to
contain the tarball to download.
Fixes issues like those reported by the Travis build bots:
https://travis-ci.org/buildroot/buildroot-defconfig-testing/jobs/123624879
[Peter: Move logic to beginning of DOWNLOAD_INNER so it also applies for BR2_PRIMARY_SITE]
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
SITE_METHOD is always set in inner-generic-package. So, it is useless
to test it in pkg-download.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This feature was already discuted here:
http://lists.busybox.net/pipermail/buildroot/2015-April/125419.html
Personally, I have a big central directory with all tarballs I have
ever downloaded. I use this feature to isolate tarballs necessary to
build a configuration:
make project_defconfig
make BR2_DL_DIR=/tmp/mirror-project BR2_PRIMARY_SITE=file:///home/user/dl source
tar -C /tmp -czvf mirror-project.tgz mirror-project
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Following commit 95a572282e (pkg-infra: move the git download helper to a
script, 2014-07-02), move the comment describing the shallow clone trickery as
well. Merge this comment with the existing helper comment that was added in
7e40a1103a (support/download: convert git to use the wrapper, 2014-08-03).
Rename $($(PKG)_DL_VERSION) to ${cset} to match the helper code context.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, assigning to BR_NO_CHECK_HASH_FOR but referencing variable
assigned to later, fails. Here's a failing test-case, which is the
reduced test-case of how we handle BR_NO_CHECK_HASH_FOR for now (warning
added for test-case):
export BR_NO_CHECK_HASH_FOR
BR_NO_CHECK_HASH_FOR += $(XENOMAI_SOURCE)
XENOMAI_VERSION = 2.6.4
XENOMAI_SOURCE = xenomai-$(XENOMAI_VERSION).tar.bz2
$(warning BR_NO_CHECK_HASH_FOR='$(BR_NO_CHECK_HASH_FOR)')
all:
Run it with simply make:
$ make
BR_NO_CHECK_HASH_FOR=' '
Now, change the first line to read:
export BR_NO_CHECK_HASH_FOR =
And we now get:
$ make
BR_NO_CHECK_HASH_FOR=' xenomai-2.6.4.tar.bz2'
This new behaviour will be needed later for Xenomai, which handles the
version string in an unusual way, so we can exclude its custom versions.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some of the SOURCE_CHECK_* macros are using a non-standard two-spaces
indentation. This commit switches them to use a single tab based
indentation, like in the rest of Buildroot.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[with git show -w]
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
The DL_MODE variable is now no longer used with any other value than
"DOWNLOAD", so it no longer makes sense to have this variable at
all. Therefore, this commit gets rid of it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
As part of moving to a package infrastructure based source-check
implementation, we are going to move away from the global DL_MODE
variable to select the behavior of the DOWNLOAD_INNER macro.
As a preparation to this, this commit makes the DOWNLOAD_INNER macro
take a third argument, which is the action to be done: either DOWNLOAD
or SOURCE_CHECK. For now, the DOWNLOAD macro passes $(DL_MODE) as this
third argument, in order to keep the existing behavior.
In addition, a SOURCE_CHECK macro is added, which calls DOWNLOAD_INNER
with the appropriate action. This macro will be used in the upcoming
package infra based implementation of source-check.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested by doing a "make source" on a randpackageconfig]
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Now that the external-deps implementation relies on the per-package
<pkg>-all-external-deps and <pkg>-external-deps targets and no longer
on the 'source' target with a custom DL_MODE, we can get rid of the
support for the SHOW_EXTERNAL_DEPS DL_MODE value in the pkg-download
logic.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When downloading from a repository, we have no way to ensure the
reproducibility of the generated archives, so we can't check the hashes.
Do not specifiy a hash file in those cases.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Do not hard-code QUIET in our download commands, since it is handled in
the backends.
Suggested by Fabio.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
If doing a silent build (make -s -> QUIET=-q), silence all downloads,
by passing the -q flag downward to backends as well as to check-hash.
Change a printf to use the trace functions.
Signed-off-by: Fabio Porcedda <fabio.porcedda@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of repeating the check in our download rules, delegate the check
of the hashes to the download wrapper.
This needs three different changes:
- add a new argument to the download wrapper, that is the full path to
the hash file; if the hash file does not exist, that does not change
the current behaviour, as the existence of the hash file is checked
for in the check-hash script;
- add a third argument to the check-hash script, to be the basename of
the file to check; this is required because we no longer check the
final file with the final filename, but an intermediate file with a
temporary filename;
- do the actual call to the check-hash script from within the download
wrapper.
This further paves the way to doing pre-download checks of the hashes
for the locally cached files.
Note: this patch removes the check for hashes for already downloaded
files, since the wrapper script exits early. The behaviour to check
localy cached files will be restored and enhanced in the following
patch.
[Thomas: fix minor typo in comment.]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of repeating the same test again and again in all our download
rules, just delegate the check for an already downloaded file to the
download wrapper.
This clears up the path for doing the hash checks on a cached file
before the download.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Instead of relying on argument ordering, use actual options in the
download wrapper.
Download backends (bzr, cp, hg...) are left as-is, because it does not
make sense to complexify them, since they are almost very trivial shell
scripts, and adding option parsing would be really overkill.
This commit also renames the script to dl-wrapper so it looks better in
the traces, and it is not confused with another wrapper.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Fixes (or atleast stops the build with a saner description):
http://autobuild.buildroot.net/results/1a9/1a9643f8633db038d4fe5ca4a32e4d52e70a3a1a/
We're using the same sources for host and target downloads, so it makes
sense to use the same <pkg>.hash file as well to ensure the host version
gets verified without us having to maintain a seperate host-<pkg>.hash file.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This drastically simplifies the wget helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make busybox-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the svn helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make open2300-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the scp helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by setting a primary site to 'scp://localhost:/tmp' and
running 'make vim-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the hg helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make vim-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the git helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by running 'make fmc-fsl-sdk-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the cvs helper, as it no longer has to deal
with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the localfiles helper, as it no longer has
to deal with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
(Tested by setting BUSYBOX_SITE = file:///tmp and running 'make busybox-source')
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This drastically simplifies the bzr helper, as it no longer has to
deal with atomically saving the downloaded archive.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The cp helperscript expects a filename as target, but just the folder
name was given.
Signed-off-by: Andreas Naumann <anaumann@ultratronik.de>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
[tested a particular scenario that used to fail, when the 'hg archive'
step is interrupted, now working fine]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Maintaining the download helpers in the Makefile has proved to be a bit
complex, so move it to a shell script.
[Peter: redirect pushd/popd output to /dev/null]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The git download helper is getting a bit more complex. Fixing it in the
Makefile when it breaks (like the recent breakage with a non-existing
sha1-cset) proves to be challenging, to say the least.
Move it into a shell script in support/download/git, which will make
it much easier to read, maintain, fix and enhance in the future.
[Peter: redirect pushd/popd output to /dev/null]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
If a SVN path has been deleted in a future revision, downloading a
package's source will fail using 'svn checkout -r'. Fix using
URL@REV syntax since the SVN command will then verify the path at the
specified version. Without using the URL@REV syntax, the SVN command
will attempt to verify the path exists on the the HEAD revision before
checking out the path at the specified revision.
Error seen if SVN path has been deleted in HEAD using svn checkout -r:
svn: E160013: '/svn/path/' path not found
Signed-off-by: Ryan Barnett <ryan.barnett@rockwellcollins.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The correct capitalisation pattern is "GitHub"; fix manual and makefile
commentary.
Signed-off-by: Simon Dawson <spdawson@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Once again, github updates the source download url.
Even if only the zip archive link is advertised on the repositories'
page, the *.tar.gz is still available.
It is worthy to note that the tarball's content differs depending if
it has been fetched from the former and the new url (the root directory
name changes).
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the DOWNLOAD helper accepts one or two arguments:
- the source URI
- the source filename; if missing, it is derived from the URI
But we have no caller of DOWNLOAD that passes a second argument.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The legacy support for the old BUILDROOT_DL_DIR and BUILDROOT_CONFIG
breaks down when make is invoked recursively - which is done in a few
cases, e.g. silentoldconfig, external-defs, ... These targets always
give a legacy error.
For BUILDROOT_DL_DIR, this is fixed by making sure that the original
value of BR2_DL_DIR taken from the environment is also exported again.
For BUILDROOT_CONFIG, this is fixed with an additional comparison of
the environment variable's value with the fake value that we introduce
ourselves.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To make the naming consistent (all user-visible options should be
prefixed BR2_).
An entry is added to Makefile.legacy to warn users who have set
BUILDROOT_DL_DIR but not BR2_DL_DIR.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The current git download helper creates the tarball by doing:
git archive <version> | gzip -c > <tarball>
Unfortunately, even if "git archive" fails and returns a non-zero
error code, gzip ignores that, compresses nothing, and returns success
(zero error code). The consequence of this behavior is that when the
git version provided in the package is incorrect, we are not failing
during the download step, but later on when trying to extract the
tarball (which was incorrectly created as a result of the failing git
archive).
To fix this, we change the tarball creation logic to:
git archive -o <tarball>.tmp <version> &&
gzip -c <tarball>.tmp > <tarball> &&
rm -f <tarball>.tmp
If the build is interrupted during the "gzip" command, we may leave
the .tmp file behind us, but this also happens with wget downloads,
and is generally not considered a problem, since this temporary file
will be overwritten next time we attempt to do download this package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Previously, when cloning a Mercurial repository,
aborting the clone could (in some cases) leave behind an empty directory.
As a result, a later clone would fail.
Signed-off-by: Mathias De Maré <mathias.demare@gmail.com>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In order to make it easier to cope with changes in github download-URL's,
this patch introduces the github helper function. It generates the site
URL of a github repository. It's usage is $(call github,user,pkg,version).
[Thomas: extracted from Mischa original commit, macro moved to
pkg-download.mk]
Signed-off-by: Mischa Jonker <mjonker@synopsys.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
