2014-10-21 16:05:56 +02:00
|
|
|
#!/usr/bin/env bash
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
set -e
|
|
|
|
|
|
|
|
# Helper to check a file matches its known hash
|
|
|
|
# Call it with:
|
2015-04-08 02:53:36 +02:00
|
|
|
# $1: the path of the file containing all the expected hashes
|
2014-12-11 23:52:07 +01:00
|
|
|
# $2: the full path to the temporary file that was downloaded, and
|
|
|
|
# that is to be checked
|
|
|
|
# $3: the final basename of the file, to which it will be ultimately
|
|
|
|
# saved as, to be able to match it to the corresponding hashes
|
|
|
|
# in the .hash file
|
2015-04-01 00:15:04 +02:00
|
|
|
#
|
|
|
|
# Exit codes:
|
|
|
|
# 0: the hash file exists and the file to check matches all its hashes,
|
|
|
|
# or the hash file does not exist
|
|
|
|
# 1: unknown command-line option
|
|
|
|
# 2: the hash file exists and the file to check does not match at least
|
|
|
|
# one of its hashes
|
|
|
|
# 3: the hash file exists and there was no hash to check the file against
|
|
|
|
# 4: the hash file exists and at least one hash type is unknown
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
|
2015-01-02 16:53:39 +01:00
|
|
|
while getopts :q OPT; do
|
|
|
|
case "${OPT}" in
|
|
|
|
q) exec >/dev/null;;
|
|
|
|
\?) exit 1;;
|
|
|
|
esac
|
|
|
|
done
|
|
|
|
shift $((OPTIND-1))
|
|
|
|
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
h_file="${1}"
|
|
|
|
file="${2}"
|
2014-12-11 23:52:07 +01:00
|
|
|
base="${3}"
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
|
2015-04-06 14:13:06 +02:00
|
|
|
# Bail early if no hash to check
|
|
|
|
if [ -z "${h_file}" ]; then
|
|
|
|
exit 0
|
|
|
|
fi
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
# Does the hash-file exist?
|
2015-04-06 14:13:06 +02:00
|
|
|
if [ ! -f "${h_file}" ]; then
|
2015-04-01 00:15:07 +02:00
|
|
|
printf "WARNING: no hash file for %s\n" "${base}" >&2
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
exit 0
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Check one hash for a file
|
2017-09-11 21:32:32 +02:00
|
|
|
# $1: algo hash
|
|
|
|
# $2: known hash
|
|
|
|
# $3: file (full path)
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
check_one_hash() {
|
|
|
|
_h="${1}"
|
|
|
|
_known="${2}"
|
|
|
|
_file="${3}"
|
|
|
|
|
2014-07-03 21:36:22 +02:00
|
|
|
# Note: md5 is supported, but undocumented on purpose.
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
# Note: sha3 is not supported, since there is currently no implementation
|
|
|
|
# (the NIST has yet to publish the parameters).
|
|
|
|
case "${_h}" in
|
|
|
|
md5|sha1) ;;
|
|
|
|
sha224|sha256|sha384|sha512) ;;
|
|
|
|
*) # Unknown hash, exit with error
|
|
|
|
printf "ERROR: unknown hash '%s' for '%s'\n" \
|
2014-12-11 23:52:07 +01:00
|
|
|
"${_h}" "${base}" >&2
|
2015-04-01 00:15:04 +02:00
|
|
|
exit 4
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
;;
|
|
|
|
esac
|
|
|
|
|
|
|
|
# Do the hashes match?
|
|
|
|
_hash=$( ${_h}sum "${_file}" |cut -d ' ' -f 1 )
|
|
|
|
if [ "${_hash}" = "${_known}" ]; then
|
2014-12-11 23:52:07 +01:00
|
|
|
printf "%s: OK (%s: %s)\n" "${base}" "${_h}" "${_hash}"
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
return 0
|
|
|
|
fi
|
|
|
|
|
2014-12-11 23:52:07 +01:00
|
|
|
printf "ERROR: %s has wrong %s hash:\n" "${base}" "${_h}" >&2
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
printf "ERROR: expected: %s\n" "${_known}" >&2
|
|
|
|
printf "ERROR: got : %s\n" "${_hash}" >&2
|
|
|
|
printf "ERROR: Incomplete download, or man-in-the-middle (MITM) attack\n" >&2
|
|
|
|
|
2015-04-01 00:15:04 +02:00
|
|
|
exit 2
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
# Do we know one or more hashes for that file?
|
|
|
|
nb_checks=0
|
|
|
|
while read t h f; do
|
|
|
|
case "${t}" in
|
|
|
|
''|'#'*)
|
|
|
|
# Skip comments and empty lines
|
|
|
|
continue
|
|
|
|
;;
|
|
|
|
*)
|
2014-12-11 23:52:07 +01:00
|
|
|
if [ "${f}" = "${base}" ]; then
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
check_one_hash "${t}" "${h}" "${file}"
|
|
|
|
: $((nb_checks++))
|
|
|
|
fi
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
done <"${h_file}"
|
|
|
|
|
|
|
|
if [ ${nb_checks} -eq 0 ]; then
|
support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-23 00:08:38 +02:00
|
|
|
case " ${BR_NO_CHECK_HASH_FOR} " in
|
|
|
|
*" ${base} "*)
|
|
|
|
# File explicitly has no hash
|
|
|
|
exit 0
|
|
|
|
;;
|
|
|
|
esac
|
2015-04-01 00:15:06 +02:00
|
|
|
printf "ERROR: No hash found for %s\n" "${base}" >&2
|
2015-04-23 00:08:41 +02:00
|
|
|
exit 3
|
pkg-infra: add possiblity to check downloaded files against known hashes
Some of the packages that Buildroot might build are sensitive packages,
related to security: openssl, dropbear, ca-certificates...
Some of those packages are downloaded over plain http, because there is
no way to get them over a secure channel, such as https.
In these dark times of pervasive surveillance, the potential for harm that
a tampered-with package could generate, we may want to check the integrity
of those sensitive packages.
So, each package may now provide a list of hashes for all files that needs
to be downloaded, and Buildroot will just fail if any downloaded file does
not match its known hash, in which case it is removed.
Hashes can be any of the md5, sha1 or sha2 variants, and will be checked
even if the file was pre-downloaded.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-07-03 21:36:21 +02:00
|
|
|
fi
|