c9f27fdc1b
When a --transform expression is provided, it is by default also applied
to the target of a symlink.
When we create tarballs (from git or svn checkouts), we use a --transform
expression to replace the leading ./ with the package name and version.
This causes issues when a package contains symlinks that points to
./something, as the leading './' is also replaced.
Fix that by using the 'S' transformation scope flag, as described in the
tar manual:
https://www.gnu.org/software/tar/manual/html_node/transform.html#transform
In addition, several transformation scope flags are supported, that
control to what files transformations apply. These are:
‘r’ Apply transformation to regular archive members.
‘R’ Do not apply transformation to regular archive members.
‘s’ Apply transformation to symbolic link targets.
‘S’ Do not apply transformation to symbolic link targets.
‘h’ Apply transformation to hard link targets.
‘H’ Do not apply transformation to hard link targets.
Default is ‘rsh’ [...].
Fixes: #13616
This has been checked to not change any of the existing hash for any of
our git-downloaded package (some are host-only, hence the few fixups):
---8<---
$ m="$( git grep -l -E -- -br[[:digit:]]+.tar.gz boot package/ \
|awk -F/ '{print $(NF-1)}' \
|sed -r -e 's/(imx-mkimage|netsurf-buildsystem|prelink-cross|qoriq-rcw|vboot-utils)/host-\1/g' \
-e 's/$/-source/'
)"
$ make defconfig; make clean; BR2_DL_DIR=$(pwd)/trash-me make ${m}
---8<---
Note: it is unclear what the 'H' flag does nor how it works, because the
concept of "target of a hardlink" is not obvious; probably it has to do
with how tar internally detects and stores hardlinks. Since we do not
yet have any issue with hardlinks, just ignore the problem for now, and
postpone until we have an actual issue with a real test-case.
Signed-off-by: Jean-pierre Cartal <jpcartal@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- re-indent commit log
- add scriptlet to test existing hashes
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
77 lines
2.4 KiB
Bash
Executable File
77 lines
2.4 KiB
Bash
Executable File
# Generate a reproducible archive from the content of a directory
|
|
#
|
|
# $1 : input directory
|
|
# $2 : leading component in archive
|
|
# $3 : ISO8601 date: YYYY-MM-DDThh:mm:ssZZ
|
|
# $4 : output file
|
|
# $5... : globs of filenames to exclude from the archive, suitable for
|
|
# find's -path option, and relative to the input directory $1
|
|
#
|
|
# Notes :
|
|
# - the timestamp is internally rounded to the highest entire second
|
|
# less than or equal to the timestamp (i.e. any sub-second fractional
|
|
# part is ignored)
|
|
# - must not be called with CWD as, or below, the input directory
|
|
# - some temporary files are created in CWD, and removed at the end
|
|
#
|
|
# Example:
|
|
# $ find /path/to/temp/dir
|
|
# /path/to/temp/dir/
|
|
# /path/to/temp/dir/some-file
|
|
# /path/to/temp/dir/some-dir/
|
|
# /path/to/temp/dir/some-dir/some-other-file
|
|
#
|
|
# $ mk_tar_gz /path/to/some/dir \
|
|
# foo_bar-1.2.3 \
|
|
# 1970-01-01T00:00:00Z \
|
|
# /path/to/foo.tar.gz \
|
|
# '.git/*' '.svn/*'
|
|
#
|
|
# $ tar tzf /path/to/foo.tar.gz
|
|
# foo_bar-1.2.3/some-file
|
|
# foo_bar-1.2.3/some-dir/some-other-file
|
|
#
|
|
mk_tar_gz() {
|
|
local in_dir="${1}"
|
|
local base_dir="${2}"
|
|
local date="${3}"
|
|
local out="${4}"
|
|
shift 4
|
|
local glob tmp pax_options
|
|
local -a find_opts
|
|
|
|
for glob; do
|
|
find_opts+=( -or -path "./${glob#./}" )
|
|
done
|
|
|
|
# Drop sub-second precision to play nice with GNU tar's valid_timespec check
|
|
date="$(date -d "${date}" -u +%Y-%m-%dT%H:%M:%S+00:00)"
|
|
|
|
pax_options="delete=atime,delete=ctime,delete=mtime"
|
|
pax_options+=",exthdr.name=%d/PaxHeaders/%f,exthdr.mtime={${date}}"
|
|
|
|
tmp="$(mktemp --tmpdir="$(pwd)")"
|
|
pushd "${in_dir}" >/dev/null
|
|
|
|
# Establish list
|
|
find . -not -type d -and -not \( -false "${find_opts[@]}" \) >"${tmp}.list"
|
|
# Sort list for reproducibility
|
|
LC_ALL=C sort <"${tmp}.list" >"${tmp}.sorted"
|
|
|
|
# Create POSIX tarballs, since that's the format the most reproducible
|
|
tar cf - --transform="s#^\./#${base_dir}/#S" \
|
|
--numeric-owner --owner=0 --group=0 --mtime="${date}" \
|
|
--format=posix --pax-option="${pax_options}" \
|
|
-T "${tmp}.sorted" >"${tmp}.tar"
|
|
|
|
# Compress the archive
|
|
gzip -6 -n <"${tmp}.tar" >"${out}"
|
|
|
|
rm -f "${tmp}"{.list,.sorted,.tar}
|
|
|
|
popd >/dev/null
|
|
}
|
|
|
|
# Keep this line and the following as last lines in this file.
|
|
# vim: ft=bash
|