Commit 768f9f80f6 (support/download: generate even more reproducible
tarballs) causes non-reproducibility in tarballs we previousy
generated, especially the archives for two cargo-vendored packages,
ripgrep and sentry-cli.
The cause is that those two pakcages eventually vendor a file that has
the u+x bit set, but is otehrwise go-x. With 768f9f80f6, the files are
now go+x, so the hash for those generated archives has changed.
Besides, that commit was wrong: it did not account for the 'r' bit for
go part, leaving some non-reproducibility still unaccounted for.
So, to generate really reproducible archives, we would need to fix that
read bit as well, and that has the potential to affect all the archives
we generated so far. If we wanted to do so, we'd need a way to version
all generated archives, like we do for git and svn, but now for all the
different CVSes, as well as for all the vendoring post-processes.
For 768f9f80f6, all that was of conern was the working copies of CVSes
(i.e. git, svn, cvs...) that we cache in the Buildroot download dir, not
the temporary files during post-processing. Indeed, in that latter case,
the user has virtually no way to mangle with the mode of the
intermediate extract before repack.
And we do have a big fat warning that users should not attempt to meddle
with the git tree that Buildroot caches.
As 768f9f80f6 however demonstrates, is that it took quite a long time
between the introduction of the git caching, and the time someone
eventually discovered they could meddle in there. This shows that the
issue it not actually critical in most setups.
Also, the tar manual [0] hints at a better solution to handle
reproducibility, which even avoids touching the files on disk which is
even nicer:
‘--mode='go+u,go-w'’
Omit irrelevant information about file permissions.
If we were to actually handle the mode bit for reproducibility, we'd
need to:
- introduce archive versioning for all download backends and
prost-processing
- use the tar officially suggested method
So, revert that change, as it was incomplete, was not really fixing much
issues, and causes actual issues.
This reverts commit 768f9f80f6.
[0] https://www.gnu.org/software/tar/manual/tar.html#Reproducibility
Thanks to Vincent and Arnout for pointing at the tar manual.
Reported-by: Antoine Coutant <antoine.coutant@smile.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Antoine Coutant <antoine.coutant@smile.fr>
9fbd3d8574