Files in a git repository can be given attributes, like the usual eol
that can convert to-from crlf, cr, lf; those are applied when comitting
or checking-out a file.
There are also two attributes that are meant to be used when generating
an archive (with git archive): export-subst, and export-ignore, that
respectively substitutes format placeholders in a file, and excludes a
file from the archive.
Some package (e.g. pcm-tools, luajit) use the export-subst attribute
to generate versioning information. luajit, specifically, uses the UNIX
timestamp of the commit as the patch-level for its semantic versioning.
We don't use git-archive, because we need to get submodules and LFS
blob, which git-archive does not handle. So, our git backend tries to
impersonate git-archive as much as possible, but the support for git
attributes was lost when we converted it from using git-archive to
manually creating the tarball in 3abd5ba424 (support/download/git: do
not use git archive, handle it manually) in preparation for f109e7eeb5
(support/download/git: add support for submodules) (arguably, a long
time ago...)
Extend the git backend to handle the export-subst attribute. There is
no git tool (that we could find) that does that automatically, except
git-archive, which we can't use; "git check-attr" however can report
whether a file has a specific attribute (and git check-attr can work
with \0-delimited fields and records).
So, we iterate over all the files in the repository, and filter those
that have the export-subst attribute set. Then for each file, we use a
bit of awk to do the replacement:
- for each line (managed natively by awk), we iterate over each
format placeholder,
- for each placeholer, we query "git log" with the requested format,
- we emit the replacement.
When doing the replacement, we decided to force abbreviating short
hashes to 40 chars, which is the length of a full sha1, rather than
actually abbreviating them:
- letting git decide of the length is not reproducible over time:
- as new commits are added, the short length will increase to avoid
collisions,
- newer git versions may decide on a different heuristic to shorten
hashes,
- users may have local settings with an arbitrary length (in their
~/.gitconfig for example);
- deciding on our side of an "small" arbitrary value would not be
viable long term either, as it might be too large to be minimum, or
too short to avoid collisions.
The only reproducible solution is to use unabbreviated hashes.
Handling git-attributes also implies that the format of the generated
archives has changed, since we now expand placeholders, so we bump our
git format version.
Hash files for all git-downloaded packages will be updated in followup
commits.
Of all our git-downloaded packages, 5 are affected, and their hashes
will be updated in a followup commit too:
- pcm-tools, which was known, and the one that triggered this commit;
since we now expand placeholders, we can drop the post-extract hook;
switching to a full hash in replacements also changes the hash of
the generated archive;
- qt5knx, qt5location, qt5mqtt, and qt5opcua: the file .tag at the
repository root, contains only the full hash placeholder; that file
is not used at all during the build (AFAICS);
Finally, a sixth package, luajit, uses export-subst; it currently relies
on the github-generated archive (because it happens to currently use a
format that is reproducible); it will also be converted in a follow-up
patch.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Woody Douglass <wdouglass@carnegierobotics.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Francois Perrad <fperrad@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Buildroot is a simple, efficient and easy-to-use tool to generate embedded
Linux systems through cross-compilation.
The documentation can be found in docs/manual. You can generate a text
document with 'make manual-text' and read output/docs/manual/manual.text.
Online documentation can be found at http://buildroot.org/docs.html
To build and use the buildroot stuff, do the following:
1) run 'make menuconfig'
2) select the target architecture and the packages you wish to compile
3) run 'make'
4) wait while it compiles
5) find the kernel, bootloader, root filesystem, etc. in output/images
You do not need to be root to build or run buildroot. Have fun!
Buildroot comes with a basic configuration for a number of boards. Run
'make list-defconfigs' to view the list of provided configurations.
Please feed suggestions, bug reports, insults, and bribes back to the
buildroot mailing list: buildroot@buildroot.org
You can also find us on #buildroot on OFTC IRC.
If you would like to contribute patches, please read
https://buildroot.org/manual.html#submitting-patches
ac92ce4961