fbddd12601
Commit8450b76918(package/pkg-cargo: move CARGO_HOME into DL_DIR) allowed for a shared cargo cache of crates. Internally, cargo is supposed to lock themselves when accessing that cache, and that commit even had some research in that area, pointing at [0] for complaints about too-coarse the lock, so it was deemed safe to have a shared cargo home. However, in practice, the locking as implemented by cargo, fails to properly protect the concurrent accesses to the crates cache, with random failures that manifest themselves like so: Blocking waiting for file lock on package cache Blocking waiting for file lock on package cache Downloading crates ... error: failed to sync Caused by: failed to download packages Caused by: failed to download `autocfg v1.1.0` Caused by: unable to get packages from source Caused by: failed to unpack package `autocfg v1.1.0` Caused by: failed to unpack entry at `autocfg-1.1.0/src/tests.rs` Caused by: No such file or directory (os error 2) while canonicalizing [...] with the last few errors sometime being: Caused by: failed to parse manifest at `[...]/aho-corasick-0.7.18/Cargo.toml` Caused by: can't find library `aho_corasick`, rename file to `src/lib.rs` or specify lib.path So, as we do not systematically use our own cargo build (we can use a pre-built one with host-rust-bin), we can't patch cargo (even if we knew what to do!). Instead, we implement a lock ourselves, by wrapping the call to "cargo vendor" with a flock(1) on cargo home. Note: the download wrapper is already flock-ed, but it is a per-package lock, so it does not prevent different packages from being downloaded in parallel; if those packages need cargo vendoring, that will not be protected by the flock on the dl wrapper. So we really do need a flock on cargo home. [0] https://github.com/rust-lang/cargo/issues/6930 Fixes:8450b76918Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Moritz Bitsch <moritz@h6t.eu> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
39 lines
856 B
Bash
Executable File
39 lines
856 B
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
set -e
|
|
set -o pipefail
|
|
|
|
. "${0%/*}/helpers"
|
|
|
|
while getopts "n:o:" OPT; do
|
|
case "${OPT}" in
|
|
o) output="${OPTARG}";;
|
|
n) base_name="${OPTARG}";;
|
|
:) error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
|
|
\?) error "unknown option '%s'\n" "${OPTARG}";;
|
|
esac
|
|
done
|
|
|
|
# Already vendored tarball, nothing to do
|
|
if tar tf "${output}" | grep -q "^[^/]*/VENDOR" ; then
|
|
exit 0
|
|
fi
|
|
|
|
post_process_unpack "${base_name}" "${output}"
|
|
|
|
# Do the Cargo vendoring
|
|
pushd "${base_name}" > /dev/null
|
|
|
|
# Create the local .cargo/config with vendor info
|
|
mkdir -p .cargo/
|
|
mkdir -p "${CARGO_HOME}"
|
|
flock "${CARGO_HOME}"/.br-lock \
|
|
cargo vendor \
|
|
--manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} \
|
|
--locked VENDOR \
|
|
| tee .cargo/config
|
|
|
|
popd > /dev/null
|
|
|
|
post_process_repack "$(pwd)" "${base_name}" "${output}"
|