294fc3218c
The major bottleneck in pkg-stats is the time spent waiting for answers from remote servers. Two functions involve such communication with remote servers: - 'check_package_urls' which checks that each package upstream website is up, it is efficient due to the use of process-pools thanks to Matt Weber. - 'check_package_latest_version' which fetches the latest package version from release-monitoring, it uses a http-pool but runs sequentially. This patch extends the use of process-pools to 'check_latest_version'. Due to some limitations of multiprocess callbacks, this patch loses the overall progress of packages in favour of just the current package name. Runtimes for this function are ~3m vs ~25m for the linear version. Tested on an i7 7500U (2/4 cores/threads @3.5GHz) with 15ms ping. Note: There have already been work trying to parallelize this function using threads but there were a failure on some configurations [1]. This implementation rely on a dedicated module already in use on this script, so it's unlikely to see failure with this version. [1] http://lists.busybox.net/pipermail/buildroot/2018-March/215368.html Signed-off-by: Victor Huesca <victor.huesca@bootlin.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> |
||
|---|---|---|
| .. | ||
| apply-patches.sh | ||
| br2-external | ||
| brpkgutil.py | ||
| check-bin-arch | ||
| check-host-rpath | ||
| check-kernel-headers.sh | ||
| check-merged-usr.sh | ||
| check-uniq-files | ||
| eclipse-register-toolchain | ||
| expunge-gconv-modules | ||
| fix-configure-powerpc64.sh | ||
| fix-rpath | ||
| generate-gitlab-ci-yml | ||
| genimage.sh | ||
| graph-build-time | ||
| graph-depends | ||
| hardlink-or-copy | ||
| mkmakefile | ||
| mkusers | ||
| pkg-stats | ||
| pycompile.py | ||
| setlocalversion | ||
| size-stats | ||