Some python packages seem to use underscores in inconsistent ways. We can
attempt to normalize these by always using dashes for the buildroot name and
attempting to autodetect the correct metadata name format.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For details, see https://github.com/snyk/zip-slip-vulnerability
Older python versions do not validate that the extracted files are inside
the target directory. Detect and error out on evil paths before extracting
.zip / .tar file.
Given the scope of this (zip issue was fixed in python 2.7.4, released
2013-04-06, scanpypi is only used by a developer when adding a new python
package), the security impact is fairly minimal, but it is good to get it
fixed anyway.
Reported-by: Bas van Schaik <security-reports@semmle.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For some packages, there's no need to add the _SOURCE variable, since
the name of the source file is the same as the name of the package
(like python-engineio). Hence, we'll add it to the .mk file only if
needed.
Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Move imports from six package after the standard modules. Resolves
pylint warnings.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some packages have archive name that is different from package name.
For example websocket-client's archive name is websocket_client-*.tar.gz.
scanpypi expects the temporary extract folder to be:
/tmp-folder/BR-package-name/PyPI-packagename-and-version
In the case of websocket-client package the real extraction folder
will be different from the expected one because of the '_' in the
archive file name.
Use archive file name instead of package name to specify the extraction
folder. As the version is already part of this file, we don't need to
specify it.
Bonus: remove obsolete "return None, None" as the function doesn't return
anything. OSError class doesn't provide "message" member, so replace it
with "strerror".
Fixes:
https://bugs.busybox.net/show_bug.cgi?id=11251
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When used without spdx_lookup the BSD licence cannot be
detected correctly because many Python packages just specify
BSD without the exact version in their metadata. So add a
special message warning the user instead of the licence id.
Bonus: fix typo.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
If a license file could be found, but license id couldn't be detected place
following warning into *.mk file:
FOO_LICENSE = FIXME: license id couldn't be detected
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some packages don't provide source archive but only a wheel file. In
this case download variable is not defined. So define this variable at
the very beginning and check whether it is None after searching for
source archives in the metadata.
Bonus: fix PEP8 issue with wrong indentation.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
https://pypi.python.org URL has been changed to https://pypi.org.
Package's JSON object now contains sha256 checksum, so use it
instead of locally computed one. Change comments in the hash
file accordingly.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Each line must fit in <tab><2 spaces><62 chars>.
The default width for textwrap.wrap() is 70, so explicit set it to 62.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 3a0c20c530 (scanpypi: add support for Python3) adapted the script
to work with python 3.x, but the shebang still said python2 making it
unlikely to work on systems without python 2.x.
Change it to just 'python' instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fix these warnings:
E401 multiple imports on one line
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
create_hash_file() used basename(licensefile) when it writes the entry for
the license file in the .hash, which is obviously not correct when license
file is locate in a sub directory.
Instead copy the logic from __create_mk_license() to strip the directory
prefix from the absolute filename instead.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The script was changed via modernize utility. The only manual
made part was the handling of StringIO.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fix these warnings:
E101 indentation contains mixed spaces and tabs
E128 continuation line under-indented for visual indent
E231 missing whitespace after ','
E261 at least two spaces before inline comment
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
W191 indentation contains tabs
Ignore these warnings:
E402 module level import not at top of file
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Depending on how setup.py reads requirements files empty elements can occur.
This patch takes care, that such elements will be ignored and don't crash
the scanpypi script.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Store the list of detected licence files in the main object and
automatically add their sha256 checksums when creating *.hash file.
Bonus: fix wrong indentation.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Use spdx_lookup package to compare packages' license file texts
with SPDX database.
This feature is optional.
Bonus: fix wrong indentation.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When package installation fails it is good to know what happened.
Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
'if __name__ == "__main__"' idiom typically calls main function that
doesn't take any arguments in most cases. We shouldn't pass any tuple to
it.
I've tested the script with python-idna-2.5 and now it works with this
little change.
Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
After some discussion, we found out that "tools" has the four first
letters identical to the "toolchain" subfolder, which makes it a bit
unpractical with tab-completion. So, this commit renames "tools" to
"utils", which is more tab-completion-friendly.
This has been discussed with Arnout and Yann.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
