Commit Graph

24 Commits

Author SHA1 Message Date
Yegor Yefremov
e273c36ad0 utils/scanpypi: import third party modules after the standard ones
Move imports from six package after the standard modules. Resolves
pylint warnings.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-12-10 11:46:36 +01:00
Yegor Yefremov
fd29797f65 utils/scanpypi: use archive file name to specify the extraction folder
Some packages have archive name that is different from package name.
For example websocket-client's archive name is websocket_client-*.tar.gz.
scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder
will be different from the expected one because of the '_' in the
archive file name.

Use archive file name instead of package name to specify the extraction
folder. As the version is already part of this file, we don't need to
specify it.

Bonus: remove obsolete "return None, None" as the function doesn't return
anything. OSError class doesn't provide "message" member, so replace it
with "strerror".

Fixes:
https://bugs.busybox.net/show_bug.cgi?id=11251

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-02 21:35:08 +01:00
Yegor Yefremov
d05e41eb1a scanpypi: improve BSD licence handling
When used without spdx_lookup the BSD licence cannot be
detected correctly because many Python packages just specify
BSD without the exact version in their metadata. So add a
special message warning the user instead of the licence id.

Bonus: fix typo.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-11 09:32:33 +02:00
Yegor Yefremov
0101ac62c8 scanpypi: place a warning into *.mk file if licence id couldn't be detected
If a license file could be found, but license id couldn't be detected place
following warning into *.mk file:

FOO_LICENSE = FIXME: license id couldn't be detected

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-31 23:12:57 +02:00
Yegor Yefremov
fb775f4c87 scanpypi: rework download_package error handling
Some packages don't provide source archive but only a wheel file. In
this case download variable is not defined. So define this variable at
the very beginning and check whether it is None after searching for
source archives in the metadata.

Bonus: fix PEP8 issue with wrong indentation.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-06-15 14:06:43 +02:00
Yegor Yefremov
6766ff9d12 scanpypi: add support for the new PyPI infrastructure
https://pypi.python.org URL has been changed to https://pypi.org.

Package's JSON object now contains sha256 checksum, so use it
instead of locally computed one. Change comments in the hash
file accordingly.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-18 17:11:47 +02:00
Ricardo Martincoski
3ff0728480 scanpypi: generate help text compliant to check-package
Each line must fit in <tab><2 spaces><62 chars>.
The default width for textwrap.wrap() is 70, so explicit set it to 62.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 10:16:09 +02:00
Peter Korsgaard
1490558b75 utils/scanpypi: don't hardcode python2
Commit 3a0c20c530 (scanpypi: add support for Python3) adapted the script
to work with python 3.x, but the shebang still said python2 making it
unlikely to work on systems without python 2.x.

Change it to just 'python' instead.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-01 07:53:58 +02:00
Ricardo Martincoski
ede92be10d scanpypi: fix code style
Fix these warnings:
E401 multiple imports on one line

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-13 22:32:01 +01:00
Peter Korsgaard
7cfceeb91e utils/scanpypi: correctly handle license dirs in subdirs for .hash files
create_hash_file() used basename(licensefile) when it writes the entry for
the license file in the .hash, which is obviously not correct when license
file is locate in a sub directory.

Instead copy the logic from __create_mk_license() to strip the directory
prefix from the absolute filename instead.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-09 22:23:35 +01:00
Peter Korsgaard
92b8bd0879 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-05 21:04:14 +01:00
Peter Korsgaard
7cf95a8a78 utils/scanpypi: fix 'downloas' typo in error message
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-01 17:14:01 +01:00
Yegor Yefremov
c46f72b61e scanpypi: fix licence detection handling for unknown licences
Check for match object not being None.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-26 20:55:19 +01:00
Yegor Yefremov
cfafcfa9af scanpypi: fix Py2/3 conversion leftover
Use urlparse from six package.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-26 20:54:00 +01:00
Yegor Yefremov
3a0c20c530 scanpypi: add support for Python3
The script was changed via modernize utility. The only manual
made part was the handling of StringIO.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-02-25 22:21:54 +01:00
Ricardo Martincoski
2455e5a091 scanpypi: fix code style
Fix these warnings:
E101 indentation contains mixed spaces and tabs
E128 continuation line under-indented for visual indent
E231 missing whitespace after ','
E261 at least two spaces before inline comment
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
W191 indentation contains tabs

Ignore these warnings:
E402 module level import not at top of file

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-29 23:14:24 +01:00
Yegor Yefremov
09ec6d7f64 scanpypi: ignore empty elements in package requirements
Depending on how setup.py reads requirements files empty elements can occur.
This patch takes care, that such elements will be ignored and don't crash
the scanpypi script.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-01-26 14:43:28 +01:00
Yegor Yefremov
d2e29fccdf scanpypi: get rid of commented lines and also strip the package strings
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-18 22:11:09 +01:00
Yegor Yefremov
13d94669b9 scanpypi: add support for licence files checksums
Store the list of detected licence files in the main object and
automatically add their sha256 checksums when creating *.hash file.

Bonus: fix wrong indentation.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-15 22:12:47 +01:00
Yegor Yefremov
d2ac1ec6f4 scanpypi: get license names from SPDX database
Use spdx_lookup package to compare packages' license file texts
with SPDX database.

This feature is optional.

Bonus: fix wrong indentation.

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2018-01-12 22:23:50 +01:00
Alexey Roslyakov
5d2c69dad5 utils/scanpypi: increase error message verbosity
When package installation fails it is good to know what happened.

Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-12-18 15:18:40 +01:00
Alexey Roslyakov
caa3760014 utils/scanpypi: include LICENSE.RST to supported license files
Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-20 22:28:00 +02:00
Alexey Roslyakov
c2877a2a8b utils/scanpypi: don't pass any arguments to main()
'if __name__ == "__main__"' idiom typically calls main function that
doesn't take any arguments in most cases. We shouldn't pass any tuple to
it.
I've tested the script with python-idna-2.5 and now it works with this
little change.

Signed-off-by: Alexey Roslyakov <alexey.roslyakov@gmail.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-20 22:27:58 +02:00
Thomas Petazzoni
7ca9fc3170 tools: rename to 'utils'
After some discussion, we found out that "tools" has the four first
letters identical to the "toolchain" subfolder, which makes it a bit
unpractical with tab-completion. So, this commit renames "tools" to
"utils", which is more tab-completion-friendly.

This has been discussed with Arnout and Yann.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-07-01 18:07:00 +02:00