Start counting the line numbers in 1 instead of 0, in case an error
must be printed.
Both the error about a developer entry with no file entry and the error
about a file entry with no developer entry actually belong to the
non-empty line previous the one being analysed, so in these cases print
the line number from the line before.
Also count empty and comment lines, so a developer fixing the file can
jump to the correct line (or the nearest one).
At same time standardize the messages, printing the line number
also in the case of a warning for a file that is not in the tree
anymore.
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently 4 types of parsing errors/warnings can be found:
- entry for a file that is not in the tree anymore (warning)
- developer entry with no file entry (error)
- file entry with no developer (error)
- entry that is not a developer, a file or a comment (hard error)
Currently only the last one ends the script with -v with error code.
Make all 3 error types into hard errors and bail out at the first error
found, because the rest of the state machine is not designed to handle
malformed input.
Suggested-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In preparation to the introduction of test cases for the
get-developers script, make it possible to pass a custom DEVELOPERS
file. A normal user of get-developers will most likely never use this,
but the test suite will use it.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: extracted from a larger patch from Ricardo, initially posted
at https://patchwork.ozlabs.org/project/buildroot/patch/20220528014832.289907-1-ricardo.martincoski@gmail.com/]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Currently, by following the instructions in the manual and querying for
developers for a patch that changes path
package/foobar
the script reports both developers that have these entries in the
DEVELOPERS file:
F: package/foo/
F: package/foobar/
Starting from commit "afc112b0e4 utils/getdeveloperlib.py: fix issue
with hasfile()" get-developers script uses os.path.abspath() and
os.path.relpath().
The catch is that those functions return the absolute path and the
relative path without the trailing slash.
When the paths associated to a developer are then compared to the paths
a patch touches, using the string.startswith(), any substring returns
True, leading to developers for package/foo/ being wrongly reported
for package/foobar/ .
Fix this by re-adding the trailing slash after using relpath().
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Heiko Thiery <heiko.thiery@gmail.com>
Cc: James Knight <james.d.knight@live.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In commit
40bb37bd70 ("utils/getdeveloperlib.py:
use relative paths for files"), the Developer class was changed to use
relative paths, including for its .hasfile() method.
However the check_developers() function of getdeveloperlib.py was not
updated accordingly, and continued to pass absolute paths. This caused
"get-developers -c" to return the entire list of files in Buildroot as
being unmaintained, as none of them were matching the file listed in
the DEVELOPERS file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In Python 3.x, check_output() returns a "bytes" array, and not a
string. Its result needs to be decoded to be turned into a
string. Without this fix, "get-developers -c" bails out with:
Traceback (most recent call last):
File "/home/thomas/projets/buildroot/./utils/get-developers", line 105, in <module>
__main__()
File "/home/thomas/projets/buildroot/./utils/get-developers", line 53, in __main__
files = getdeveloperlib.check_developers(devs)
File "/home/thomas/projets/buildroot/utils/getdeveloperlib.py", line 280, in check_developers
files = subprocess.check_output(cmd).strip().split("\n")
TypeError: a bytes-like object is required, not 'str'
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).
Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import
Tested with python3 -m py_compile.
[1] https://www.python.org/doc/sunset-python-2/
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Force relative file path resolution of DEVELOPERS file entries to use
forward-slash separators since pattern matching assumes forward slashes.
This is to help permit uses invoking `get-developers` on Platforms where
`os.sep` may not be a forward slash.
Signed-off-by: James Knight <james.d.knight@live.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Explicitly indicate the file encoding to UTF-8 for the DEVELOPERS
document. This prevents Unicode decoding errors when printing E-Mail
entries with Unicode characters on systems using an alternative default
encoding (e.g. 'CP1252').
This corrects the following observed error:
$ ./utils/get-developers outgoing/*
Traceback (most recent call last):
File "utils\get-developers", line 105, in <module>
__main__()
File "utils\get-developers", line 47, in __main__
devs = getdeveloperlib.parse_developers()
File "...\buildroot\utils\getdeveloperlib.py", line 239, in parse_developers
for line in f:
File "...\Python<ver>\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 6659: character maps to <undefined>
Signed-off-by: James Knight <james.d.knight@live.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When a developer has package/pkg-<infra>.mk assigned to him/her in the
DEVELOPERS file, this has 3 implications:
(1) Patches adding new packages using this infrastructure are Cc'ed
to this developer. This is done by the analyze_patch() function,
which matches the regexp r"^\+\$\(eval
\$\((host-)?([^-]*)-package\)\)$" in the patch, i.e where an
added line contains a reference to the infra maintained by the
developer.
(2) Patches touching the package/pkg-<infra>.mk file itself are Cc'ed
to this developer.
(3) Any patch touching a package using this infra are also Cc'ed to
this developer.
Point (3) causes a significant amount of patches to be sent to
developers who have package/pkg-generic.mk and
package/pkg-autotools.mk assigned to them in the DEVELOPERS
file. Basically, all patches touching generic or autotools packages
get CC'ed to such developers, which causes a massive amount of patches
to be received.
So this patch adjusts the getdeveloperlib.py to drop point (3), but
preserves point (1) and (2). Indeed, it makes sense to be Cc'ed on new
package additions (to make a review that they use the package
infrastructure correctly), and it makes sense to be Cc'ed on patches
that touch the infrastructure code itself.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Using absolute paths within getdeveloperlib isn't very sensible, it
makes a lot more sense to handle everything as relative paths from the
top-level Buildroot source directory.
parse_developers() is changed to no longer take the base path as
argument: it is automatically calculated based on the location of
utils/getdeveloperlib.py. Then, the rest of the logic is adjusted to
use relative paths, and prepend them with the base "brpath" when
needed.
This commit allows pkg-stats to report correct developers information
even when executed from an out of tree directory.
Before this patch:
$ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json
$ cat out.json | jq '.packages.ipmitool.developers'
[]
$ cat out.json | jq '.defconfigs.stm32f469_disco'
{
"name": "stm32f469_disco",
"path": "configs/stm32f469_disco_defconfig",
"developers": []
}
After this patch:
$ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json
$ cat out.json | jq '.packages.ipmitool.developers'
[
"Floris Bos <bos@je-eigen-domein.nl>",
"Heiko Thiery <heiko.thiery@gmail.com>"
]
$ cat out.json | jq '.defconfigs.stm32f469_disco'
{
"name": "stm32f469_disco",
"path": "configs/stm32f469_disco_defconfig",
"developers": [
"Christophe Priouzeau <christophe.priouzeau@st.com>"
]
}
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
pkg-stats is not able anymore to set the developers for defconfigs and
packages. This issue is introduced with
ae86067a15. The hasfile() method from
Developer object tries to check an absolute path against a relative path.
Convert the filepath to be checked also into an absolute path.
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Fix a weird blank-line missing and 2 trailing blank-space.
Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch extends the Developer class so that it associates each
developer with the defconfigs (in configs/) is in responsible for,
according to the DEVELOPERS file.
It will allow using the getdeveloperlib module to find which developer
is responsible for which defconfig, and send e-mail notifications of
defconfig build failures.
Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch extends the Developer class so that it associates each
developer with the runtime tests (in support/testing) is in
responsible for, according to the DEVELOPERS file.
The implementation relies on the unittest module to list all test
cases and does some manual parsing of these test-case objects to get
the actual list of test-cases per test-suite.
A global variable is used to compute the list of unittest only once.
This feature will allow to use the getdeveloperlib module to find
which developer is responsible for which runtime test, and send e-mail
notifications of runtime tests failures.
Signed-off-by: Victor Huesca <victor.huesca@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Instead of stdout where it gets mixed with the normal output, confusing
software parsing the output (E.G. get-developers -e as git sendemail.ccCmd).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This reverts commit 62d5558f76.
This actually does not work, as patches contain paths relative to the
toplevel directory as well.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
get-developers tries to open DEVELOPERS in the current directory, so it
breaks when calling it from elsewhere than the toplevel Buildroot directory.
Traceback (most recent call last):
File "../utils/get-developers", line 107, in <module>
__main__()
File "../utils/get-developers", line 26, in __main__
devs = getdeveloperlib.parse_developers(os.path.dirname()
File "/home/peko/source/buildroot/utils/getdeveloperlib.py", line 161, in parse_developers
with open(os.path.join(basepath, "DEVELOPERS"), "r") as f:
IOError: [Errno 2] No such file or directory: '/home/peko/source/buildroot/output-foo/DEVELOPERS'
Fix it by instead figuring out where the DEVELOPERS file is relative to the
location of get-developers (E.G. one level up).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
[Arnout:
- add realpath to support a symlinked get-developers script;
- pass devs_dir argument to check_developers() to support -c in subdir;
- convert basepath to absolute path to support -f option.
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fix these warnings:
E202 whitespace before ']'
E203 whitespace before ':'
E302 expected 2 blank lines, found 1
E305 expected 2 blank lines after class or function definition, found 1
E711 comparison to None should be 'if cond is None:'
E741 ambiguous variable name 'l'
F401 'sys' imported but unused
W391 blank line at end of file
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
After some discussion, we found out that "tools" has the four first
letters identical to the "toolchain" subfolder, which makes it a bit
unpractical with tab-completion. So, this commit renames "tools" to
"utils", which is more tab-completion-friendly.
This has been discussed with Arnout and Yann.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
