-
Notifications
You must be signed in to change notification settings - Fork 29
/
TODO
78 lines (55 loc) · 2.95 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
- report_links: previous builds can be in wrong order, should be sorted by
build, see https://openqa.suse.de/tests/588121#previous
- optional inclusion of "incompletes" to review them, too
- parse soft fails from test details pages as they show a nice icon for
instances of record_soft_failure
- think about updating a wiki page on redmine with a report
http://www.redmine.org/projects/redmine/wiki/Rest_WikiPages
- Add mode to count bug occurences (e.g. parse previous)
- try pylibrary cookiecutter template or "bumpversion"? see cookiecutter-pylibrary template
- offer to automatically post comment on webpage
- for new issues,
- offer hints what to do, e.g 'fix the needle', etc.
- parse and show link (clickable) for browser OR auto-open in browser
- show detailed fail reason (failed needles are always shown, easy)
- for fixed issues,
- also offer link to crosscheck
- offer to close bug
- compare architectures against each other, if same test fails for all
architectures, report as "common issues"
- consider "build number drift", e.g. some builds have
i586 and x86_64 and some the corresponding 'arm' tests
- for further investigation of failed issues
- compare list of needle candidates for failed needles
- empty list
- less needles
- more needles
- changed names
- search for open bugs in bugzilla
- how to find bugs which only have a test_url which does not specify the
test module? e.g. if someone only mentions
https://openqa.opensuse.org/tests/167348/file/autoinst-log.txt
- diff os-autoinst.txt logfiles, e.g. download and offer wizard like in
desired user diff editor
- can skip differing timestamps and near matches, maybe python difflib can
do this?
- offer links to
- autoinst.txt (should be easy, it is the test-url + '/file/autoinst-log.txt', e.g. https://openqa.opensuse.org/tests/167348/file/autoinst-log.txt')
- the test source code
- the git changelog of test + openQA + os-autoinst
- the OBS packages changed during period
- redirect report to file
- template for failed openqa product bugs in wiki
- improve detection about "new product issue" or "new openqa issue".
Assumption: If the source code of the test did not change
(previous->current) AND the needle candidates (or tags) did not change (AND
maybe the test one step in before also did not change and not fail) THEN a
failed test is a new product issue
- compare log output, e.g. see
- http://stackoverflow.com/questions/6486450/python-compute-list-difference
- http://stackoverflow.com/questions/3462143/get-difference-between-two-lists
- https://pypi.python.org/pypi/fuzzywuzzy
- http://streamhacker.com/2011/10/31/fuzzy-string-matching-python/
- http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/
- http://marcobonzanini.com/2015/02/25/fuzzy-string-matching-in-python/
- http://stackoverflow.com/questions/2923420/what-is-a-simple-fuzzy-string-matching-algorithm-in-python