Evaluate multiple output component model #906

chidauri · 2019-08-22T17:27:16Z

We are calculating metrics for each output(i.e., product, component and conflated_component) separately.

codecov-io · 2019-08-23T06:46:16Z

Codecov Report

Merging #906 into master will decrease coverage by 0.17%.
The diff coverage is 33.33%.

@@            Coverage Diff             @@
##           master     #906      +/-   ##
==========================================
- Coverage   53.03%   52.85%   -0.18%     
==========================================
  Files          81       81              
  Lines        5461     5487      +26     
==========================================
+ Hits         2896     2900       +4     
- Misses       2565     2587      +22

Impacted Files	Coverage Δ
bugbug/models/component.py	`18.48% <20%> (+1.24%)`	⬆️
bugbug/model.py	`63.86% <37.14%> (-4.26%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2810c2...9c2cee0. Read the comment docs.

chidauri · 2019-08-23T16:41:32Z

No confidence threshold - 12463 classified for product model

                                           pre       rec       spe        f1       geo       iba       sup

                               Core       0.71      0.97      0.46      0.82      0.66      0.46      7257
                           DevTools       0.87      0.57      0.99      0.69      0.75      0.54      1074
                 DevTools Graveyard       0.83      0.26      1.00      0.40      0.51      0.24        19
External Software Affecting Firefox       0.62      0.22      1.00      0.32      0.47      0.20        23
                            Firefox       0.80      0.38      0.98      0.52      0.61      0.35      1852
               Firefox Build System       0.85      0.37      1.00      0.51      0.60      0.34       620
                Firefox for Android       0.81      0.36      1.00      0.50      0.60      0.34       267
                            Toolkit       0.82      0.25      1.00      0.38      0.50      0.23       884
                      WebExtensions       0.81      0.57      0.99      0.67      0.76      0.55       467

                        avg / total       0.76      0.74      0.68      0.70      0.65      0.43     12463

╒══════════════════════════════════════════════╤════════════════════╤════════════════════════╤══════════════════════════════════╤═══════════════════════════════════════════════════╤═══════════════════════╤════════════════════════════════════╤═══════════════════════════════════╤═══════════════════════╤═════════════════════════════╕
│                                              │   Core (Predicted) │   DevTools (Predicted) │   DevTools Graveyard (Predicted) │   External Software Affecting Firefox (Predicted) │   Firefox (Predicted) │   Firefox Build System (Predicted) │   Firefox for Android (Predicted) │   Toolkit (Predicted) │   WebExtensions (Predicted) │
╞══════════════════════════════════════════════╪════════════════════╪════════════════════════╪══════════════════════════════════╪═══════════════════════════════════════════════════╪═══════════════════════╪════════════════════════════════════╪═══════════════════════════════════╪═══════════════════════╪═════════════════════════════╡
│ Core (Actual)                                │               7042 │                     63 │                                0 │                                                 3 │                    72 │                                 33 │                                 9 │                    18 │                          17 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ DevTools (Actual)                            │                436 │                    609 │                                1 │                                                 0 │                    21 │                                  1 │                                 0 │                     1 │                           5 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ DevTools Graveyard (Actual)                  │                 12 │                      2 │                                5 │                                                 0 │                     0 │                                  0 │                                 0 │                     0 │                           0 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ External Software Affecting Firefox (Actual) │                 18 │                      0 │                                0 │                                                 5 │                     0 │                                  0 │                                 0 │                     0 │                           0 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox (Actual)                             │               1094 │                      8 │                                0 │                                                 0 │                   707 │                                  2 │                                 7 │                    18 │                          16 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox Build System (Actual)                │                380 │                      5 │                                0 │                                                 0 │                     1 │                                227 │                                 4 │                     0 │                           3 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox for Android (Actual)                 │                145 │                      0 │                                0 │                                                 0 │                    19 │                                  3 │                                96 │                     3 │                           1 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Toolkit (Actual)                             │                600 │                      3 │                                0 │                                                 0 │                    39 │                                  1 │                                 0 │                   222 │                          19 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ WebExtensions (Actual)                       │                150 │                     13 │                                0 │                                                 0 │                    24 │                                  0 │                                 3 │                     9 │                         268 │
╘══════════════════════════════════════════════╧════════════════════╧════════════════════════╧══════════════════════════════════╧═══════════════════════════════════════════════════╧═══════════════════════╧════════════════════════════════════╧═══════════════════════════════════╧═══════════════════════╧═════════════════════════════╛

No confidence threshold - 12463 classified for component model

                                                      pre       rec       spe        f1       geo       iba       sup

                           Accessibility Tools       0.61      0.65      1.00      0.63      0.80      0.62        17
                               Activity Stream       0.62      0.29      1.00      0.40      0.54      0.27        17
                                Add-on Manager       0.50      0.10      1.00      0.17      0.32      0.09        10
                               Add-ons Manager       0.60      0.62      1.00      0.61      0.79      0.59       126
                                   Address Bar       0.60      0.62      0.99      0.61      0.79      0.60       157
                                       Android       0.50      0.23      1.00      0.32      0.48      0.21        13
         Android Studio and Gradle Integration       0.57      0.34      1.00      0.43      0.58      0.32        47
                                  Android Sync       0.30      0.15      1.00      0.20      0.39      0.14        20
                             Application Panel       0.86      0.86      1.00      0.86      0.93      0.84         7
                            Application Update       0.45      0.53      1.00      0.48      0.72      0.50        57
                                   Audio/Video       0.46      0.25      1.00      0.32      0.50      0.23        64
                              Audio/Video: GMP       0.30      0.25      1.00      0.27      0.50      0.23        12
                 Audio/Video: MediaStreamGraph       0.50      0.11      1.00      0.18      0.33      0.10        18
                         Audio/Video: Playback       0.61      0.64      0.99      0.63      0.80      0.61       356
                        Audio/Video: Recording       0.82      0.64      1.00      0.72      0.80      0.62        14
                            Audio/Video: cubeb       0.47      0.30      1.00      0.36      0.54      0.28        27
                                 Awesomescreen       0.23      0.18      1.00      0.20      0.42      0.16        17
                     Blocklist Policy Requests       0.94      0.89      1.00      0.91      0.94      0.88        54
                           Bookmarks & History       0.51      0.67      1.00      0.58      0.82      0.65        83
                       Bootstrap Configuration       0.50      0.57      1.00      0.53      0.76      0.55        14
                   CSS Parsing and Computation       0.71      0.71      0.99      0.71      0.84      0.68       576
                CSS Transitions and Animations       0.67      0.22      1.00      0.33      0.47      0.20         9
                                CSS and Themes       0.00      0.00      1.00      0.00      0.00      0.00         2
                               Canvas Debugger       1.00      0.50      1.00      0.67      0.71      0.48         4
                                    Canvas: 2D       0.80      0.60      1.00      0.69      0.77      0.58        40
                                 Canvas: WebGL       0.90      0.91      1.00      0.90      0.95      0.90       170
                                 Compatibility       0.00      0.00      1.00      0.00      0.00      0.00         8
                                       Console       0.69      0.79      1.00      0.74      0.88      0.77       168
                               Crash Reporting       0.42      0.55      1.00      0.48      0.74      0.52        31
                                           DOM       0.00      0.00      1.00      0.00      0.00      0.00         1
                                DOM: Animation       0.59      0.42      1.00      0.49      0.65      0.40        52
                        DOM: Bindings (WebIDL)       0.00      0.00      1.00      0.00      0.00      0.00         5
                         DOM: CSS Object Model       0.78      0.45      1.00      0.57      0.67      0.43        40
                        DOM: Content Processes       0.20      0.13      1.00      0.16      0.36      0.12        39
                              DOM: Core & HTML       0.30      0.62      0.92      0.40      0.76      0.56       656
                        DOM: Device Interfaces       0.81      0.49      1.00      0.61      0.70      0.46        45
                                   DOM: Events       0.62      0.47      1.00      0.53      0.69      0.44        85
                                     DOM: File       0.00      0.00      1.00      0.00      0.00      0.00        11
                              DOM: Geolocation       1.00      0.75      1.00      0.86      0.87      0.73         4
                                DOM: IndexedDB       0.73      0.70      1.00      0.71      0.84      0.68        50
                               DOM: Networking       0.50      0.20      1.00      0.29      0.45      0.18        10
                       DOM: Push Notifications       0.67      0.50      1.00      0.57      0.71      0.47         8
                            DOM: Quota Manager       0.50      0.27      1.00      0.35      0.52      0.25        11
                                 DOM: Security       0.78      0.55      1.00      0.65      0.74      0.52       220
                          DOM: Service Workers       0.84      0.59      1.00      0.69      0.76      0.56        82
                       DOM: Web Authentication       0.33      0.20      1.00      0.25      0.45      0.18         5
                               DOM: Web Crypto       1.00      0.33      1.00      0.50      0.58      0.31         3
                             DOM: Web Payments       0.54      0.28      1.00      0.37      0.53      0.26        25
                              DOM: Web Storage       0.40      0.36      1.00      0.38      0.60      0.34        11
                                  DOM: Workers       0.62      0.56      1.00      0.59      0.75      0.54        41
                                      Debugger       0.71      0.77      1.00      0.74      0.88      0.75       152
             Developer Environment Integration       0.00      0.00      1.00      0.00      0.00      0.00         1
                            Developer Outreach       0.00      0.00      1.00      0.00      0.00      0.00         6
                               Developer Tools       0.67      0.36      1.00      0.47      0.60      0.34        11
                        Disability Access APIs       0.82      0.77      1.00      0.79      0.88      0.75       118
                           Document Navigation       0.48      0.24      1.00      0.32      0.49      0.22        42
                                 Documentation       0.00      0.00      1.00      0.00      0.00      0.00         1
                                 Downloads API       0.29      0.33      1.00      0.31      0.58      0.31        12
                               Downloads Panel       0.39      0.68      1.00      0.50      0.83      0.66        19
                                 Drag and Drop       0.55      0.38      1.00      0.44      0.61      0.35        16
                                        Editor       0.67      0.53      1.00      0.59      0.72      0.50        78
                           Enterprise Policies       0.72      0.75      1.00      0.74      0.87      0.73        28
                                   Experiments       1.00      0.33      1.00      0.50      0.58      0.31         3
                       Extension Compatibility       0.40      0.10      1.00      0.16      0.32      0.09        20
                                 File Handling       0.36      0.21      1.00      0.26      0.46      0.19        24
                                  Find Toolbar       0.67      0.43      1.00      0.53      0.66      0.41        23
                              Firefox Accounts       0.38      0.29      1.00      0.32      0.53      0.27        21
                                 Flash (Adobe)       0.00      0.00      1.00      0.00      0.00      0.00         2
                                 Form Autofill       0.55      0.72      1.00      0.63      0.85      0.70        29
                                  Form Manager       0.69      0.46      1.00      0.55      0.68      0.44        39
                                     Framework       0.50      0.35      1.00      0.41      0.59      0.33        43
                                      Frontend       0.45      0.23      1.00      0.31      0.48      0.21        78
                                Gecko Profiler       0.56      0.54      1.00      0.55      0.74      0.52        70
                                       General       0.32      0.57      0.93      0.41      0.73      0.51       689
                General: Unsupported Platforms       0.43      0.33      1.00      0.38      0.58      0.31         9
                       Generated Documentation       0.00      0.00      1.00      0.00      0.00      0.00         1
               Graphic Commandline and Toolbar       0.67      0.25      1.00      0.36      0.50      0.23         8
                                      Graphics       0.54      0.37      0.99      0.44      0.61      0.35       245
                              Graphics: Layers       0.38      0.29      1.00      0.33      0.53      0.26        77
                                Graphics: Text       0.38      0.28      1.00      0.32      0.52      0.26        58
                           Graphics: WebRender       0.81      0.71      0.99      0.76      0.84      0.68       425
                                  HTML: Parser       0.29      0.14      1.00      0.19      0.38      0.13        14
                                      Headless       0.92      0.69      1.00      0.79      0.83      0.67        16
                                           IPC       0.43      0.40      1.00      0.41      0.63      0.38        57
                                    IPC: MSCOM       0.25      0.20      1.00      0.22      0.45      0.18         5
                                      ImageLib       0.51      0.52      1.00      0.52      0.72      0.50        44
                                     Inspector       0.69      0.70      1.00      0.70      0.83      0.68       163
                         Inspector: Animations       0.69      0.73      1.00      0.71      0.86      0.71        15
                            Inspector: Changes       0.29      0.50      1.00      0.36      0.71      0.47         4
                           Inspector: Computed       0.00      0.00      1.00      0.00      0.00      0.00         2
                              Inspector: Fonts       0.73      0.62      1.00      0.67      0.78      0.59        13
                             Inspector: Layout       0.00      0.00      1.00      0.00      0.00      0.00         1
                              Inspector: Rules       0.71      0.45      1.00      0.56      0.67      0.43        44
                                     Installer       0.67      0.64      1.00      0.65      0.80      0.62        28
                          Internationalization       0.76      0.61      1.00      0.67      0.78      0.58        56
                                   JSON Viewer       0.61      0.78      1.00      0.68      0.88      0.76        18
                             JavaScript Engine       0.51      0.60      0.98      0.55      0.77      0.56       392
                        JavaScript Engine: JIT       0.75      0.48      1.00      0.59      0.70      0.46       159
                                JavaScript: GC       0.65      0.48      1.00      0.55      0.69      0.45       119
          JavaScript: Internationalization API       0.80      0.35      1.00      0.48      0.59      0.33        23
                  JavaScript: Standard Library       0.00      0.00      1.00      0.00      0.00      0.00        10
                       Javascript: WebAssembly       0.46      0.43      1.00      0.45      0.66      0.41        44
                           Keyboard Navigation       0.00      0.00      1.00      0.00      0.00      0.00         7
                             Keyboards and IME       0.57      0.50      1.00      0.53      0.71      0.47         8
                                        Layout       0.50      0.56      0.98      0.53      0.74      0.52       405
                      Layout: Block and Inline       0.58      0.32      1.00      0.42      0.57      0.30        34
                               Layout: Columns       0.50      0.57      1.00      0.53      0.76      0.55         7
                               Layout: Flexbox       0.00      0.00      1.00      0.00      0.00      0.00         7
                                Layout: Floats       0.50      0.62      1.00      0.56      0.79      0.60         8
                         Layout: Form Controls       0.61      0.55      1.00      0.58      0.74      0.52       102
Layout: Generated Content, Lists, and Counters       0.00      0.00      1.00      0.00      0.00      0.00         0
                                  Layout: Grid       0.00      0.00      1.00      0.00      0.00      0.00         4
        Layout: Images, Video, and HTML Frames       0.14      0.07      1.00      0.10      0.27      0.06        14
                            Layout: Positioned       0.62      0.71      1.00      0.67      0.85      0.69         7
                                  Layout: Ruby       0.00      0.00      1.00      0.00      0.00      0.00         2
                Layout: Scrolling and Overflow       0.42      0.28      1.00      0.33      0.53      0.26        18
                                Layout: Tables       0.38      0.36      1.00      0.37      0.60      0.34        22
                        Layout: Text and Fonts       0.31      0.25      1.00      0.28      0.50      0.23        76
                           Lint and Formatting       0.59      0.52      1.00      0.55      0.72      0.50        46
                                          MFBT       0.33      0.04      1.00      0.07      0.19      0.03        27
                                     Mach Core       0.38      0.27      1.00      0.32      0.52      0.25        11
                                        MathML       1.00      0.76      1.00      0.87      0.87      0.75        17
                                Measuring Tool       0.00      0.00      1.00      0.00      0.00      0.00         2
                                        Memory       0.60      0.43      1.00      0.50      0.65      0.40         7
                              Memory Allocator       0.83      0.26      1.00      0.40      0.51      0.24        19
                                         Menus       0.34      0.40      1.00      0.37      0.63      0.37        55
                              Messaging System       0.89      0.48      1.00      0.62      0.69      0.45        50
                                       Metrics       0.50      0.57      1.00      0.53      0.76      0.55         7
                                     Migration       0.71      0.50      1.00      0.59      0.71      0.47        24
                                    Netmonitor       0.76      0.74      1.00      0.75      0.86      0.72       101
                                    Networking       0.47      0.30      1.00      0.37      0.54      0.28       128
                             Networking: Cache       0.58      0.35      1.00      0.44      0.60      0.33        31
                           Networking: Cookies       0.40      0.56      1.00      0.47      0.74      0.53        18
                               Networking: DNS       0.63      0.67      1.00      0.65      0.82      0.64        18
                      Networking: Domain Lists       0.00      0.00      1.00      0.00      0.00      0.00         0
                               Networking: FTP       1.00      0.50      1.00      0.67      0.71      0.48         2
                              Networking: File       0.00      0.00      1.00      0.00      0.00      0.00         3
                              Networking: HTTP       0.52      0.32      1.00      0.40      0.56      0.30        72
                               Networking: JAR       0.00      0.00      1.00      0.00      0.00      0.00         3
                        Networking: WebSockets       0.60      0.55      1.00      0.57      0.74      0.52        11
                                  New Tab Page       0.70      0.59      0.99      0.64      0.77      0.57       246
                               Normandy Client       0.83      0.92      1.00      0.87      0.96      0.92        26
                      Notifications and Alerts       0.10      0.07      1.00      0.08      0.27      0.06        14
                              Object Inspector       0.00      0.00      1.00      0.00      0.00      0.00         1
                                      OpenH264       0.00      0.00      1.00      0.00      0.00      0.00         0
                                         Other       0.80      0.42      1.00      0.55      0.65      0.40        19
                                    PDF Viewer       0.74      0.78      1.00      0.76      0.88      0.76        40
                              Page Info Window       0.62      0.62      1.00      0.62      0.79      0.60         8
                           Panning and Zooming       0.49      0.52      1.00      0.51      0.72      0.49        73
                              Password Manager       0.65      0.75      1.00      0.70      0.86      0.73        63
          Password Manager: Site Compatibility       0.00      0.00      1.00      0.00      0.00      0.00         3
                                   Performance       0.67      0.29      1.00      0.40      0.53      0.27         7
                        Performance Monitoring       0.91      0.50      1.00      0.65      0.71      0.47        20
         Performance Tools (Profiler/Timeline)       0.78      0.58      1.00      0.67      0.76      0.56        24
                                        Places       0.58      0.47      1.00      0.52      0.68      0.44        62
                                      Plug-ins       0.52      0.67      1.00      0.59      0.82      0.64        48
                                        Pocket       0.73      0.89      1.00      0.80      0.94      0.88        18
                                   Preferences       0.58      0.68      0.99      0.63      0.82      0.65       150
                          Preferences: Backend       0.78      0.37      1.00      0.50      0.61      0.35        19
                                 Print Preview       0.43      0.50      1.00      0.46      0.71      0.47         6
                              Printing: Output       0.33      0.41      1.00      0.37      0.64      0.39        17
                               Printing: Setup       0.33      0.25      1.00      0.29      0.50      0.23         4
                        Privacy: Anti-Tracking       0.33      0.23      1.00      0.27      0.48      0.21        13
                              Private Browsing       0.45      0.70      1.00      0.55      0.84      0.68        20
                                   Reader Mode       0.92      0.85      1.00      0.88      0.92      0.83        26
                        Remote Settings Client       0.60      0.19      1.00      0.29      0.43      0.17        16
                              Request Handling       0.75      0.47      1.00      0.58      0.69      0.45        38
                        Responsive Design Mode       0.76      0.79      1.00      0.77      0.89      0.78        39
                                           SVG       0.57      0.75      1.00      0.64      0.86      0.73       103
                                 Safe Browsing       0.78      0.63      1.00      0.70      0.79      0.61        46
                                    Scratchpad       0.00      0.00      1.00      0.00      0.00      0.00         1
                                   Screenshots       0.59      0.67      1.00      0.62      0.82      0.64        15
                                        Search       0.45      0.56      1.00      0.50      0.74      0.53        54
                                      Security       0.55      0.32      1.00      0.41      0.57      0.30        74
                                 Security: PSM       0.72      0.57      1.00      0.64      0.76      0.55       108
                  Security: Process Sandboxing       0.76      0.55      1.00      0.64      0.74      0.52        62
                                     Selection       0.26      0.31      1.00      0.29      0.56      0.29        16
                               Session Restore       0.61      0.60      1.00      0.60      0.77      0.57        57
                      Settings and Preferences       0.67      0.25      1.00      0.36      0.50      0.23         8
                             Shared Components       1.00      0.15      1.00      0.26      0.39      0.14        20
                             Shell Integration       0.29      0.29      1.00      0.29      0.53      0.27         7
           Site Identity and Permission Panels       0.71      0.50      1.00      0.59      0.71      0.47        44
                          Source Code Analysis       0.44      0.42      1.00      0.43      0.65      0.39        36
                                 Source Editor       1.00      0.67      1.00      0.80      0.82      0.64         6
                              Spelling checker       0.80      0.71      1.00      0.75      0.84      0.68        17
                    Startup and Profile System       0.35      0.32      1.00      0.33      0.57      0.30        25
                                       Storage       0.28      0.29      1.00      0.29      0.54      0.27        17
                             Storage Inspector       0.80      0.57      1.00      0.67      0.76      0.55        14
                                        String       0.60      0.25      1.00      0.35      0.50      0.23        12
                                  Style Editor       0.67      0.67      1.00      0.67      0.82      0.64         6
                                          Sync       0.59      0.70      1.00      0.64      0.84      0.68       108
                                Tabbed Browser       0.41      0.54      0.99      0.47      0.73      0.51       133
                            Task Configuration       0.55      0.45      1.00      0.50      0.67      0.43        84
                                     Telemetry       0.61      0.68      0.99      0.64      0.82      0.66       152
                                       Testing       0.56      0.63      1.00      0.59      0.79      0.60        70
                                         Theme       0.46      0.45      0.99      0.45      0.67      0.42       137
                       Theme and Visual Design       0.36      0.24      1.00      0.29      0.49      0.22        21
                                        Themes       0.52      0.31      1.00      0.39      0.56      0.29        45
                                       Toolbar       0.33      0.20      1.00      0.25      0.45      0.18        10
                    Toolbars and Customization       0.69      0.51      1.00      0.58      0.71      0.48        99
                                    Toolchains       0.44      0.21      1.00      0.29      0.46      0.19        19
                                         Tours       0.67      0.75      1.00      0.71      0.87      0.73         8
                           Tracking Protection       0.67      0.55      1.00      0.61      0.74      0.52        60
                                           Try       0.50      0.33      1.00      0.40      0.58      0.31         9
                                     Untriaged       0.39      0.18      1.00      0.25      0.42      0.16        78
                User events and focus handling       0.40      0.14      1.00      0.21      0.38      0.13        56
                          Video/Audio Controls       1.00      0.42      1.00      0.59      0.65      0.40        19
                                      Web Apps       0.93      0.67      1.00      0.78      0.82      0.64        21
                                     Web Audio       0.67      0.44      1.00      0.53      0.66      0.41        32
                              Web Audio Editor       1.00      0.75      1.00      0.86      0.87      0.73         4
                                  Web Painting       0.60      0.37      1.00      0.46      0.61      0.35        84
                                    Web Replay       0.94      0.76      1.00      0.84      0.87      0.74        38
                           WebGL Shader Editor       0.00      0.00      1.00      0.00      0.00      0.00         3
                                        WebIDE       0.80      0.67      1.00      0.73      0.82      0.64         6
                                WebPayments UI       0.81      0.82      1.00      0.82      0.91      0.81        51
                                        WebRTC       0.69      0.65      1.00      0.67      0.80      0.62       161
                           WebRTC: Audio/Video       0.53      0.31      1.00      0.39      0.56      0.29        86
                            WebRTC: Networking       0.42      0.32      1.00      0.36      0.57      0.30        25
                             WebRTC: Signaling       0.43      0.18      1.00      0.26      0.43      0.17        33
                                         WebVR       0.90      0.73      1.00      0.81      0.85      0.71        52
                                        Widget       0.00      0.00      1.00      0.00      0.00      0.00        16
                                 Widget: Cocoa       0.44      0.38      1.00      0.41      0.62      0.36        58
                                   Widget: Gtk       0.42      0.54      0.99      0.47      0.73      0.51        84
                                 Widget: Win32       0.31      0.31      1.00      0.31      0.56      0.29        35
                                           XML       0.25      0.20      1.00      0.22      0.45      0.18         5
                                         XPCOM       0.42      0.32      1.00      0.36      0.57      0.30       127
                                     XPConnect       0.71      0.40      1.00      0.52      0.64      0.38        42
                                           XUL       0.30      0.22      1.00      0.26      0.47      0.20        27
                                   XUL Widgets       0.62      0.41      1.00      0.49      0.64      0.38        32
                               about:debugging       0.86      0.76      1.00      0.81      0.87      0.74        63
                                       mozglue       0.71      0.42      1.00      0.53      0.65      0.39        12

                                   avg / total       0.58      0.55      0.99      0.54      0.72      0.52     12463

@marco-c I have a trained model saved in case you need to see full results.

marco-c · 2019-09-02T22:08:54Z

These lines should not be there, as these components should be ignored:

                                 Untriaged       0.39      0.18      1.00      0.25      0.42      0.16        78

                                  General       0.32      0.57      0.93      0.41      0.73      0.51       689

marco-c · 2019-09-02T22:10:12Z

The results seem worse than the current model (can you post the results of the current model?), so perhaps this is not a good idea. The classifier chain might work better in this case.

chidauri · 2019-09-07T09:50:43Z

@marco-c we cannot use classifier chain here , as it is used to combine binary classifiers which is not our case i think as written here https://scikit-learn.org/stable/modules/multiclass.html#classifier-chain

chidauri · 2019-09-07T09:55:40Z

@marco-c The results of the current model for no threshold case..

/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
  'precision', 'predicted', average, warn_for)
                                                                   pre       rec       spe        f1       geo       iba       sup

                                          Core::Audio/Video       0.00      0.00      1.00      0.00      0.00      0.00        22
                                     Core::Audio/Video: GMP       1.00      0.19      1.00      0.32      0.43      0.17        16
                        Core::Audio/Video: MediaStreamGraph       0.43      0.20      1.00      0.27      0.45      0.18        15
                                Core::Audio/Video: Playback       0.61      0.69      0.99      0.65      0.83      0.66       339
                               Core::Audio/Video: Recording       0.70      0.39      1.00      0.50      0.62      0.37        18
                                   Core::Audio/Video: cubeb       0.71      0.52      1.00      0.60      0.72      0.50        42
                          Core::CSS Parsing and Computation       0.72      0.69      0.99      0.70      0.83      0.66       557
                       Core::CSS Transitions and Animations       1.00      0.25      1.00      0.40      0.50      0.23         4
                                           Core::Canvas: 2D       0.61      0.67      1.00      0.64      0.82      0.64        33
                                        Core::Canvas: WebGL       0.86      0.81      1.00      0.84      0.90      0.79       180
                                                  Core::DOM       0.25      0.70      0.87      0.37      0.78      0.60       711
                                       Core::DOM: Animation       0.59      0.52      1.00      0.55      0.72      0.49        52
                               Core::DOM: Bindings (WebIDL)       0.00      0.00      1.00      0.00      0.00      0.00        12
                                Core::DOM: CSS Object Model       0.88      0.35      1.00      0.50      0.59      0.33        40
                               Core::DOM: Content Processes       0.31      0.17      1.00      0.22      0.42      0.16        52
                               Core::DOM: Device Interfaces       0.63      0.49      1.00      0.55      0.70      0.46        39
                                          Core::DOM: Events       0.57      0.57      1.00      0.57      0.75      0.54        81
                                            Core::DOM: File       0.00      0.00      1.00      0.00      0.00      0.00        11
                                       Core::DOM: IndexedDB       0.82      0.74      1.00      0.78      0.86      0.72        43
                                      Core::DOM: Networking       0.60      0.25      1.00      0.35      0.50      0.23        12
                              Core::DOM: Push Notifications       0.50      0.33      1.00      0.40      0.58      0.31         6
                                   Core::DOM: Quota Manager       0.67      0.75      1.00      0.71      0.87      0.73         8
                                        Core::DOM: Security       0.73      0.60      1.00      0.66      0.77      0.57       188
                                 Core::DOM: Service Workers       0.95      0.70      1.00      0.81      0.84      0.68        80
                              Core::DOM: Web Authentication       0.33      0.08      1.00      0.13      0.29      0.08        12
                                    Core::DOM: Web Payments       0.89      0.59      1.00      0.71      0.77      0.57        27
                                     Core::DOM: Web Storage       0.50      0.27      1.00      0.35      0.52      0.25        11
                                         Core::DOM: Workers       0.76      0.61      1.00      0.68      0.78      0.59        41
                               Core::Disability Access APIs       0.71      0.65      1.00      0.68      0.81      0.63       104
                                  Core::Document Navigation       0.42      0.23      1.00      0.30      0.48      0.21        43
                                        Core::Drag and Drop       0.17      0.22      1.00      0.19      0.47      0.20         9
                                               Core::Editor       0.75      0.63      1.00      0.68      0.79      0.61        84
                                       Core::Gecko Profiler       0.66      0.54      1.00      0.59      0.73      0.51        80
                                             Core::Graphics       0.44      0.34      0.99      0.38      0.58      0.32       210
                                     Core::Graphics: Layers       0.39      0.23      1.00      0.29      0.48      0.21        78
                                       Core::Graphics: Text       0.61      0.34      1.00      0.43      0.58      0.32        68
                                  Core::Graphics: WebRender       0.84      0.70      1.00      0.77      0.84      0.68       435
                                         Core::HTML: Parser       0.64      0.44      1.00      0.52      0.66      0.41        16
                                                  Core::IPC       0.41      0.42      1.00      0.41      0.65      0.39        67
                                             Core::ImageLib       0.44      0.47      1.00      0.45      0.68      0.44        43
                                 Core::Internationalization       0.62      0.42      1.00      0.50      0.65      0.40        57
                                           Core::JavaScript       0.56      0.59      0.98      0.58      0.76      0.56       396
                               Core::JavaScript Engine: JIT       0.63      0.53      1.00      0.58      0.73      0.51       144
                                       Core::JavaScript: GC       0.53      0.52      1.00      0.53      0.72      0.50        82
                 Core::JavaScript: Internationalization API       0.46      0.35      1.00      0.40      0.59      0.33        17
                         Core::JavaScript: Standard Library       0.67      0.15      1.00      0.25      0.39      0.14        13
                              Core::Javascript: WebAssembly       0.53      0.63      1.00      0.58      0.80      0.61        41
                                               Core::Layout       0.56      0.53      0.98      0.55      0.72      0.50       445
                             Core::Layout: Block and Inline       0.80      0.32      1.00      0.45      0.56      0.29        38
                                      Core::Layout: Columns       0.67      0.57      1.00      0.62      0.76      0.55         7
                                       Core::Layout: Floats       0.40      0.67      1.00      0.50      0.82      0.64         3
                                Core::Layout: Form Controls       0.58      0.61      1.00      0.59      0.78      0.58        76
               Core::Layout: Images, Video, and HTML Frames       0.27      0.16      1.00      0.20      0.40      0.14        19
                                   Core::Layout: Positioned       0.75      0.33      1.00      0.46      0.58      0.31        18
                       Core::Layout: Scrolling and Overflow       0.38      0.50      1.00      0.43      0.71      0.47         6
                                       Core::Layout: Tables       0.62      0.55      1.00      0.58      0.74      0.53        29
                               Core::Layout: Text and Fonts       0.39      0.46      1.00      0.42      0.68      0.43        85
                                                 Core::MFBT       0.67      0.31      1.00      0.42      0.55      0.29        26
                                               Core::MathML       0.88      0.75      1.00      0.81      0.87      0.73        20
                                     Core::Memory Allocator       0.33      0.17      1.00      0.22      0.41      0.15        18
                                           Core::Networking       0.59      0.36      1.00      0.45      0.60      0.34       141
                                    Core::Networking: Cache       0.54      0.37      1.00      0.44      0.61      0.35        19
                                  Core::Networking: Cookies       0.67      0.57      1.00      0.62      0.76      0.55        21
                                      Core::Networking: DNS       0.61      0.55      1.00      0.58      0.74      0.52        20
                                     Core::Networking: HTTP       0.51      0.27      1.00      0.35      0.52      0.25        81
                               Core::Networking: WebSockets       0.75      0.43      1.00      0.55      0.65      0.40         7
                                  Core::Panning and Zooming       0.58      0.56      1.00      0.57      0.75      0.54        94
                                          Core::Performance       0.33      0.17      1.00      0.22      0.41      0.15         6
                                             Core::Plug-ins       0.72      0.57      1.00      0.63      0.75      0.54        58
                                 Core::Preferences: Backend       0.64      0.38      1.00      0.47      0.61      0.35        24
                                                Core::Print       0.48      0.46      1.00      0.47      0.68      0.44        28
                                        Core::Print Preview       0.67      0.43      1.00      0.52      0.65      0.40        14
                                      Core::Printing: Setup       0.67      0.57      1.00      0.62      0.76      0.55         7
                               Core::Privacy: Anti-Tracking       0.00      0.00      1.00      0.00      0.00      0.00         8
                                                  Core::SVG       0.62      0.71      1.00      0.66      0.84      0.69       107
                                             Core::Security       0.56      0.17      1.00      0.26      0.41      0.15        30
                                        Core::Security: PSM       0.61      0.64      1.00      0.62      0.80      0.62        78
                         Core::Security: Process Sandboxing       0.61      0.58      1.00      0.59      0.76      0.55        59
                                            Core::Selection       0.28      0.29      1.00      0.29      0.54      0.27        17
                                     Core::Spelling checker       0.93      0.54      1.00      0.68      0.74      0.52        24
                                               Core::String       0.25      0.06      1.00      0.10      0.24      0.05        17
                       Core::User events and focus handling       0.21      0.15      1.00      0.17      0.38      0.13        34
                                            Core::Web Audio       0.88      0.61      1.00      0.72      0.78      0.59        49
                                         Core::Web Painting       0.48      0.26      1.00      0.34      0.51      0.24        76
                                           Core::Web Replay       0.77      0.62      1.00      0.69      0.79      0.60        32
                                               Core::WebRTC       0.67      0.68      1.00      0.67      0.82      0.65       143
                                  Core::WebRTC: Audio/Video       0.56      0.27      1.00      0.36      0.52      0.25        82
                                   Core::WebRTC: Networking       0.60      0.17      1.00      0.26      0.41      0.15        36
                                    Core::WebRTC: Signaling       0.52      0.28      1.00      0.36      0.52      0.25        40
                                                Core::WebVR       0.88      0.73      1.00      0.80      0.86      0.71        49
                                               Core::Widget       0.25      0.06      1.00      0.10      0.25      0.06        16
                                        Core::Widget: Cocoa       0.38      0.31      1.00      0.34      0.56      0.29        54
                                          Core::Widget: Gtk       0.51      0.48      1.00      0.50      0.69      0.45       109
                                        Core::Widget: Win32       0.54      0.33      1.00      0.41      0.58      0.31        57
                                                Core::XPCOM       0.38      0.24      1.00      0.30      0.49      0.22       123
                                            Core::XPConnect       0.69      0.23      1.00      0.35      0.48      0.22        47
                                                  Core::XUL       0.17      0.08      1.00      0.11      0.28      0.07        26
                                              Core::mozglue       0.25      0.11      1.00      0.15      0.33      0.10         9
                                                   DevTools       0.59      0.52      0.99      0.55      0.72      0.49       193
                              DevTools::Accessibility Tools       0.44      0.40      1.00      0.42      0.63      0.38        20
                                DevTools::Application Panel       1.00      0.50      1.00      0.67      0.71      0.48        10
                                          DevTools::Console       0.67      0.74      0.99      0.70      0.86      0.72       176
                                         DevTools::Debugger       0.73      0.78      1.00      0.75      0.88      0.76       169
                                        DevTools::Framework       0.57      0.33      1.00      0.42      0.58      0.31        39
                                        DevTools::Inspector       0.70      0.61      1.00      0.66      0.78      0.59       161
                            DevTools::Inspector: Animations       0.83      0.79      1.00      0.81      0.89      0.77        24
                                 DevTools::Inspector: Fonts       0.89      0.89      1.00      0.89      0.94      0.88        18
                                 DevTools::Inspector: Rules       0.58      0.42      1.00      0.49      0.65      0.39        43
                                      DevTools::JSON Viewer       0.65      0.79      1.00      0.71      0.89      0.77        19
                                       DevTools::Netmonitor       0.72      0.74      1.00      0.73      0.86      0.72        94
            DevTools::Performance Tools (Profiler/Timeline)       0.50      0.36      1.00      0.42      0.60      0.33        14
                           DevTools::Responsive Design Mode       0.88      0.76      1.00      0.82      0.87      0.74        38
                                DevTools::Shared Components       0.40      0.13      1.00      0.20      0.37      0.12        15
                                DevTools::Storage Inspector       1.00      0.71      1.00      0.83      0.84      0.69        17
                                     DevTools::Style Editor       0.80      0.67      1.00      0.73      0.82      0.64         6
                                  DevTools::about:debugging       0.81      0.81      1.00      0.81      0.90      0.80        64
                        External Software Affecting Firefox       0.62      0.59      1.00      0.60      0.77      0.57        27
                                       Firefox Build System       0.58      0.64      0.99      0.61      0.80      0.61       348
Firefox Build System::Android Studio and Gradle Integration       0.60      0.60      1.00      0.60      0.77      0.58        30
              Firefox Build System::Bootstrap Configuration       0.52      0.58      1.00      0.55      0.76      0.55        19
       Firefox Build System::General: Unsupported Platforms       0.62      0.28      1.00      0.38      0.53      0.26        18
                  Firefox Build System::Lint and Formatting       0.73      0.68      1.00      0.70      0.82      0.66        47
                            Firefox Build System::Mach Core       0.33      0.08      1.00      0.13      0.29      0.08        12
                 Firefox Build System::Source Code Analysis       0.58      0.51      1.00      0.54      0.72      0.49        37
                   Firefox Build System::Task Configuration       0.67      0.58      1.00      0.62      0.76      0.55        95
                           Firefox Build System::Toolchains       0.00      0.00      1.00      0.00      0.00      0.00        15
                                  Firefox Build System::Try       0.55      0.60      1.00      0.57      0.77      0.58        10
                       Firefox for Android::Activity Stream       0.71      0.31      1.00      0.43      0.56      0.29        16
                        Firefox for Android::Add-on Manager       0.67      0.25      1.00      0.36      0.50      0.23         8
                          Firefox for Android::Android Sync       0.71      0.38      1.00      0.50      0.62      0.36        13
                           Firefox for Android::Audio/Video       0.50      0.27      1.00      0.35      0.52      0.25        26
                         Firefox for Android::Awesomescreen       0.60      0.20      1.00      0.30      0.45      0.18        15
                      Firefox for Android::Firefox Accounts       0.50      0.17      1.00      0.25      0.41      0.15         6
                     Firefox for Android::Keyboards and IME       0.67      0.25      1.00      0.36      0.50      0.23         8
                               Firefox for Android::Metrics       0.50      0.40      1.00      0.44      0.63      0.38        10
              Firefox for Android::Settings and Preferences       0.00      0.00      1.00      0.00      0.00      0.00        12
                               Firefox for Android::Testing       0.58      0.71      1.00      0.64      0.84      0.69        90
               Firefox for Android::Theme and Visual Design       0.56      0.20      1.00      0.29      0.45      0.18        25
                               Firefox for Android::Toolbar       0.50      0.22      1.00      0.31      0.47      0.20         9
                              Firefox for Android::Web Apps       0.65      0.61      1.00      0.63      0.78      0.59        18
                                       Firefox::Address Bar       0.62      0.63      0.99      0.63      0.79      0.61       160
                               Firefox::Bookmarks & History       0.62      0.68      1.00      0.65      0.83      0.66       101
                                   Firefox::Downloads Panel       0.37      0.43      1.00      0.40      0.66      0.41        23
                               Firefox::Enterprise Policies       0.81      0.83      1.00      0.82      0.91      0.82        36
                           Firefox::Extension Compatibility       0.20      0.08      1.00      0.11      0.28      0.07        13
                                     Firefox::File Handling       0.48      0.30      1.00      0.37      0.55      0.28        33
                                  Firefox::Firefox Accounts       0.50      0.33      1.00      0.40      0.58      0.31         9
                                          Firefox::Headless       1.00      0.73      1.00      0.84      0.85      0.71        11
                                         Firefox::Installer       0.74      0.62      1.00      0.68      0.79      0.60        32
                               Firefox::Keyboard Navigation       0.00      0.00      1.00      0.00      0.00      0.00         9
                                             Firefox::Menus       0.27      0.32      1.00      0.29      0.56      0.30        44
                                  Firefox::Messaging System       0.71      0.49      1.00      0.58      0.70      0.46        41
                                         Firefox::Migration       0.50      0.60      1.00      0.55      0.77      0.58        10
                                      Firefox::New Tab Page       0.72      0.58      1.00      0.64      0.76      0.55       229
                                   Firefox::Normandy Client       0.75      0.80      1.00      0.77      0.89      0.78        15
                                        Firefox::PDF Viewer       0.76      0.90      1.00      0.82      0.95      0.89        49
                                  Firefox::Page Info Window       0.83      0.50      1.00      0.62      0.71      0.47        10
                                            Firefox::Pocket       0.57      0.80      1.00      0.67      0.89      0.78        15
                                       Firefox::Preferences       0.63      0.69      1.00      0.66      0.83      0.67       143
                                  Firefox::Private Browsing       0.52      0.67      1.00      0.59      0.82      0.64        18
                            Firefox::Remote Settings Client       0.80      0.67      1.00      0.73      0.82      0.64         6
                                       Firefox::Screenshots       0.50      0.75      1.00      0.60      0.87      0.73        16
                                            Firefox::Search       0.55      0.53      1.00      0.54      0.73      0.50        72
                                          Firefox::Security       0.62      0.24      1.00      0.34      0.49      0.22        42
                                   Firefox::Session Restore       0.62      0.63      1.00      0.63      0.80      0.61        52
                                 Firefox::Shell Integration       0.57      0.40      1.00      0.47      0.63      0.38        10
               Firefox::Site Identity and Permission Panels       0.66      0.43      1.00      0.52      0.66      0.41        44
                                              Firefox::Sync       0.58      0.74      1.00      0.65      0.86      0.72        92
                                    Firefox::Tabbed Browser       0.41      0.63      0.99      0.50      0.79      0.60       123
                                             Firefox::Theme       0.46      0.46      0.99      0.46      0.68      0.44       155
                        Firefox::Toolbars and Customization       0.46      0.48      1.00      0.47      0.69      0.45        86
                                             Firefox::Tours       0.56      0.62      1.00      0.59      0.79      0.60         8
                               Firefox::Tracking Protection       0.62      0.67      1.00      0.64      0.82      0.65        55
                                    Firefox::WebPayments UI       0.83      0.80      1.00      0.81      0.89      0.78        44
                                   Toolkit::Add-ons Manager       0.62      0.64      1.00      0.63      0.80      0.62       128
                                Toolkit::Application Update       0.58      0.66      1.00      0.62      0.81      0.63        70
                         Toolkit::Blocklist Policy Requests       0.91      0.95      1.00      0.93      0.98      0.95        43
                                   Toolkit::Crash Reporting       0.52      0.53      1.00      0.52      0.73      0.51        32
                                     Toolkit::Downloads API       0.40      0.32      1.00      0.35      0.56      0.29        19
                                      Toolkit::Find Toolbar       0.89      0.42      1.00      0.57      0.65      0.40        19
                                     Toolkit::Form Autofill       0.82      0.60      1.00      0.69      0.77      0.58        30
                                      Toolkit::Form Manager       0.62      0.64      1.00      0.63      0.80      0.62        28
                          Toolkit::Notifications and Alerts       0.40      0.36      1.00      0.38      0.60      0.34        11
                                  Toolkit::Password Manager       0.77      0.79      1.00      0.78      0.89      0.77        62
                            Toolkit::Performance Monitoring       0.88      0.50      1.00      0.64      0.71      0.47        14
                                            Toolkit::Places       0.48      0.36      1.00      0.41      0.60      0.33        56
                                       Toolkit::Preferences       0.75      0.46      1.00      0.57      0.68      0.44        13
                                       Toolkit::Reader Mode       0.75      0.75      1.00      0.75      0.87      0.73        20
                                     Toolkit::Safe Browsing       0.88      0.64      1.00      0.74      0.80      0.62        45
                        Toolkit::Startup and Profile System       0.56      0.41      1.00      0.47      0.64      0.38        37
                                           Toolkit::Storage       0.43      0.25      1.00      0.32      0.50      0.23        12
                                         Toolkit::Telemetry       0.54      0.68      0.99      0.61      0.82      0.66       135
                                            Toolkit::Themes       0.50      0.17      1.00      0.25      0.41      0.15        30
                              Toolkit::Video/Audio Controls       0.94      0.60      1.00      0.73      0.77      0.58        25
                                       Toolkit::XUL Widgets       0.34      0.32      1.00      0.33      0.57      0.30        31
                                              WebExtensions       0.54      0.61      0.99      0.57      0.78      0.58       299
                                     WebExtensions::Android       0.50      0.40      1.00      0.44      0.63      0.38        15
                               WebExtensions::Compatibility       0.00      0.00      1.00      0.00      0.00      0.00         8
                             WebExtensions::Developer Tools       1.00      0.18      1.00      0.31      0.43      0.17        11
                                    WebExtensions::Frontend       0.30      0.11      1.00      0.16      0.33      0.10        65
                            WebExtensions::Request Handling       0.78      0.44      1.00      0.56      0.66      0.41        41
                                     WebExtensions::Storage       0.50      0.11      1.00      0.18      0.33      0.10         9
                                      WebExtensions::Themes       0.67      0.50      1.00      0.57      0.71      0.47        12

                                                avg / total       0.59      0.56      0.99      0.56      0.73      0.53     12431

chidauri · 2019-09-07T10:11:20Z

These lines should not be there, as these components should be ignored:

                                 Untriaged       0.39      0.18      1.00      0.25      0.42      0.16        78

                                  General       0.32      0.57      0.93      0.41      0.73      0.51       689

I think these aren't ignored because these inverse mapping contains both General and Untriaged.

bugbug/bugbug/models/component.py

Lines 51 to 54 in 8b59321

    
           "DevTools": "DevTools::General", 
        
           "External Software Affecting Firefox": "External Software Affecting Firefox::Other", 
        
           "WebExtensions": "WebExtensions::Untriaged", 
        
           "Firefox Build System": "Firefox Build System::General",

marco-c

Will need to be rebased but I will keep this open in case we want to pick it up again.

chidauri added 3 commits August 22, 2019 11:43

Implement separate component models

7574b06

calculate accuracy on test set

efab2d5

remove training set restrictions

c1f9e32

chidauri requested a review from marco-c August 22, 2019 17:35

use separate metrics for each output

2f06ce9

minor fixes

9c2cee0

marco-c requested changes Apr 11, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Evaluate multiple output component model #906

Evaluate multiple output component model #906

chidauri commented Aug 22, 2019 •

edited

Loading

codecov-io commented Aug 23, 2019 •

edited

Loading

chidauri commented Aug 23, 2019 •

edited

Loading

marco-c commented Sep 2, 2019

marco-c commented Sep 2, 2019

chidauri commented Sep 7, 2019 •

edited

Loading

chidauri commented Sep 7, 2019 •

edited

Loading

chidauri commented Sep 7, 2019

marco-c left a comment

Evaluate multiple output component model #906

Are you sure you want to change the base?

Evaluate multiple output component model #906

Conversation

chidauri commented Aug 22, 2019 • edited Loading

codecov-io commented Aug 23, 2019 • edited Loading

Codecov Report

chidauri commented Aug 23, 2019 • edited Loading

marco-c commented Sep 2, 2019

marco-c commented Sep 2, 2019

chidauri commented Sep 7, 2019 • edited Loading

chidauri commented Sep 7, 2019 • edited Loading

chidauri commented Sep 7, 2019

marco-c left a comment

Choose a reason for hiding this comment

chidauri commented Aug 22, 2019 •

edited

Loading

codecov-io commented Aug 23, 2019 •

edited

Loading

chidauri commented Aug 23, 2019 •

edited

Loading

chidauri commented Sep 7, 2019 •

edited

Loading

chidauri commented Sep 7, 2019 •

edited

Loading