Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Summary counters are not accurate #1673

Open
dwertent opened this issue Jan 23, 2024 · 3 comments
Open

Summary counters are not accurate #1673

dwertent opened this issue Jan 23, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@dwertent
Copy link

What happened:
I sanned the following:

% grype quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6 --by-cve
 ✔ Scanned for vulnerabilities     [30 vulnerability matches]
   ├── by severity: 1 critical, 18 high, 11 medium, 0 low, 0 negligible

Here is the list of vulnerabilities:

stdlib             go1.21.0             go-module  CVE-2023-39320  Critical
golang.org/x/net   v0.13.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net   v0.13.0    0.17.0    go-module  CVE-2023-39325  High
golang.org/x/net   v0.14.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net   v0.14.0    0.17.0    go-module  CVE-2023-39325  High
k8s.io/kubernetes  v1.28.0    1.28.4    go-module  CVE-2023-5528   High
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  CVE-2023-3955   High
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  CVE-2023-3676   High
stdlib             go1.20.7             go-module  CVE-2023-45285  High
stdlib             go1.20.7             go-module  CVE-2023-45283  High
stdlib             go1.20.7             go-module  CVE-2023-44487  High
stdlib             go1.20.7             go-module  CVE-2023-39325  High
stdlib             go1.20.7             go-module  CVE-2023-39323  High
stdlib             go1.21.0             go-module  CVE-2023-45285  High
stdlib             go1.21.0             go-module  CVE-2023-45283  High
stdlib             go1.21.0             go-module  CVE-2023-44487  High
stdlib             go1.21.0             go-module  CVE-2023-39325  High
stdlib             go1.21.0             go-module  CVE-2023-39323  High
stdlib             go1.21.0             go-module  CVE-2023-39322  High
stdlib             go1.21.0             go-module  CVE-2023-39321  High
stdlib             go1.21.0             go-module  CVE-2023-45284  Medium
stdlib             go1.21.0             go-module  CVE-2023-39326  Medium
stdlib             go1.21.0             go-module  CVE-2023-39319  Medium
stdlib             go1.21.0             go-module  CVE-2023-39318  Medium
stdlib             go1.20.7             go-module  CVE-2023-45284  Medium
stdlib             go1.20.7             go-module  CVE-2023-39326  Medium
stdlib             go1.20.7             go-module  CVE-2023-39319  Medium
stdlib             go1.20.7             go-module  CVE-2023-39318  Medium

As you can tell, there are 19 High and 8 Medium, not 18 high, and 11 medium as it says in the summary.

What you expected to happen:
The numbers should add up.

How to reproduce it (as minimally and precisely as possible):
Run the following:

grype quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6 --by-cve

Anything else we need to know?:

Full output:

% grype quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6 --by-cve
 ✔ Vulnerability DB                [no update available]
 ✔ Loaded image                                                                                                                                                                                                                                                                        quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6
 ✔ Parsed image                                                                                                                                                                                                                                                                                                   sha256:b747f07011a2c82631683d826b2275a6fcfbe6dfb5c26ac48d69446767cd7511
 ✔ Cataloged packages              [176 packages]
 ✔ Scanned for vulnerabilities     [30 vulnerability matches]
   ├── by severity: 1 critical, 18 high, 11 medium, 0 low, 0 negligible
   └── by status:   9 fixed, 21 not-fixed, 0 ignored
[0003]  WARN some package(s) are missing CPEs. This may result in missing vulnerabilities. You may autogenerate these using: --add-cpes-if-none
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/api, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/api" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/apimachinery, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/apimachinery" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/cli-runtime, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/cli-runtime" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/client-go, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/client-go" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/component-base, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/component-base" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/component-helpers, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/component-helpers" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/kubectl, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/kubectl" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/metrics, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/metrics" ver="(devel)": Malformed version: (devel)
[0003]  WARN could not match by package language (package=Pkg(type=go-module, name=github.com/tigera/cc-operator, version=(devel), upstreams=0)): matcher failed to parse version pkg="github.com/tigera/cc-operator" ver="(devel)": Malformed version: (devel)
NAME               INSTALLED  FIXED-IN  TYPE       VULNERABILITY   SEVERITY
golang.org/x/net   v0.13.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net   v0.13.0    0.17.0    go-module  CVE-2023-39325  High
golang.org/x/net   v0.14.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net   v0.14.0    0.17.0    go-module  CVE-2023-39325  High
k8s.io/kubernetes  v1.28.0    1.28.4    go-module  CVE-2023-5528   High
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  CVE-2023-3955   High
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  CVE-2023-3676   High
stdlib             go1.20.7             go-module  CVE-2023-45285  High
stdlib             go1.20.7             go-module  CVE-2023-45283  High
stdlib             go1.20.7             go-module  CVE-2023-44487  High
stdlib             go1.20.7             go-module  CVE-2023-39325  High
stdlib             go1.20.7             go-module  CVE-2023-39323  High
stdlib             go1.20.7             go-module  CVE-2023-45284  Medium
stdlib             go1.20.7             go-module  CVE-2023-39326  Medium
stdlib             go1.20.7             go-module  CVE-2023-39319  Medium
stdlib             go1.20.7             go-module  CVE-2023-39318  Medium
stdlib             go1.21.0             go-module  CVE-2023-39320  Critical
stdlib             go1.21.0             go-module  CVE-2023-45285  High
stdlib             go1.21.0             go-module  CVE-2023-45283  High
stdlib             go1.21.0             go-module  CVE-2023-44487  High
stdlib             go1.21.0             go-module  CVE-2023-39325  High
stdlib             go1.21.0             go-module  CVE-2023-39323  High
stdlib             go1.21.0             go-module  CVE-2023-39322  High
stdlib             go1.21.0             go-module  CVE-2023-39321  High
stdlib             go1.21.0             go-module  CVE-2023-45284  Medium
stdlib             go1.21.0             go-module  CVE-2023-39326  Medium
stdlib             go1.21.0             go-module  CVE-2023-39319  Medium
stdlib             go1.21.0             go-module  CVE-2023-39318  Medium
A newer version of grype is available for download: 0.74.2 (installed version is 0.73.4)

And here are the results without the --by-cve flag:

 % grype quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6
 ⠴ Vulnerability DB                ━━━━━━━━━━━━━━━━━━━━  [checking for update]
 ⠴ Vulnerability DB                ━━━━━━━━━━━━━━━━━━━━  [checking for update]
 ✔ Vulnerability DB                [no update available]
 ✔ Vulnerability DB                [no update available]
 ✔ Vulnerability DB                [no update available]
 ✔ Vulnerability DB                [no update available]
 ✔ Vulnerability DB                [no update available]
 ✔ Loaded image                                                                                                                                                           quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6
 ✔ Parsed image                                                                                                                                                                                      sha256:b747f07011a2c82631683d826b2275a6fcfbe6dfb5c26ac48d69446767cd7511
 ✔ Cataloged packages              [176 packages]
 ✔ Scanned for vulnerabilities     [30 vulnerability matches]
   ├── by severity: 1 critical, 18 high, 11 medium, 0 low, 0 negligible
   └── by status:   9 fixed, 21 not-fixed, 0 ignored
[0002]  WARN some package(s) are missing CPEs. This may result in missing vulnerabilities. You may autogenerate these using: --add-cpes-if-none
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/api, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/api" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/apimachinery, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/apimachinery" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/cli-runtime, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/cli-runtime" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/client-go, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/client-go" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/component-base, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/component-base" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/component-helpers, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/component-helpers" ver="(devel)": Malformed version: (dev
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/kubectl, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/kubectl" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=./staging/src/k8s.io/metrics, version=(devel), upstreams=0)): matcher failed to parse version pkg="./staging/src/k8s.io/metrics" ver="(devel)": Malformed version: (devel)
[0002]  WARN could not match by package language (package=Pkg(type=go-module, name=github.com/tigera/cc-operator, version=(devel), upstreams=0)): matcher failed to parse version pkg="github.com/tigera/cc-operator" ver="(devel)": Malformed version: (devel)
NAME               INSTALLED  FIXED-IN  TYPE       VULNERABILITY        SEVERITY
golang.org/x/net   v0.13.0    0.17.0    go-module  GHSA-4374-p667-p6c8  High
golang.org/x/net   v0.13.0    0.17.0    go-module  GHSA-qppj-fm5r-hxr3  Medium
golang.org/x/net   v0.14.0    0.17.0    go-module  GHSA-4374-p667-p6c8  High
golang.org/x/net   v0.14.0    0.17.0    go-module  GHSA-qppj-fm5r-hxr3  Medium
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  GHSA-q78c-gwqw-jcmc  High
k8s.io/kubernetes  v1.28.0    1.28.4    go-module  GHSA-hq6q-c2x6-hmch  High
k8s.io/kubernetes  v1.28.0    1.28.1    go-module  GHSA-7fxm-f474-hf8w  High
stdlib             go1.20.7             go-module  CVE-2023-45285       High
stdlib             go1.20.7             go-module  CVE-2023-45283       High
stdlib             go1.20.7             go-module  CVE-2023-44487       High
stdlib             go1.20.7             go-module  CVE-2023-39325       High
stdlib             go1.20.7             go-module  CVE-2023-39323       High
stdlib             go1.20.7             go-module  CVE-2023-45284       Medium
stdlib             go1.20.7             go-module  CVE-2023-39326       Medium
stdlib             go1.20.7             go-module  CVE-2023-39319       Medium
stdlib             go1.20.7             go-module  CVE-2023-39318       Medium
stdlib             go1.21.0             go-module  CVE-2023-39320       Critical
stdlib             go1.21.0             go-module  CVE-2023-45285       High
stdlib             go1.21.0             go-module  CVE-2023-45283       High
stdlib             go1.21.0             go-module  CVE-2023-44487       High
stdlib             go1.21.0             go-module  CVE-2023-39325       High
stdlib             go1.21.0             go-module  CVE-2023-39323       High
stdlib             go1.21.0             go-module  CVE-2023-39322       High
stdlib             go1.21.0             go-module  CVE-2023-39321       High
stdlib             go1.21.0             go-module  CVE-2023-45284       Medium
stdlib             go1.21.0             go-module  CVE-2023-39326       Medium
stdlib             go1.21.0             go-module  CVE-2023-39319       Medium
stdlib             go1.21.0             go-module  CVE-2023-39318       Medium
A newer version of grype is available for download: 0.74.2 (installed version is 0.73.4)

As you can tell, also here the summary does not add up.

Environment:

  • Output of grype version:
 % grype version
Application:         grype
Version:             0.73.4
BuildDate:           2023-11-30T14:22:34Z
GitCommit:           brew
GitDescription:      [not provided]
Platform:            darwin/arm64
GoVersion:           go1.21.4
Compiler:            gc
Syft Version:        v0.98.0
Supported DB Schema: 5
  • OS (e.g: cat /etc/os-release or similar): macos M1
@dwertent dwertent added the bug Something isn't working label Jan 23, 2024
@tgerla
Copy link
Contributor

tgerla commented Feb 1, 2024 •
edited
Loading

Hi @dwertent, thank you for the detailed report. We do have some circumstances where the summary count won't line up with the default table view output, specifically when there are duplicate rows in the table referring to the same vulnerability against the same package but installed in different places on your system. The JSON output should be correct relative to the the summary counts, but we agree that this is confusing.

Here is a similar issue with our proposed fix: annotate de-duplicated rows in the table output with something like (2) to indicate that a vulnerability has a duplicate: #1327

We need to do some more investigation to make sure that our explanation is correct, we'll add this issue to our backlog to look at when we are able. Thanks again!

@tgerla tgerla moved this to Backlog in OSS Feb 1, 2024
@wagoodman
Copy link
Contributor

I quick check looking at the JSON output too shows that there is a count issue still:

$ grype quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6 --by-cve -o json | jq '.matches[] | select(.vulnerability.severity == "Medium") | .vulnerability.id ' | wc -l

 ✔ Vulnerability DB                [no update available]
 ✔ Loaded image                                                                                             quay.io/tigera/cc-operator@sha256:d6451279b74c73a669d2901c321d386163f09fdc3cccabaf4d34b2ad065ea6c6
 ✔ Parsed image                                                                                                                        sha256:b747f07011a2c82631683d826b2275a6fcfbe6dfb5c26ac48d69446767cd7511
 ✔ Scanned for vulnerabilities     [30 vulnerability matches]
   ├── by severity: 1 critical, 18 high, 11 medium, 0 low, 0 negligible
   └── by status:   9 fixed, 21 not-fixed, 0 ignored
...
       8

@spiffcs spiffcs moved this from Backlog to Ready in OSS Feb 22, 2024
@wagoodman
Copy link
Contributor

For this image at this time, here are all of the matches with a severity of medium (13)

GHSA-qppj-fm5r-hxr3 	golang.org/x/[email protected]
GHSA-qppj-fm5r-hxr3 	golang.org/x/[email protected]
GHSA-qppj-fm5r-hxr3 	golang.org/x/[email protected]
GHSA-8r3f-844c-mc37 	google.golang.org/[email protected]
GHSA-8r3f-844c-mc37 	google.golang.org/[email protected]
CVE-2023-39318      	[email protected]
CVE-2023-39319      	[email protected]
CVE-2023-39326      	[email protected]
CVE-2023-45284      	[email protected]
CVE-2023-39318      	[email protected]
CVE-2023-39319      	[email protected]
CVE-2023-39326      	[email protected]
CVE-2023-45284      	[email protected]

Taking a look at the non-stdlib matches, when not using --by-cve you'll see:

NAME                        INSTALLED  FIXED-IN  TYPE       VULNERABILITY        SEVERITY
golang.org/x/net            v0.13.0    0.17.0    go-module  GHSA-4374-p667-p6c8  High
golang.org/x/net            v0.13.0    0.17.0    go-module  GHSA-qppj-fm5r-hxr3  Medium
golang.org/x/net            v0.14.0    0.17.0    go-module  GHSA-4374-p667-p6c8  High
golang.org/x/net            v0.14.0    0.17.0    go-module  GHSA-qppj-fm5r-hxr3  Medium
google.golang.org/protobuf  v1.30.0    1.33.0    go-module  GHSA-8r3f-844c-mc37  Medium
google.golang.org/protobuf  v1.31.0    1.33.0    go-module  GHSA-8r3f-844c-mc37  Medium
k8s.io/kubernetes           v1.28.0    1.28.1    go-module  GHSA-q78c-gwqw-jcmc  High
k8s.io/kubernetes           v1.28.0    1.28.4    go-module  GHSA-hq6q-c2x6-hmch  High
k8s.io/kubernetes           v1.28.0    1.28.1    go-module  GHSA-7fxm-f474-hf8w  High

But with --by-cve you'll see:

golang.org/x/net            v0.13.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net            v0.13.0    0.17.0    go-module  CVE-2023-39325  High
golang.org/x/net            v0.14.0    0.17.0    go-module  CVE-2023-44487  High
golang.org/x/net            v0.14.0    0.17.0    go-module  CVE-2023-39325  High
google.golang.org/protobuf  v1.30.0    1.33.0    go-module  CVE-2024-24786  Unknown
google.golang.org/protobuf  v1.31.0    1.33.0    go-module  CVE-2024-24786  Unknown
k8s.io/kubernetes           v1.28.0    1.28.4    go-module  CVE-2023-5528   High
k8s.io/kubernetes           v1.28.0    1.28.1    go-module  CVE-2023-3955   High
k8s.io/kubernetes           v1.28.0    1.28.1    go-module  CVE-2023-3676   High

Note that the severities are different for the CVEs than the GHSA's for the same matches.

This is mostly due to the way we are implementing --by-cve, which I think will need to be heavily changed. Today this is a presentation concern... we make no changes to the matching process but instead after the matching is done transform non-CVEs to CVEs (if possible) and coalesce the set of transformed matches. I think this is where most of the issue is occuring.

However, there is still one more issue: we search for matches without looking for duplicate packages first:

[0000] TRACE searching for vulnerability matches package=pkg:golang/golang.org/x/[email protected]
[0000] DEBUG found 2 vulnerabilities package=pkg:golang/golang.org/x/[email protected]
[0000] DEBUG   ├── namespace=github:language:go vuln=GHSA-4374-p667-p6c8
[0000] DEBUG   └── namespace=github:language:go vuln=GHSA-qppj-fm5r-hxr3
[0000] TRACE searching for vulnerability matches package=pkg:golang/golang.org/x/[email protected]
[0000] DEBUG found 2 vulnerabilities package=pkg:golang/golang.org/x/[email protected]
[0000] DEBUG   ├── namespace=github:language:go vuln=GHSA-4374-p667-p6c8
[0000] DEBUG   └── namespace=github:language:go vuln=GHSA-qppj-fm5r-hxr3

This is a debug log snippet showing the back-to-back (duplicate) lookup. This hints at either:

  • more intelligent tracking to look for already discovered results
  • or more intelligent deduplication logic before attempting searches

@wagoodman wagoodman moved this from Ready to Backlog in OSS Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
OSS
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tgerla @wagoodman @dwertent