Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fcos-policy-engine: fetch graph from fcos-graph-builder instance #21

Merged
merged 1 commit into from
Jul 27, 2020
Merged

fcos-policy-engine: fetch graph from fcos-graph-builder instance #21

merged 1 commit into from
Jul 27, 2020

Conversation

zonggen
Copy link
Member

@zonggen zonggen commented Jul 15, 2020

Removes previous empty dummy graph returned by fcos-policy-engine and fetches the graph from fcos-graph-builder instance, which is running under the same pod as fcos-policy-engine.

Also changes ports used by fcos-* such that they don't conflict with dumnati ports.

Related to: #3
Signed-off-by: Allen Bai [email protected]

@zonggen
Copy link
Member Author

zonggen commented Jul 15, 2020

A local run shows the expected and same behavior as the dumnati counterpart:

fcos-graph-builder

GET http://127.0.0.1:6060/metrics 
# HELP fcos_cincinnati_gb_scraper_graph_final_edges Number of edges in the cached graph, after processing
# TYPE fcos_cincinnati_gb_scraper_graph_final_edges gauge
fcos_cincinnati_gb_scraper_graph_final_edges{stream="next"} 9
fcos_cincinnati_gb_scraper_graph_final_edges{stream="stable"} 14
fcos_cincinnati_gb_scraper_graph_final_edges{stream="testing"} 29
# HELP fcos_cincinnati_gb_scraper_graph_final_releases Number of releases in the cached graph, after processing
# TYPE fcos_cincinnati_gb_scraper_graph_final_releases gauge
fcos_cincinnati_gb_scraper_graph_final_releases{stream="next"} 10
fcos_cincinnati_gb_scraper_graph_final_releases{stream="stable"} 15
fcos_cincinnati_gb_scraper_graph_final_releases{stream="testing"} 30
# HELP fcos_cincinnati_gb_scraper_graph_last_refresh_timestamp UTC timestamp of last graph refresh
# TYPE fcos_cincinnati_gb_scraper_graph_last_refresh_timestamp gauge
fcos_cincinnati_gb_scraper_graph_last_refresh_timestamp{stream="next"} 1594844374
fcos_cincinnati_gb_scraper_graph_last_refresh_timestamp{stream="stable"} 1594844373
fcos_cincinnati_gb_scraper_graph_last_refresh_timestamp{stream="testing"} 1594844373
# HELP fcos_cincinnati_gb_scraper_upstream_scrapes_total Total number of upstream scrapes
# TYPE fcos_cincinnati_gb_scraper_upstream_scrapes_total counter
fcos_cincinnati_gb_scraper_upstream_scrapes_total{stream="next"} 9
fcos_cincinnati_gb_scraper_upstream_scrapes_total{stream="stable"} 9
fcos_cincinnati_gb_scraper_upstream_scrapes_total{stream="testing"} 9
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1594844132
GET http://127.0.0.1:5050/v1/graph?basearch=x86_64&stream=stable 
{
  "nodes": [
    {
      "version": "31.20200108.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "0",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "113aa27efe1bbcf6324af7423f64ef7deb0acbf21b928faec84bf66a60a5c933"
    },
    {
      "version": "31.20200113.3.1",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "1"
      },
      "payload": "f480038412cba26ab010d2cd5a09ddec736204a6e9faa8370edaa943cf33c932"
    },
    {
      "version": "31.20200118.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "2"
      },
      "payload": "093f7da6ffa161ae1648a05be9c55f758258ab97b55c628bea5259f6ac6e370e"
    },
    {
      "version": "31.20200127.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "3",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "2ba7bb9678d86fc469f1920a03c270d7057e0b93d06fcd1a437f1f79bdc26d83"
    },
    {
      "version": "31.20200210.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "4"
      },
      "payload": "4ea6beed22d0adc4599452de85820f6e157ac1750e688d062bfedc765b193505"
    },
    {
      "version": "31.20200223.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "5"
      },
      "payload": "3477082298eb5e2213433415c78f0919a991b40e62726b8fde434d244c1ec1b6"
    },
    {
      "version": "31.20200310.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "6",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "436592e6eb93e899bebab8dbd17514c85be683390ef8bbce8c6d96069ce4c543"
    },
    {
      "version": "31.20200323.3.2",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "7"
      },
      "payload": "9b322b9cead87ebf80881b75560e481b6e5a26966814b01bf0c12b61fa363708"
    },
    {
      "version": "31.20200407.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "8",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "89e17cc21b6aa3bea8959d1e6957fda157168d57ba6805d8a36142184edc2901"
    },
    {
      "version": "31.20200420.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "9",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "b3fc3a3e8513d7e424d0ced1e2517484cb766d238951f2fdec3da2fed3522efb"
    },
    {
      "version": "31.20200505.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "10",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "01f074cc6cd88d8d2b43f821da692f2367c101eb4377802cb35092bde0ef02f7"
    },
    {
      "version": "31.20200517.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "11"
      },
      "payload": "967b7b8d624e6d10ff51c2e81ef198fae966c567ac2e9b479771c693d0987949"
    },
    {
      "version": "32.20200601.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "12",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "b51037798e93e5aae5123633fb596c80ddf30302b5110b0581900dbc5b2f0d24"
    },
    {
      "version": "32.20200615.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "13",
        "org.fedoraproject.coreos.updates.barrier": "true",
        "org.fedoraproject.coreos.updates.barrier_reason": "https://github.com/coreos/fedora-coreos-tracker/issues/484"
      },
      "payload": "abb4b978999bd689ff36114e99c737be16026bc154a4428d4c84a340758c5b01"
    },
    {
      "version": "32.20200629.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "14",
        "org.fedoraproject.coreos.updates.start_epoch": "1594650600",
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.updates.rollout": "true",
        "org.fedoraproject.coreos.updates.duration_minutes": "4320",
        "org.fedoraproject.coreos.updates.start_value": "0"
      },
      "payload": "6df95bdb2fe2d36e091d4d18e3844fa84ce4b80ea3bd0947db5d7a286ff41890"
    }
  ],
  "edges": [
    [
      13,
      14
    ],
    [
      0,
      13
    ],
    [
      1,
      13
    ],
    [
      2,
      13
    ],
    [
      3,
      13
    ],
    [
      4,
      13
    ],
    [
      5,
      13
    ],
    [
      6,
      13
    ],
    [
      7,
      13
    ],
    [
      8,
      13
    ],
    [
      9,
      13
    ],
    [
      10,
      13
    ],
    [
      11,
      13
    ],
    [
      12,
      13
    ]
  ]
}

fcos-policy-engine

GET http://127.0.0.1:6061/metrics 
# HELP fcos_cincinnati_pe_v1_graph_incoming_requests_total Total number of incoming HTTP client request to /v1/graph
# TYPE fcos_cincinnati_pe_v1_graph_incoming_requests_total counter
fcos_cincinnati_pe_v1_graph_incoming_requests_total 1
# HELP fcos_cincinnati_pe_v1_graph_rollout_wariness Per-request rollout wariness.
# TYPE fcos_cincinnati_pe_v1_graph_rollout_wariness histogram
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0"} 0
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.1"} 0
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.2"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.30000000000000004"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.4"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.5"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.6000000000000001"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.7000000000000001"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.8"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="0.9"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="1"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_bucket{le="+Inf"} 1
fcos_cincinnati_pe_v1_graph_rollout_wariness_sum 0.18848315746051456
fcos_cincinnati_pe_v1_graph_rollout_wariness_count 1
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1594844145
GET http://127.0.0.1:5051/v1/graph?basearch=x86_64&stream=stable 
{
  "nodes": [
    {
      "version": "31.20200108.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "0",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "113aa27efe1bbcf6324af7423f64ef7deb0acbf21b928faec84bf66a60a5c933"
    },
    {
      "version": "31.20200113.3.1",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "1",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "f480038412cba26ab010d2cd5a09ddec736204a6e9faa8370edaa943cf33c932"
    },
    {
      "version": "31.20200118.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "2"
      },
      "payload": "093f7da6ffa161ae1648a05be9c55f758258ab97b55c628bea5259f6ac6e370e"
    },
    {
      "version": "31.20200127.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "3"
      },
      "payload": "2ba7bb9678d86fc469f1920a03c270d7057e0b93d06fcd1a437f1f79bdc26d83"
    },
    {
      "version": "31.20200210.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "4",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "4ea6beed22d0adc4599452de85820f6e157ac1750e688d062bfedc765b193505"
    },
    {
      "version": "31.20200223.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "5",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "3477082298eb5e2213433415c78f0919a991b40e62726b8fde434d244c1ec1b6"
    },
    {
      "version": "31.20200310.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "6",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "436592e6eb93e899bebab8dbd17514c85be683390ef8bbce8c6d96069ce4c543"
    },
    {
      "version": "31.20200323.3.2",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "7"
      },
      "payload": "9b322b9cead87ebf80881b75560e481b6e5a26966814b01bf0c12b61fa363708"
    },
    {
      "version": "31.20200407.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "8",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "89e17cc21b6aa3bea8959d1e6957fda157168d57ba6805d8a36142184edc2901"
    },
    {
      "version": "31.20200420.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "9",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "b3fc3a3e8513d7e424d0ced1e2517484cb766d238951f2fdec3da2fed3522efb"
    },
    {
      "version": "31.20200505.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.releases.age_index": "10"
      },
      "payload": "01f074cc6cd88d8d2b43f821da692f2367c101eb4377802cb35092bde0ef02f7"
    },
    {
      "version": "31.20200517.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "11",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "967b7b8d624e6d10ff51c2e81ef198fae966c567ac2e9b479771c693d0987949"
    },
    {
      "version": "32.20200601.3.0",
      "metadata": {
        "org.fedoraproject.coreos.releases.age_index": "12",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "b51037798e93e5aae5123633fb596c80ddf30302b5110b0581900dbc5b2f0d24"
    },
    {
      "version": "32.20200615.3.0",
      "metadata": {
        "org.fedoraproject.coreos.scheme": "checksum",
        "org.fedoraproject.coreos.updates.barrier_reason": "https://github.com/coreos/fedora-coreos-tracker/issues/484",
        "org.fedoraproject.coreos.updates.barrier": "true",
        "org.fedoraproject.coreos.releases.age_index": "13"
      },
      "payload": "abb4b978999bd689ff36114e99c737be16026bc154a4428d4c84a340758c5b01"
    },
    {
      "version": "32.20200629.3.0",
      "metadata": {
        "org.fedoraproject.coreos.updates.rollout": "true",
        "org.fedoraproject.coreos.updates.duration_minutes": "4320",
        "org.fedoraproject.coreos.updates.start_epoch": "1594650600",
        "org.fedoraproject.coreos.releases.age_index": "14",
        "org.fedoraproject.coreos.updates.start_value": "0",
        "org.fedoraproject.coreos.scheme": "checksum"
      },
      "payload": "6df95bdb2fe2d36e091d4d18e3844fa84ce4b80ea3bd0947db5d7a286ff41890"
    }
  ],
  "edges": [
    [
      13,
      14
    ],
    [
      0,
      13
    ],
    [
      1,
      13
    ],
    [
      2,
      13
    ],
    [
      3,
      13
    ],
    [
      4,
      13
    ],
    [
      5,
      13
    ],
    [
      6,
      13
    ],
    [
      7,
      13
    ],
    [
      8,
      13
    ],
    [
      9,
      13
    ],
    [
      10,
      13
    ],
    [
      11,
      13
    ],
    [
      12,
      13
    ]
  ]
}

@zonggen zonggen requested a review from lucab July 15, 2020 20:26
@zonggen zonggen force-pushed the split-dumnati-binary branch 2 times, most recently from 2a7c110 to d470688 Compare July 15, 2020 21:50
@zonggen zonggen force-pushed the split-dumnati-binary branch from d470688 to 80a328c Compare July 16, 2020 16:41
@zonggen
Copy link
Member Author

zonggen commented Jul 16, 2020 •
edited
Loading

Removed the port bits out of this PR and updated according to reviews.

Rebased on master.

@zonggen zonggen force-pushed the split-dumnati-binary branch from 80a328c to ca4b76f Compare July 16, 2020 17:54
@zonggen zonggen force-pushed the split-dumnati-binary branch from ca4b76f to 20b1f74 Compare July 21, 2020 15:35
fcos-policy-engine: fetch graph from fcos-graph-builder instance
a43cb89 
Removes previous empty dummy graph returned by fcos-policy-engine and
fetches the graph from fcos-graph-builder instance, which is running
under the same pod as fcos-policy-engine.

Signed-off-by: Allen Bai <[email protected]>
@zonggen zonggen force-pushed the split-dumnati-binary branch from 20b1f74 to a43cb89 Compare July 21, 2020 15:42
lucab
lucab reviewed Jul 27, 2020
View reviewed changes
fcos-policy-engine/src/utils.rs
node_uuid: None,
};
// Cannot use `?` directly here otherwise will produce the error:
// the trait `std::marker::Sync` is not implemented for `(dyn std::error::Error + std::marker::Send + 'static)`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because errors coming from serde_qs are not Sync (due to their use of error-chain).
Can you please open a bug to the upstream repo and ask them to consider using a better library for error-handling (i.e. thiserror or snafu)?

@lucab lucab merged commit 23e9985 into coreos:master Jul 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucab lucab lucab approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zonggen @lucab