Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade pattern for handling blackbox exporter CA in older hosts #747

Closed
hellais opened this issue Oct 12, 2023 · 2 comments
Closed

Upgrade pattern for handling blackbox exporter CA in older hosts #747

hellais opened this issue Oct 12, 2023 · 2 comments
Assignees
@FedericoCeratto
Labels
ooni/devops Issues related to https://github.com/ooni/sysadmin priority/high

Comments

@hellais
Copy link
Member

hellais commented Oct 12, 2023

Older hosts are using ooca_ca to generate the certificates used by the prometheus scraper to collect blackbox exported metrics.

Newer hosts are handling this "on the fly", like this: https://github.com/ooni/sysadmin/blob/master/ansible/roles/base-bullseye/tasks/main.yml#L219.

https://github.com/ooni/sysadmin/blob/master/ansible/roles/base-bullseye/tasks/main.yml#L229 <--- this deploys the updated cert to all hosts sending data to vector without running a full host bootstrap

We should update the old hosts to make use of this new pattern.
@hellais hellais added the ooni/devops Issues related to https://github.com/ooni/sysadmin label Oct 12, 2023
@hellais
Copy link
Member Author

hellais commented Oct 16, 2023

This was leading to the following error when running the node exporter scrape:
Screenshot 2023-10-16 at 17 33 22

As a hotfix we just put in the tls_config section the following:

    tls_config:
      # XXX this is a hotfix to https://github.com/ooni/backend/issues/747
      insecure_skip_verify: true

@bassosimone bassosimone mentioned this issue Dec 20, 2023
Merged
bassosimone added a commit to ooni/sysadmin that referenced this issue Dec 20, 2023
@bassosimone
feat(monitoring): monitor .well-known/acme-challenge for ams-pg-test (#…
8125d56 
…507)

Alert if we screwed the pooch with haproxy and nginx hence we cannot
access .well-know/acme-challenge anymore.

Based on a true story.

While there:

1. remove monitoring code for onion services that was removed on
monitoring.ooni.org but was still in ansible
2. make sure ansible contains the hotfix for
ooni/backend#747
@hellais
Copy link
Member Author

hellais commented Jan 7, 2025

Given that we are discontinuing these hosts (ooni/devops#121), we can consider this as no longer relevant. Closing.

@hellais hellais closed this as completed Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FedericoCeratto FedericoCeratto

Labels
ooni/devops Issues related to https://github.com/ooni/sysadmin priority/high
Projects
Sprint Planning
Archived in project
Milestone
No milestone
Development

No branches or pull requests
2 participants
@FedericoCeratto @hellais