Display historical hub usage data as a heatmap over time by using Prometheus data

[choldgraf]

Currently, we directly query each hub's metrics for Daily / Weekly / Monthly active users. We can then display this as a snapshot in time.

It would be much more insightful and actionable if we could track the trends in usage over time. This would let us:

• Get a better high-level view of the trajectory of our service usage.
• Identify causal relationships between things we've done and their impact on usage.
• Use that to make better decisions about how to grow the service.

It could also be a way to provide this information to communities (e.g., via a MyST site similar to https://2i2c.org/kpis).

In particular this would be useful in fundraising efforts where we want to show off the growth of the service, so I'm attaching it to:

• https://github.com/2i2c-org/meta/issues/973

Implementation

@yuvipanda has described how to use PromQL to grab Prometheus data from our clusters/hubs in this HackMD. It is being tracked in this issue:

• Document how to programmatically access Grafana / Prometheus data infrastructure#1785

Ideal metrics to track over time:

• Hub and cluster name
• Daily, Weekly, and Monthly Active Users per hub

Ideally we'd have this data indefinitely.

Inspiration

An alternative approach is to automate the creation of an easier-to-parse data archive. For example mybinder.org does this with repository launches at:

https://archive.analytics.mybinder.org/

Tasks

Beta Give feedback

1. Store Monthly, Weekly, and Daily active users for each cluster and hub in a persistent space infrastructure#3917
   +0
2. Utilize the output of the above in our KPIs dashboard and document how to do so

[choldgraf]

I think we can consider this one completed now that these two PRs have been merged:

• Partnerships: Document how to track hub usage with Grafana  team-compass#859
• Add grafana data updates #33