Retain binderhub analytics events

[yuvipanda]

As part of #4365, binderhub events now flow into a log in a separate google cloud project made just for this (https://console.cloud.google.com/logs/query;query=logName%3D%22projects%2Fbinderhub-event-logs%2Flogs%2Fbinderhub-event-logs%22;cursorTimestamp=2024-07-12T19:57:06.488977Z;duration=PT1H?referrer=search&project=binderhub-event-logs). However, this is retained only for 30 days.

What we want to do with this data is to be determined eventually in collaboration with partnerships. In the meantime, we should take efforts to not lose it after 30 days.

Google Cloud logging supports routing logs to a specific destination where they can survive long term. Since we want the most cloud agnostic option, that would be routing it to a cloud storage bucket

This task is to create configuration so that only the logs of these launches (identified by logName="projects/binderhub-event-logs/logs/binderhub-event-logs") get archived into a new storage bucket (let's name it binder-2i2c-events). This can be done in the cloud console, no terraform needed at this moment.

[consideRatio]

I think i may have already done this, but not confirmed it works because i was low on time and it said it was getting done every hour.

I also changed the default log storage bucket to retain things for two years in this project in case it fails to transfer to the dedicated bucket for binderhub event logs.

[consideRatio]

Log router + log storage was things i tried using for this

[GeorgianaElena]

@consideRatio, I believe your setup was working indeed 🚀 It was however storing the logs into a logs into a Cloud Logging Bucket, and @yuvipanda's request was for a cloud storage bucket.

I just createad a new storage bucket with the name binder-2i2c-events, with the following configuration:
https://console.cloud.google.com/storage/browser/binder-2i2c-events;tab=configuration?project=binderhub-event-logs&supportedpurview=project

And then re-configured the log router that @consideRatio created to route the logs into the cloud storage bucket rather than the log bucket.

[GeorgianaElena]

@yuvipanda, before considering this done, can you please confirm that:

• the bucket's config looks good to you as I had to make some decissions like making it multi-region etc
• is ok to delete the logs bucket that Erik created (it currently has a few days worth of data)?

[yuvipanda]

@GeorgianaElena looks good to me! You can delete the old log bucket (which I assume no longer receives events), and close this ticket once you confirm new events have come to the cloud bucket!

[GeorgianaElena]

Data is in the new bucket, so deleting the old one. Thank you @yuvipanda!