Questions on Etsy's Cloud Jewels

[srini1978]

@seanmcilroy29 @Henry-WattTime

Had some questions based on Etsy's Cloud Jewels presentation

1. In the approximation for compute, you have produced average utilization of 45%. How close is this to your actual workload utilization values ?
2. Do you have co-efficient for RAM ? If your workload uses a lot of in memory architecture, would you think it will be impacted.
3. What is the future state of your calculations. Now that we have gcp releasing emissions for the workloads does it make sense to set carbon goals rather than energy intensity goals. Is there a plan to calibrate Google official emissions with the cloud jewels values ?
4. How many workloads were used to produce this Watt-hour/VPU hours ratio ?
5. Do we have homogenous workloads or heterogenous workloads as part of your migration ? For e.g is your total migration mix of databases, web applications, web apis, batch processes etc.
6. Any thoughts around why carbon intensity was not used. This is reflection of the green energy in the grid right?

[esommer]

Hi! Great questions! Here's a stab at answers:

1. Average CPU util: Our utilization varies a lot based on service type. At the time, and to keep the calculation as generally applicable as we could, we decide to use the 45% as stated in the U.S. Data Center Energy Report. It is worth re-evaluating this decision, but it would add significant complexity to the calculation.
2. We do not have a coefficient for RAM. We know it is significant, and we caveat our reporting that we know we don't have full coverage of our energy use. At the time we released Cloud Jewels, we had not found a way to estimate RAM energy that we were confident in. It is definitely worth more research.
3. We would like to begin leveraging Thoughtworks' coefficients from their Cloud Carbon Footprint tool as a way to support standardization across the industry. I'll respond to why energy vs carbon in question 6. We have not tried to directly compare GCP's carbon data to Cloud Jewels energy data because a) we know that Cloud Jewels does not cover a lot of resources that GCP's carbon data does (RAM, Networking, proprietary service use like BigQuery), and b) we don't have granular-enough data about carbon at Google's data centers without paying for data from Electricity Map.
4. I'm not sure what ratio you're referring to with VPU or what you mean by "workload" here. To get Watt/vCPUh we use data from the SPEC server database and decide which rows are applicable based on processor models.
5. We run many, many different types of workloads, but GCP billing data only reports on generic vCPU hours. This is part of why we use the 45% average CPU util.
6. Carbon intensity is the industry standard, and we rely on GCP's data for carbon accounting. We additionally report energy because in 2017 we set an energy-intensity goal of a 25% reduction by 2025 and began disclosing this in our 10-k to the SEC. For continuity and to measure progress against our goal, we continue to calculate and report on energy. Also, beyond selecting a cloud provider (renewable energy was a significant factor in our decision to use GCP at the time), we have more direct control over how much energy we use and closer to real-time reporting on it for teams to respond to. While we would like to begin optimizing for carbon awareness for certain workloads in the future, it is not something we have invested in yet, and therefore energy is a good-enough proxy for now.

[srini1978]

Thanks @esommer . Appreciate your inputs.

Would you be interested in having a discussion on a GSF project that we are running ? IF so, please let me know and I can setup some time. The project that we are running is called CarbonQL https://github.com/Green-Software-Foundation/carbon-ql and in this we are planning to use online carbon calculation models. Your model is one of the ones we have considered. Please do take the time to read through our specs here https://github.com/Green-Software-Foundation/carbon-ql/blob/dev/specs/index.md. Let us know if you have any questions.

[esommer]

Hi @srini1978. Thanks for reaching out. CarbonQL looks like a really interesting standardization structure. I'd love to discuss it further!