Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: default composer image version #149

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

alexis974
Copy link

The current default composer image is not supported anymore on gcp

image

The previous image was not supported anymore on gcp: googleapi: Error 400
@alexis974 alexis974 requested review from imrannayer and a team as code owners November 15, 2024 14:03
Copy link

google-cla bot commented Nov 15, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@alexis974 alexis974 changed the title Update default composer image version fix: default composer image version Nov 15, 2024
@imrannayer
Copy link
Collaborator

@alexis974 can u plz follow this guide to fix lint issue.

@alexis974
Copy link
Author

Hello @imrannayer, done !

@imrannayer
Copy link
Collaborator

/gcbrun

@@ -103,7 +103,7 @@ variable "env_variables" {
variable "image_version" {
type = string
description = "The version of the aiflow running in the cloud composer environment."
default = "composer-2.5.0-airflow-2.6.3"
default = "composer-3-airflow-2.10.2"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not the valid image version, in composer v2, you need to use a v2.x image version

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did work when I used image_version = "composer-3-airflow-2.10.2" with the create_environment_v2 module in version v5.2.0

If we check the Cloud composer version list we can see that for composer v3, minor and patch are not present in version name

@imrannayer
Copy link
Collaborator

/gcbrun

@imrannayer imrannayer self-assigned this Jan 10, 2025
@imrannayer
Copy link
Collaborator

@alexis974 can u plz use image from composer 2? Integration test is failing with following error. We need to handle this exception for composer 3.

TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185: Error: error in configuration, config.0.private_environment_config should not be used in Composer 3
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185: 
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185:   with module.simple-composer.module.simple-composer-environment.google_composer_environment.composer_env,
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185:   on ../../../modules/create_environment_v2/main.tf line 27, in resource "google_composer_environment" "composer_env":
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185:   27: resource "google_composer_environment" "composer_env" {
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z command.go:185: 
TestSimpleComposerEnvV2Module 2025-01-10T13:21:42Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: error in configuration, config.0.private_environment_config should not be used in Composer 3

As of now, v3 is currently not supported by integration tests in the CI
@alexis974
Copy link
Author

Alright @imrannayer ,
I downgraded to the latest composer v2 version which is composer-2.10.2-airflow-2.10.2 as stated here.

Notice that it is supported until January 10, 2026

@alexis974 alexis974 requested a review from nlevee January 17, 2025 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants