Skip to content

FAQ

Jump to bottom
Sarah French edited this page Nov 18, 2024 · 20 revisions

Frequently Asked Questions

Does the provider have support for ____?

Resources in the provider tend to match up with the resource name in the REST API for a product. The Provider Documentation lists all resources available in the most recent version of the provider.

Note that not all GCP functionality makes sense to be added to Terraform. For something to make sense, it must:

  • have an associated REST API: To add it to the google or google-beta provider, that API must be publicly-accessible.
  • be able to be represented declaratively and idempotently: Long-lived infrastructure is a good fit for Terraform. Short-lived jobs are not, as it's not clear what the intent is if applied multiple times.

I've made a change to the providers. When will the feature be released?

Google provider releases are made every week on Monday close to EOD in Pacific time, unless there are extenuating circumstances.

You can monitor the release pages for the appropriate provider (google (GA) or google-beta) to determine if a weekly release cut is made, or a specific feature is included. We do not provide tracking bugs as part of our release process, nor tracking for individual features' release.

We cut our release branch on Tuesday nights, and release the following Monday. This means that any code change that is merged on a Monday or Tuesday will be released the following Monday, and any change that is merged W-F will be released the Monday after.

I've added the beta provider in my config. Why does it still use the GA provider?

If you want to use beta provider, provider = google-beta should be set explicitly on every resource, even if you’ve only defined a google-beta provider block or you've already added the beta provider in the required_providers block.

Please refer to Google Provider Versions for more detailed information.

What is the difference between id and project_id in the google_project resource?
Which one should I use?

id is an identifier for the project with the format projects/{{project}}, while project_id is the real project ID with the format {{project}}.

project_id should be used as the project argument in your configuration, as all resources should be able to accept the {{project}} format for the project field.

How do I update the Go version used by the Terraform providers `google` and `google-beta`?

Currently the process to update the Go version requires multiple PRs, across more than one repository.

Below we describe the PRs to prepare, and then there's a section describing how these should be merged.

1) Update Dockerfiles in GoogleCloudPlatform/magic-modules

  • Create a new branch in GoogleCloudPlatform/magic-modules
  • Edit .ci/containers/build-environment/Dockerfile to reference the new Go version
  • Edit .ci/containers/go-plus/Dockerfile to reference the new Go version
  • If there are newer Dockerfiles not mentioned here inspect them too!
  • Commit those changes, and open a PR containing only changes to the Dockerfiles.

Don't merge this PR yet, see the section on merge order below.

Example diffs expected in this step from this past example:

-FROM golang:1.20-bullseye AS builder
+FROM golang:1.21-bullseye AS builder

2) Updates to CI, GHAs and other code in GoogleCloudPlatform/magic-modules

This step could be separated out to be CI & GHA and 'everything else', but it's ok to do them all in one PR.

The third PR will update the code that uses the containers made from the Dockerfiles above:

  • Create a new branch in GoogleCloudPlatform/magic-modules
  • Update the go.mo, go.sum, and .go-version files in the TPG and TPGB downstream repos
    • Update go.mod in your local clone of hashicorp/terraform-provider-google to contain the new Go version
    • Run go mod tidy in the project root to update go.sum
    • Copy the contents of the updated go.mod file into mmv1/third_party/terraform/go.mod.erb in magic-modules. Ensure <% autogen_exception -%> is still at the top of the file afterwards.
    • Copy the contents of the updated go.sum file into mmv1/third_party/terraform/go.sum in magic-modules.
    • Edit mmv1/third_party/terraform/.go-version to contain the new Go version. Note: this file impacts the release process and also controls the Go version used in TeamCity to test the provider.
  • Update the GitHub Action workflows and other CI files in GoogleCloudPlatform/magic-modules used to test and build the providers
  • Update other tools that run in CI and in PR checks that need to be kept in sync with the Go version used by the provider
  • After completing the above, perform some cautionary global searches in the codebase for the old Go version string.
  • Commit all your changes from the above steps, and open a PR

Don't merge this PR yet, see the section on merge order below.

3) Updates to go.mod files not controlled by GoogleCloudPlatform/magic-modules : GoogleCloudPlatform/terraform-google-conversion

We need a PR to update go.mod and go.sum in terraform-google-conversion:

  • Create a new branch in GoogleCloudPlatform/terraform-google-conversion or your fork
  • Edit the go.mod file to include the new Go version
  • Run go mod tidy to update the go.sum file, given the new Go version
  • Commit those changes using conventional commits, e.g. chore: update to Go 1.21.

Don't merge this PR yet, see the section on merge order below.

This PR is necessary because the go.mod and go.sum files of TGC aren't controlled by magic-modules. See this issue for details: https://github.com/hashicorp/terraform-provider-google/issues/17480

4) Updates to go.mod files not controlled by GoogleCloudPlatform/magic-modules : EAP and any other downstreams from Magic Modules

I don't know details about these repositories and any quirks in how their go.mod and Go versions are managed. These PRs are likely to be in the same class as the TGC PR above.

Merge order

Ask everyone to pause making merges to main in GoogleCloudPlatform/magic-modules. This is important because any PR unrelated to the Go version upgrade that is merged at the same time as the PRs above can experience a mismatch between the container images it's using and other code used by its Cloud Build runs.

An approach to handle this could be to pre-arrange a merge window of ~30 minutes that is shared with the wider team.

Once you're confident that no other merges will occur, follow these steps:

  • Merge the first PR (1) to update Dockerfiles.
  • Wait for the new images to be published:
    • You can find information about this process by looking at Checks running on your merge commit from (1) on main. Cloud Build pipelines triggered by your merge will show images being built and published.
    • Alternatively, wait for a new image version to be listed in Container Registry.
  • Once new images are published and automatically tagged as latest you can continue.
  • Merge the PR that updates go.mod in TGC (2).
  • Similarly, merge other PRs updating the EAP provider and other downstreams.
  • Merge the PR updating everything besides Dockerfiles into main in magic-modules (3).
    • Automation triggered by merging (3) will use code from downstreams when performing syncs. This is why we perform direct updates to downstreams before this merge.

Monitor the CI triggered by merging (3) and make sure it gets through all steps that sync with the downstreams.

Afterwards it would be good to monitor merging of the next PR to ensure everything behaves as expected.

What acceptance tests should be skipped during VCR testing?

Our VCR testing system allows tests to run in 'RECORDING' mode where HTTP requests and responses are recorded to file during a test run, and tests can be re-run in 'REPLAYING' mode so that they don't interact with GCP APIs and instead only use recorded information. This process is based on a fundamental requirement that all HTTP traffic goes via a single HTTP client, as that client manages the recording and replaying process.

For the VCR system to be truly broken there would need to be code changes to the provider/VCR utils code that causes the problem. One case of this in the past was when muxing was added to the provider and this caused the provider to use multiple HTTP clients internally. This resulted in VCR failures (i.e. Requested interaction not found errors during REPLAYING mode) because HTTP traffic was no longer travelling via a single client, so RECORDING mode could only record an incomplete dataset. This was the problem identified in this issue: https://github.com/hashicorp/terraform-provider-google/issues/14158

Besides that, VCR has some other quirks that mean that tests may not behave as you expected. In those cases you will want to disable VCR mode:

Expectation What can break this expectation How to address it
"No HTTP traffic is going to GCP APIs at all in VCR REPLAYING mode" If you use ExternalProviders to pull in a previous version of the Google provider (e.g. to compare behaviour before/after a change) then the provider accessed via ExternalProviders will not use VCR recordings If using ExternalProviders is necessary, you should skip the test in VCR. This will avoid API calls in situations like tests running on PRs.
"No HTTP traffic is happening at all when using VCR REPLAYING mode! I can run them in an air-gapped environment" Some acceptance tests include code that uses client libraries to interact with GCP before the provider is even used, e.g. functions for bootstrapping resources. That code will raise errors if they're unable to run. Also, the provider is going to interact with GCP outside of provisioning resources, e.g. interacting with the Security Token Service API to get tokens from credentials the provider is configured with. Finally, if you use ExternalProviders at all those providers are unaffected by the VCR system. Running acceptance tests in a truly zero-internet traffic way isn't possible.
"My acceptance test should be fine using multiple aliased providers" Sadly, this is not the case. The VCR system works by caching how a provider is configured for a given test and reusing that data repeatedly. This means that configuring the provider in different ways during the same acceptance test is impossible; the first way the provider is configured will be reused for all aliases regardless of their inputs via different provider blocks. See https://github.com/hashicorp/terraform-provider-google/issues/20019 Skip the acceptance test in VCR is using multiple aliases is necessary for what you are testing
Clone this wiki locally