google_bigquery_table error attempting to DROP COLUMN with flexible column name

[shoumoji]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.10.2
on MacOS and GitHub Actions (ubuntu:22.04LTS)

• provider registry.terraform.io/hashicorp/google v6.14.0

Affected Resource(s)

google_bigquery_table

Terraform Configuration

resource "google_bigquery_table" "table" {
  dataset_id          = var.bq_dataset_id
  table_id            = var.bq_table_id
  deletion_protection = false

  schema =  jsonencode(
    [
      { "name" : "dummy", "type" : "JSON", "mode" : "NULLABLE" },
      
      # Any column name containing BigQuery reserved words (e.g. x-forwarded-for, newbie-id),
      # 
      # After creating the column with `terraform apply`,
      # comment out the code and run `terraform apply` again to drop the column,
      # {"name" : "x-forwarded-for", "type" : "STRING", "mode" : "NULLABLE"},
    ]
  )
}

Debug Output

2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ REQUEST ]---------------------------------------
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: GET /bigquery/v2/projects/<project_id>/datasets/<dataset_id>/tables/reserved-word-test?alt=json&prettyPrint=false HTTP/1.1
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Host: bigquery.googleapis.com
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: User-Agent: google-api-go-client/0.5 Terraform/1.10.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/6.14.0
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Goog-Api-Client: gl-go/1.23.4 gdcl/0.206.0
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Accept-Encoding: gzip
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Google API Response Details:
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ RESPONSE ]--------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: HTTP/2.0 200 OK
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Etag: vE/w0CKk7jiIiZ2sF23+JA==
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "kind": "bigquery#table",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "etag": "vE/w0CKk7jiIiZ2sF23+JA==",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "id": "<project_id>:<dataset_id>.reserved-word-test",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/<project_id>/datasets/<dataset_id>/tables/reserved-word-test",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "tableReference": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "projectId": "<project_id>",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "datasetId": "<dataset_id>",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "tableId": "reserved-word-test"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "labels": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "goog-terraform-provisioned": "true",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "managedby": "terraform",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "env": "dev"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "schema": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "fields": [
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "name": "data",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "JSON",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "mode": "NULLABLE"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "name": "x-forwarded-for",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "STRING",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "mode": "NULLABLE"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ]
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "timePartitioning": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "DAY"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numLongTermBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numRows": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "creationTime": "1734574808432",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "lastModifiedTime": "1734574843702",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "TABLE",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "US",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numTotalLogicalBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numActiveLogicalBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numLongTermLogicalBytes": "0"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Stopping retries, last request was successful
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Stopping retries, last request was successful
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [INFO] Dropping columns in-place: ALTER TABLE <project_id>.<dataset_id>.reserved-word-test DROP COLUMN x-forwarded-for
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [INFO] Instantiating Google Cloud BigQuery client for path https://bigquery.googleapis.com/bigquery/v2/
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: starting RoundTrip retry loop
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: request attempt 0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: starting RoundTrip retry loop
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: request attempt 0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Google API Request Details:
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ REQUEST ]---------------------------------------
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: POST /bigquery/v2/projects/<project_id>/queries?alt=json&prettyPrint=false HTTP/1.1
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Host: bigquery.googleapis.com
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: User-Agent: google-api-go-client/0.5 Terraform/1.10.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/6.14.0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Length: 113
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Goog-Api-Client: gl-go/1.23.4 gdcl/0.206.0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Accept-Encoding: gzip
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "query": "ALTER TABLE <project_id>.<dataset_id>.reserved-word-test DROP COLUMN x-forwarded-for",
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "useLegacySql": false
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Google API Response Details:
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ RESPONSE ]--------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:40.087+0900 [ERROR] provider.terraform-provider-google_v6.14.0_x5: Response contains error diagnostic: tf_proto_version=5.7 tf_req_id=a4dd581e-e851-12cb-f67c-c62805218138 tf_resource_type=google_bigquery_table @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_summary="googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery" tf_provider_addr=registry.terraform.io/hashicorp/google tf_rpc=ApplyResourceChange diagnostic_detail="" diagnostic_severity=ERROR timestamp="2024-12-19T14:36:40.087+0900"
2024-12-19T14:36:40.087+0900 [DEBUG] State storage *remote.State declined to persist a state snapshot
2024-12-19T14:36:40.087+0900 [ERROR] vertex "module.log_stream_reserved_word_test.google_bigquery_table.log_table" error: googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery

Expected Behavior

DROP column x-forwarded-for

Actual Behavior

Error: googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery

Steps to reproduce

1. terraform apply to create any column with BigQuery flexible column name (e.g. x**-forwarded-proto, newbie-**id)
2. comment out the column definition
3. terraform apply to drop the column
4. Error

Important Factoids

1. The cause of this bug is due to not backquoting the table name when constructing the query

terraform-provider-google/google/services/bigquery/resource_bigquery_table.go

Lines 1977 to 1987 in 81584b6

	droppedColumns := []string{}
	for _, field := range oldTable.Schema.Fields {
	if !newTableFields[field.Name] {
	droppedColumns = append(droppedColumns, field.Name)
	}
	}
	if len(droppedColumns) > 0 {
	droppedColumnsString := strings.Join(droppedColumns, ", DROP COLUMN ")
	dropColumnsDDL := fmt.Sprintf("ALTER TABLE `%s.%s.%s` DROP COLUMN %s", tableReference.project, tableReference.datasetID, tableReference.tableID, droppedColumnsString)

2. I created a PR to backquoting the DROP column names

GoogleCloudPlatform/magic-modules#12626

References

BigQuery flexible column name
https://cloud.google.com/bigquery/docs/schemas#flexible-column-names

b/385364330

[shoumoji]

The test results revealed that the cause of this error is not a reserved word, but rather the use of flexible column name with a dash character.

Therefore, I will add tests and update of the issue and PR.
There are no changes to the code, as this issue can be resolved by backquoting the column names.

e.g.
No Error : the column name for ( reserved word "for" )
Error : the column name foo-bar ( no reserved word, but with flexibile column name "-" )