Configuration for X still contains unknown values during apply

[alex-goncharov]

Terraform Version

v1.10.4

Terraform Configuration Files

resource "random_pet" "server" {
  keepers = {
    test = format("%s", null)
  }
}

Debug Output

2025-01-10T15:32:29.784+0100 [INFO]  Terraform version: 1.10.4
2025-01-10T15:32:29.784+0100 [DEBUG] using github.com/hashicorp/go-tfe v1.70.0
2025-01-10T15:32:29.784+0100 [DEBUG] using github.com/hashicorp/hcl/v2 v2.23.0
2025-01-10T15:32:29.784+0100 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2025-01-10T15:32:29.784+0100 [DEBUG] using github.com/zclconf/go-cty v1.16.0
2025-01-10T15:32:29.784+0100 [INFO]  Go runtime version: go1.23.3
2025-01-10T15:32:29.784+0100 [INFO]  CLI args: []string{"terraform", "apply", "--parallelism=50"}
2025-01-10T15:32:29.784+0100 [DEBUG] Attempting to open CLI config file: /Users/shura/.terraformrc
2025-01-10T15:32:29.784+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2025-01-10T15:32:29.784+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2025-01-10T15:32:29.784+0100 [DEBUG] ignoring non-existing provider search directory /Users/shura/.terraform.d/plugins
2025-01-10T15:32:29.784+0100 [DEBUG] ignoring non-existing provider search directory /Users/shura/Library/Application Support/io.terraform/plugins
2025-01-10T15:32:29.784+0100 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2025-01-10T15:32:29.785+0100 [INFO]  CLI command args: []string{"apply", "--parallelism=50"}
2025-01-10T15:32:29.792+0100 [DEBUG] checking for provisioner in "."
2025-01-10T15:32:29.792+0100 [DEBUG] checking for provisioner in "/Users/shura/bin"
2025-01-10T15:32:29.792+0100 [INFO]  backend/local: starting Apply operation
2025-01-10T15:32:29.793+0100 [DEBUG] created provider logger: level=debug
2025-01-10T15:32:29.793+0100 [INFO]  provider: configuring client automatic mTLS
2025-01-10T15:32:29.798+0100 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5"]
2025-01-10T15:32:29.800+0100 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 pid=20512
2025-01-10T15:32:29.800+0100 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5
2025-01-10T15:32:29.804+0100 [INFO]  provider.terraform-provider-random_v3.6.3_x5: configuring server automatic mTLS: timestamp="2025-01-10T15:32:29.804+0100"
2025-01-10T15:32:29.810+0100 [DEBUG] provider: using plugin: version=5
2025-01-10T15:32:29.810+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: plugin address: address=/var/folders/qb/yn_ltqfs3cndqf6ky450rdw80000gn/T/plugin3575132074 network=unix timestamp="2025-01-10T15:32:29.809+0100"
2025-01-10T15:32:29.815+0100 [DEBUG] No provider meta schema returned
2025-01-10T15:32:29.816+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-01-10T15:32:29.816+0100 [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 id=20512
2025-01-10T15:32:29.816+0100 [DEBUG] provider: plugin exited
2025-01-10T15:32:29.816+0100 [DEBUG] Building and walking validate graph
2025-01-10T15:32:29.816+0100 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/random"], implied first by random_pet.server
2025-01-10T15:32:29.816+0100 [DEBUG] ProviderTransformer: "random_pet.server" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:29.816+0100 [DEBUG] ReferenceTransformer: "random_pet.server" references: []
2025-01-10T15:32:29.816+0100 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/random\"]" references: []
2025-01-10T15:32:29.817+0100 [DEBUG] Starting graph walk: walkValidate
2025-01-10T15:32:29.817+0100 [DEBUG] created provider logger: level=debug
2025-01-10T15:32:29.817+0100 [INFO]  provider: configuring client automatic mTLS
2025-01-10T15:32:29.819+0100 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5"]
2025-01-10T15:32:29.820+0100 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 pid=20513
2025-01-10T15:32:29.820+0100 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5
2025-01-10T15:32:29.824+0100 [INFO]  provider.terraform-provider-random_v3.6.3_x5: configuring server automatic mTLS: timestamp="2025-01-10T15:32:29.824+0100"
2025-01-10T15:32:29.830+0100 [DEBUG] provider: using plugin: version=5
2025-01-10T15:32:29.830+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: plugin address: address=/var/folders/qb/yn_ltqfs3cndqf6ky450rdw80000gn/T/plugin1149687616 network=unix timestamp="2025-01-10T15:32:29.830+0100"
2025-01-10T15:32:29.835+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-01-10T15:32:29.835+0100 [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 id=20513
2025-01-10T15:32:29.836+0100 [DEBUG] provider: plugin exited
2025-01-10T15:32:29.836+0100 [INFO]  backend/local: apply calling Plan
2025-01-10T15:32:29.836+0100 [DEBUG] Building and walking plan graph for NormalMode
2025-01-10T15:32:29.836+0100 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/random"], implied first by random_pet.server (expand)
2025-01-10T15:32:29.836+0100 [DEBUG] ProviderTransformer: "random_pet.server (expand)" (*terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:29.836+0100 [DEBUG] ReferenceTransformer: "random_pet.server (expand)" references: []
2025-01-10T15:32:29.836+0100 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/random\"]" references: []
2025-01-10T15:32:29.836+0100 [DEBUG] Starting graph walk: walkPlan
2025-01-10T15:32:29.836+0100 [DEBUG] created provider logger: level=debug
2025-01-10T15:32:29.836+0100 [INFO]  provider: configuring client automatic mTLS
2025-01-10T15:32:29.838+0100 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5"]
2025-01-10T15:32:29.839+0100 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 pid=20514
2025-01-10T15:32:29.839+0100 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5
2025-01-10T15:32:29.843+0100 [INFO]  provider.terraform-provider-random_v3.6.3_x5: configuring server automatic mTLS: timestamp="2025-01-10T15:32:29.843+0100"
2025-01-10T15:32:29.849+0100 [DEBUG] provider: using plugin: version=5
2025-01-10T15:32:29.849+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: plugin address: address=/var/folders/qb/yn_ltqfs3cndqf6ky450rdw80000gn/T/plugin2279920968 network=unix timestamp="2025-01-10T15:32:29.849+0100"
2025-01-10T15:32:29.854+0100 [DEBUG] Resource instance state not found for node "random_pet.server", instance random_pet.server
2025-01-10T15:32:29.854+0100 [DEBUG] ReferenceTransformer: "random_pet.server" references: []
2025-01-10T15:32:29.854+0100 [DEBUG] refresh: random_pet.server: no state, so not refreshing
2025-01-10T15:32:29.855+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: Marking Computed attributes with null configuration values as unknown (known after apply) in the plan to prevent potential Terraform errors: @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=b6187ced-8f61-13f4-61fc-76b634b6966b @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_planresourcechange.go:217 tf_resource_type=random_pet tf_rpc=PlanResourceChange timestamp="2025-01-10T15:32:29.855+0100"
2025-01-10T15:32:29.855+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: marking computed attribute that is null in the config as unknown: tf_provider_addr=registry.terraform.io/hashicorp/random @module=sdk.framework tf_attribute_path="AttributeName(\"id\")" tf_req_id=b6187ced-8f61-13f4-61fc-76b634b6966b tf_resource_type=random_pet tf_rpc=PlanResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_planresourcechange.go:467 timestamp="2025-01-10T15:32:29.855+0100"
2025-01-10T15:32:29.856+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-01-10T15:32:29.856+0100 [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 id=20514
2025-01-10T15:32:29.856+0100 [DEBUG] provider: plugin exited
2025-01-10T15:32:29.856+0100 [DEBUG] building apply graph to check for errors
2025-01-10T15:32:29.856+0100 [DEBUG] Resource state not found for node "random_pet.server", instance random_pet.server
2025-01-10T15:32:29.856+0100 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/random"], implied first by random_pet.server (expand)
2025-01-10T15:32:29.856+0100 [DEBUG] ProviderTransformer: "random_pet.server (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:29.856+0100 [DEBUG] ProviderTransformer: "random_pet.server" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:29.857+0100 [DEBUG] ReferenceTransformer: "random_pet.server (expand)" references: []
2025-01-10T15:32:29.857+0100 [DEBUG] ReferenceTransformer: "random_pet.server" references: []
2025-01-10T15:32:29.857+0100 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/random\"]" references: []

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # random_pet.server will be created
  + resource "random_pet" "server" {
      + id        = (known after apply)
      + keepers   = {
          + "test" = (known after apply)
        }
      + length    = 2
      + separator = "-"
    }

Plan: 1 to add, 0 to change, 0 to destroy.
2025-01-10T15:32:29.857+0100 [DEBUG] command: asking for input: "\nDo you want to perform these actions?"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

2025-01-10T15:32:32.167+0100 [INFO]  backend/local: apply calling Apply
2025-01-10T15:32:32.167+0100 [DEBUG] Building and walking apply graph for NormalMode plan
2025-01-10T15:32:32.167+0100 [DEBUG] Resource state not found for node "random_pet.server", instance random_pet.server
2025-01-10T15:32:32.167+0100 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/random"], implied first by random_pet.server (expand)
2025-01-10T15:32:32.167+0100 [DEBUG] ProviderTransformer: "random_pet.server (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:32.167+0100 [DEBUG] ProviderTransformer: "random_pet.server" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/hashicorp/random"]
2025-01-10T15:32:32.168+0100 [DEBUG] ReferenceTransformer: "random_pet.server (expand)" references: []
2025-01-10T15:32:32.168+0100 [DEBUG] ReferenceTransformer: "random_pet.server" references: []
2025-01-10T15:32:32.168+0100 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/random\"]" references: []
2025-01-10T15:32:32.168+0100 [DEBUG] Starting graph walk: walkApply
2025-01-10T15:32:32.169+0100 [DEBUG] created provider logger: level=debug
2025-01-10T15:32:32.169+0100 [INFO]  provider: configuring client automatic mTLS
2025-01-10T15:32:32.175+0100 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5"]
2025-01-10T15:32:32.177+0100 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 pid=20515
2025-01-10T15:32:32.177+0100 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5
2025-01-10T15:32:32.187+0100 [INFO]  provider.terraform-provider-random_v3.6.3_x5: configuring server automatic mTLS: timestamp="2025-01-10T15:32:32.186+0100"
2025-01-10T15:32:32.196+0100 [DEBUG] provider: using plugin: version=5
2025-01-10T15:32:32.196+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: plugin address: address=/var/folders/qb/yn_ltqfs3cndqf6ky450rdw80000gn/T/plugin220076341 network=unix timestamp="2025-01-10T15:32:32.196+0100"
2025-01-10T15:32:32.202+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: Marking Computed attributes with null configuration values as unknown (known after apply) in the plan to prevent potential Terraform errors: @module=sdk.framework tf_req_id=b4fe47e8-1ab7-426f-79f8-a972424b9199 tf_provider_addr=registry.terraform.io/hashicorp/random tf_resource_type=random_pet tf_rpc=PlanResourceChange @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_planresourcechange.go:217 timestamp="2025-01-10T15:32:32.202+0100"
2025-01-10T15:32:32.202+0100 [DEBUG] provider.terraform-provider-random_v3.6.3_x5: marking computed attribute that is null in the config as unknown: tf_resource_type=random_pet @module=sdk.framework tf_attribute_path="AttributeName(\"id\")" tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=b4fe47e8-1ab7-426f-79f8-a972424b9199 @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_planresourcechange.go:467 tf_rpc=PlanResourceChange timestamp="2025-01-10T15:32:32.202+0100"
random_pet.server: Creating...
2025-01-10T15:32:32.202+0100 [INFO]  Starting apply for random_pet.server
2025-01-10T15:32:32.208+0100 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2025-01-10T15:32:32.208+0100 [ERROR] vertex "random_pet.server" error: Configuration contains unknown value
╷
│ Error: Configuration contains unknown value
│
│ configuration for random_pet.server still contains unknown values during apply (this is a bug in Terraform; please report it!)
│ The following paths in the resource configuration are unknown:
│ cty.Path{cty.GetAttrStep{Name:"keepers"}, cty.IndexStep{Key:cty.StringVal("test")}}
╵
2025-01-10T15:32:32.215+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-01-10T15:32:32.215+0100 [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/random/3.6.3/darwin_arm64/terraform-provider-random_v3.6.3_x5 id=20515
2025-01-10T15:32:32.215+0100 [DEBUG] provider: plugin exited

Expected Behavior

Terraform should give the same error in both cases.

$ terraform console
> format("%s", null)
(known after apply)
> format("%s", tostring(null))
╷
│ Error: Error in function call
│
│   on <console-input> line 1:
│   (source code not available)
│
│ Call to function "format" failed: unsupported value for "%s" at 0: null value cannot be formatted.

Actual Behavior

$ terraform console
> format("%s", null)
(known after apply)

Steps to Reproduce

run in terraform console

> format("%s", null)
(known after apply)
> format("%s", tostring(null))
╷
│ Error: Error in function call
│
│   on <console-input> line 1:
│   (source code not available)
│
│ Call to function "format" failed: unsupported value for "%s" at 0: null value cannot be formatted.
╵


### Additional Context

_No response_

### References

_No response_

### Generative AI / LLM assisted development?

no

[liamcervante]

This behaviour stems from the implementation of the go-cty format function: https://github.com/zclconf/go-cty/blob/main/cty/function/stdlib/format.go#L38.

I'm not sure exactly why it would be returning unknown if there are null values included, but I'll leave the link to the function in case anyone else picks this up before I get back to it.

[jbardin]

The unknown value isn't returned directly from the format implementation, the format function implementation does not handle dynamic types, so the function call returns early before even considering Impl. Because Funcion.Call is a generic wrapper for all functions, when faced with a dynamic type and a function that doesn't handle dynamic types, it automatically returns unknown until that type is resolved.

A null literal is special however, in that it is a known value with a dynamic type, which requires special casing in a lot of code paths. The upstream library could probably extend the Call method to detect and convert null to the correct type in some cases, but the problem is that format can accept any type of vararg, so there's no way for Call to know what type to convert it to. Modifying the format function to accept dynamic types and deal with null also doesn't work, because null is valid for some formats:

> format("%v", tostring(null))
"null"

I think the only way to theoretically solve this generally for most cases would be to have format inspect the format string verbs, and use those to declare what types it expects so that the arguments can be converted by the caller, but that is not currently possible and would be a large change in the upstream implementation of functions, which must declare all arguments statically.

Since this isn't completely fixable, and more or less constitutes an error in the configuration, was there a problem you were attempting to solve which didn't make use of a null literal?

[alex-goncharov]

Thank you for such an elaborate explanation. The use-case is trivial: read the config from yaml and do whatever. Some fields in the configs could be omitted which would produce null. Nothing that value checks could not solve.

This report is merely about the behaviour of the format function that essentially converts null to an unknown (even when %v is used).

[jbardin]

Thanks for the info @alex-goncharov. Yes, untyped null values from dynamic data like json or yaml can be a pain point, but we do have the ability to convert them inline with tostring to ensure we get an early error when possible. It's still a bit unfortunate that this doesn't return an error until apply where unknown values are not going to be allowed in a resource config, so we can keep this open in case the upstream package does manage to implement a check for untyped null values being applied to specific formatting verbs.