fix(terraform): Failed to expand block

[ZsoltPath]

Description

I have actually to similar issues around expansion.

Desired Behavior

1. Issue

resource "aws_route" "private_tgw_route" {
  for_each               = toset(module.routing.private_route_table_ids)
  route_table_id         = each.value
  transit_gateway_id     = aws_ec2_transit_gateway.hub.id
  destination_cidr_block = "10.0.0.0/8"
}

Where module.routing.private_route_table_ids is a list of rout table IDs as the name suggests.
For example in my current environment TF console shows

module.routing.private_route_table_ids
[
  "rtb-09e907d76055b5be8",
  "rtb-0d695e220df86a65a",
  "rtb-044da920181a54f41",
]

2. Issue

First the module call and then the actual relevant resource definition

module "cleanup" {
  source = "[email protected]:rbs-path/terraform-modules.git//modules/compute/aws-lambda?ref=v3.8.0"
  providers = {
    aws       = aws
    aws.local = aws
  }

  function_name                   = "lambda_cleanup"
  description                     = "Delete old, unused Lambda versions"
  handler                         = "lambda_cleanup.handler"
  runtime                         = "python3.12"
  timeout                         = 300
  add_dd_layers                   = true
  datadog_extension_layer_version = var.datadog_extension_layer_version
  datadog_python_layer_version    = var.datadog_python_layer_version

  content  = file("${path.module}/lambda/lambda_cleanup/lambda_cleanup.py")
  filename = "lambda_cleanup.py"

  policy = data.aws_iam_policy_document.lambda_cleanup

  environment = {
    variables = {
      DD_SERVICE = "lambda_cleanup"
    }
  }
}

resource "aws_lambda_function" "lambda" {

  function_name                  = var.function_name
  description                    = var.description
  role                           = aws_iam_role.lambda.arn
  handler                        = var.add_dd_layers && !var.lambda_at_edge ? local.dd_layer_versions[local.runtime].handler : var.handler
  memory_size                    = var.memory_size
  reserved_concurrent_executions = var.reserved_concurrent_executions
  runtime                        = var.runtime
  layers                         = concat(var.layers, local.dd_layers)
  timeout                        = var.timeout
  publish                        = local.publish

  architectures = [var.architectures]

  ephemeral_storage {
    size = var.ephemeral_storage_size # Min 512 MB and the Max 10240 MB
  }

  filename   = var.content == null ? data.external.built[0].result.filename : data.archive_file.lambda[0].output_path
  depends_on = [null_resource.archive]

  dynamic "environment" {
    for_each = (var.environment == null && !var.add_dd_layers) || var.lambda_at_edge ? [] : [local.environment]
    content {
      variables = environment.value
    }
  }
}

Actual Behavior

1. Issue

Getting: ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="aws_route.private_tgw_route" value="cty.NilVal"

2. Issue

Getting: ERROR [terraform evaluator] Failed to expand dynamic block. block="module.cleanup.aws_lambda_function.lambda" err="1 error occurred:\n\t* invalid for-each in aws_lambda_function.lambda.dynamic.environment block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
Interestingly twice.

Reproduction Steps

1. `trivy config --tf-vars environments/dev.tfvars ./`

Target

Filesystem

Scanner

Misconfiguration

Output Format

None

Mode

Standalone

Debug Output

There's nothing more in there. And it's a 153K file.

Operating System

MacOS Sequia

Version

v0.57.0

Checklist

• Run trivy clean --all
• Read the troubleshooting

[simar7]

hi @ZsoltPath

For the first issue, can you please supply a tfvars file so we can test it out? It looks like Trivy is unable to determine the value for the for_each block.

For your second issue: What does for_each = (var.environment == null && !var.add_dd_layers) || var.lambda_at_edge ? [] : [local.environment] evaluate to? It would depend on your tfvars, which you haven't shared.

[ZsoltPath]

hi @ZsoltPath

For the first issue, can you please supply a tfvars file so we can test it out? It looks like Trivy is unable to determine the value for the for_each block.

For your second issue: What does for_each = (var.environment == null && !var.add_dd_layers) || var.lambda_at_edge ? [] : [local.environment] evaluate to? It would depend on your tfvars, which you haven't shared.

Hi @simar7

The first one doesn't use any tfvars. The module creates route tables for subnets and outputs their IDs.

For the second one:

lambda_at_edge = false

locals{
  environment = tomap(
    merge(var.environment != null ? var.environment.variables : {}, var.add_dd_layers && !var.lambda_at_edge ? {
      DD_LAMBDA_HANDLER = var.handler
      DD_SITE           = "datadoghq.eu"
      DD_API_KEY        = data.aws_secretsmanager_secret_version.dd_api_key[0].secret_string
    } : {})
  )
}

I know it's quite complicated. But it works perfectly in TF.

Maybe it'd be easier for us to evaluate the TF plan instead of the code.

[nikpivkin]

Hi @ZsoltPath !

1. What does the private_route_table_ids output refer to, is it a resource or a data block?

2. An error occurs when trying to evaluate the value of the DD_API_KEY key. This happens because the schema of this data block is unknown to us. In Terraform, this works because it loads the provider (in this case AWS). Since the map contains an unknown value, the function is not applied and simply returns this unknown value, which is treated as null. This attribute is not used in the checks so you can continue to scan the configuration rather than Plan.

[ZsoltPath]

Hi @ZsoltPath !

1. What does the `private_route_table_ids` output refer to, is it a resource or a data block?

2. An error occurs when trying to evaluate the value of the `DD_API_KEY` key. This happens because the schema of this data block is unknown to us. In Terraform, this works because it loads the provider (in this case AWS). Since the map contains an unknown value, the function is not applied and simply returns this unknown value, which is treated as null. This attribute is not used in the checks so you can continue to scan the configuration rather than Plan.

Hi @nikpivkin !

1. The output comes from this code:

module "routing" {
  source = "..."

  vpc_id             = module.vpc.vpc_id
  private_subnet_ids = module.private_subnets.subnet_ids
}

resource "aws_route_table" "private" {
  vpc_id = var.vpc_id
  count  = length(var.private_subnet_ids)
}

output "private_route_table_ids" {
  value = aws_route_table.private.*.id
}

2. I understand that it isn't checked but the Trivy process still fails which terminates my GitHub workflow.
   To prevent that it should be either a Warning (not Error) or possible to switch off by a parameter.

[nikpivkin]

The error is simply logged and Trivy does not exit with code 1. Why does your workflow terminate?

[ZsoltPath]

The error is simply logged and Trivy does not exit with code 1. Why does your workflow terminate?

Uhhh, sorry! My bad then.