archive_file data source gets created during "terraform plan" vs "terraform apply" and also is not deleted during destroy

[jeffscottlevine]

Terraform CLI and Provider Versions

Terraform v1.4.6
on linux_amd64

• provider registry.terraform.io/hashicorp/archive v2.3.0
• provider registry.terraform.io/hashicorp/google v4.66.0
• provider registry.terraform.io/hashicorp/local v2.4.0
• provider registry.terraform.io/hashicorp/random v3.5.1

Terraform Configuration

terraform {
  required_providers {
    archive = {
      version = ">= 2.3.0"
    }
}

data "archive_file" "source_files" {
  type        = "zip"
  output_path = "${path.module}/src.zip"

  source {
    content  = file("${path.module}/requirements.txt")
    filename = "requirements.txt"
  }

  source {
    content  = file("${path.module}/main.py")
    filename = "main.py"
  }
}

Expected Behavior

I would expect that:

(1) The archive file would not be created during "terraform plan"
(2) The archive file would be created during "terraform apply"
(3) The archive file would be deleted during "terraform destroy"

Actual Behavior

(1) The archive file is created during "terraform plan"
(2) The archive file is not destroyed during "terraform destroy"

Steps to Reproduce

Terraform command: 1.4.6
archive provider: 2.3.0

To replicate

(1) Create some source files. I am using main.py and requirements.txt but they can be any files.

(2) Create a tf file that contains the following.

terraform {
required_providers {
archive = {
version = ">= 2.3.0"
}
}

data "archive_file" "source_files" {
type = "zip"
output_path = "${path.module}/src.zip"

source {
content = file("${path.module}/requirements.txt")
filename = "requirements.txt"
}

source {
content = file("${path.module}/main.py")
filename = "main.py"
}
}

(3) Run "terraform plan --out=plan.out"

You will see that src.zip has been created during "terraform plan" rather than waiting for "terraform apply"

(4) terraform apply

(5) terraform destroy

src.zip continues to exist

How much impact is this issue causing?

Low

Logs

No response

Additional Information

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[bendbennett]

Hi @jeffscottlevine 👋

It sounds like you might need to use the archive_file resource rather than the data source in this instance. Could you try using the resource and see if that behaves in the way that you are expecting?

[jeffscottlevine]

Hi @jeffscottlevine 👋

It sounds like you might need to use the archive_file resource rather than the data source in this instance. Could you try using the resource and see if that behaves in the way that you are expecting?

According to this page, the archive_file resource is deprecated and that we should use the data source.

[bendbennett]

Although the archive_file resource is marked as deprecated, there are currently no plans to remove it.

[raphaelschnaitl]

Can you remove the deprecation note? The datasource is no replacement for the archive_file resource.

[robertbrandso]

I very much agree with @Console32, the data source is no replacement for the archive_file resource.

One use case is when I need something to happen before the zip is created. When the zip is created during plan this is rather difficult to do.

[raphael-ratepay]

@bendbennett would it be possible to remove the deprecation note?

[Hronom]

Would it be possible to remove deprecation note?

There strong use case for it. In our case when we work with lambdas in AWS.
You can also check likes under comments above

[bendbennett]

We will discuss removal of the deprecation note from the archive resource within the team and provide feedback once a conclusion has been reached.

[bwhaley]

Can you explain the difference between the resource and the data source? The docs do not explain what the difference is.

It seems that the resource creates the archive during apply, but the data source creates it during plan. This resolved the problem I was having, so I hope that the resource isn't actually deprecated. It's confusing to suggest to use the resource if it says right at the top of the docs and during plan that the resource is deprecated and to use the data source instead.

[socketbox]

It looks as if the deprecation is proceeding, as I've just encountered this message in apply output (TF version 1.8.3; AWS provider 5.43.0):

│ Warning: Deprecated                                                                            
│                                                                                                
│   with module.lite.module.pbsorg.archive_file.lambda_archive,                                  
│   on pbsorg/resources/notifications.tf line 25, in resource "archive_file" "lambda_archive":   
│   25: resource "archive_file" "lambda_archive" {                                               
│                                                                                                
│ **NOTE**: This resource is deprecated, use data source instead.                                
│                                                                                                
│ (and 2 more similar warnings elsewhere)                                                        
╵                                                                                                

[novekm]

Any updates on this? I'm currently using the archive_file data source but as others have mentioned, the archive is created during plan instead of during apply, and such does not get deleted when running terraform destroy. I'm using it in a module that will released by AWS, so would prefer to not have the warning that @socketbox mentioned. @justinretzolk do you know of anyone who may be able to advise on this?

[nprime496]

Given the immense traction this issue has and the fact that nothing changed, it's likely that the behavior will never change. Can you just give us best practices to how we deal with this ?

I am using bitbucket and my plan and apply are different steps.

EDIT:

In my case, the solution was simple and I think that it is the generic solution for CI/CD pipelines.

Add your zipped file name as an artifact of your planning step.

Here is my example code for bitbucket :

pipelines:
  branches: 
    master:
      - step:
          name: Terraform plan
          artifacts:
            - terraform/main/.out/plan
            - terraform/main/.out/zipped-source.zip
          script:
            - sh ./.bin/deploy.sh plan
          condition:
            changesets:
              includePaths:
                - "terraform/main/**"

[SergiiKrupenko-cnic]

I use data.archive_file to and this causes a problem when dir for zip is deleted and will be recreated after terraform apply because it's a lambda layer. So for now I plan to change to resource archive_file or may be some one can advise better solution.

[EAR009]

I am also facing same issue where the terraform plan is successful but apply fails. Is there any concrete response to resolve this long pending issue?

[codymaust]

@bendbennett and the rest of the contributing team, please consider this pull request

#367

thanks!

[wayneworkman]

I wanted to provide my use case. I depend on Terraform outputting the source_code_hash for AWS Lambdas during Terraform plans. At work, we use these Terraform plans for our deployment approval processes and it’s crucial that the SHA256 Base64 encoded checksum of Lambda packages be present within Terraform plan output. This checksum within an approved Terraform plan is the easiest way to actually validate that the Lambda package currently deployed in AWS matches the approved Terraform plans.

[EAR009]

Yes it will work for very few use case like yours. In general, the replacement is not working.

I wanted to provide my use case. I depend on Terraform outputting the source_code_hash for AWS Lambdas during Terraform plans. At work, we use these Terraform plans for our deployment approval processes and it’s crucial that the SHA256 Base64 encoded checksum of Lambda packages be present within Terraform plan output. This checksum within an approved Terraform plan is the easiest way to actually validate that the Lambda package currently deployed in AWS matches the approved Terraform plans.

[SBGoods]

Hi everyone 👋🏾
Thank you for your patience on this issue. After internal discussion, the maintainer team has decided to un-deprecate the archive_file resource. As far as next steps go, the current plan is:

• Review and merge @codymaust's PR feat: remove archive_file resource deprecated status #367
• Add a guide to the provider documentation to help users determine when to use the resource vs the data source
• Release v2.7.0 by December 6th

[SBGoods]

v2.7.0 has just been released with the removal of the archive_file resource deprecation. I wasn't able to get the guide for the provider documentation in but I didn't want that to block the release. Hopefully, we will get the documentation in with a patch after the holidays.