AWS Lambda archives are detected as different based on the host computer operating system

[64kramsystem]

Current Terraform Version

0.11.7

Use-cases

The devops team uses different operating systems. When the configurations are planned/applied on machines with different O/Ss, Terraform detects lambda functions as changed, even if they aren't, due to differences in the compression libraries used.

Attempted Solutions

We've tried a few attempts, but none of them is conceptually sound or acceptably clean. The underlying workflow is assumed to be:

1. Terraform creates the deployment package (zip or jar)
2. Terraform computes a hash
3. AWS returns the hash of deployment package
4. Terraform compares them

currently, there is no way to work this around, because the file created at step 1. is bound to be different depending on the operating system library.

Proposal

Attempting to use off-the-shelf alternative archivers presents the same problems, as all the archivers have variable metadata (even tar).

The best (only) solution I can think of is to add a custom compression module that creates zip files which are exactly the same on each platform - this can be accomplished by using fixed metadata and no compression (the files are small anyway, so archive compression makes virtually no difference).

I'm not familiar with Golang compression libraries, however, the scope of the archiver logic is conceptually small/reduced - any complexity or odd edge cases, eg. symlinks, could be rejected, as they wouldn't be meaningful in a AWS Lambda context.
Additionally, this could be an alternate archival backend. Those who want full ZIP functionality, would use the standard backend.

[quixand]

is it possible you are seeing the permission issue? see hashicorp/terraform-provider-archive#34

[64kramsystem]

is it possible you are seeing the permission issue? see terraform-providers/terraform-provider-archive#34

That's part of it, yes. There are multiple causes for a zip archive with the same data to be binary different (compression method, modification time, permissions).

We've internally solved the problem by adding an option to the archive provider: https://github.com/saveriomiroddi/terraform-provider-archive-dev/tree/sav-normalize_files_metadata_option

We plan to open a PR, hoping that this solution will be accepted.

[64kramsystem]

Closing this, since it's a duplicate (and the terraform-provider-archive repo is the appropriate location).

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.