From 3d8e60f0004bb456cbf45ca187893ba664aa0403 Mon Sep 17 00:00:00 2001 From: Ben Zhang Date: Mon, 16 Sep 2024 23:45:47 -0700 Subject: [PATCH] Use intermediate stage to avoid invalidating Docker cache when copying from build context (#3179) ## Description In https://github.com/WATonomous/infra-config/pull/3176, we documented a manual procedure to fix cache invalidation issues for the provisioner container. However, there's a workaround: use a lightweight intermediate stage to serve as the courier between the context. The workaround comes from [here](https://github.com/devcontainers/cli/issues/153#issuecomment-1278293424). This PR implements the workaround and updates the docs to reflect this. **How it works**: Docker computes hashes for the input to each layer to determine whether that layer can use the cache. Previously, the permission bits on the computer that built the cache and my personal environment were different. This resulted in the hash being different, thus invalidating the cache. `COPY --chmod` doesn't help either because the hash is computed [without taking into account the parameters](https://github.com/docker/buildx/issues/1311). In this PR, we implement a workaround, where we use an extremely lightweight stage (`FROM scratch` with only `COPY` statements) to import files and set permissions from the build context. We don't care whether this stage runs or gets cached, because it's very lightweight. At build-time, if we are lucky that our system has the same permissions as the cache build environment, then this stage will be cached. If not, this stage will run. Regardless, the output of this stage will remain the same if the file is unchanged. This property allows Docker to continue subsequent steps with cache. Tested to work with the existing cache when using permission 644. This PR changes the permissions to 400 to be a bit more strict. Resolves https://github.com/WATonomous/infra-config/issues/3178 ## Checklist - [x] I have read and understood the [WATcloud Guidelines](https://cloud.watonomous.ca/docs/community-docs/watcloud/guidelines) - [x] I have performed a self-review of my code --- .../watcloud/development-manual.mdx | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/pages/docs/community-docs/watcloud/development-manual.mdx b/pages/docs/community-docs/watcloud/development-manual.mdx index 6f5e35d..f4e7ad4 100644 --- a/pages/docs/community-docs/watcloud/development-manual.mdx +++ b/pages/docs/community-docs/watcloud/development-manual.mdx @@ -156,26 +156,21 @@ The cache is used to speed up both [CI](https://github.com/WATonomous/infra-conf and [local development](https://github.com/WATonomous/infra-config/pull/3175). The cache is automatically used. Without any changes, the following command should complete quickly[^build-time-with-cache] -and show that every stage is loaded from cache: +and show that almost every stage is loaded from cache: ```bash copy docker compose build provisioner ``` -On some setups, `COPY` and `ADD` commands may not be cached. -For example, when there is a custom `umask` set when git checks out files, the permissions of the files may not match the cache. -This is due to how git and Docker handle file permissions differently[^git-docker-permission-difference]. -To fix this, we can run the following to reset the non-executable bits of the files to the default: - -```bash copy -git ls-files | xargs -I '{}' chmod u+rw,go+r-w {} -``` +Previously, there was a [cache invalidation issue](https://github.com/WATonomous/infra-config/pull/3176) when the files in +the Docker context don't have the same permissions as the cache[^git-docker-permission-difference]. +However, this issue has been fixed using a [workaround](https://github.com/WATonomous/infra-config/pull/3179). [^caching-details]: [Here](https://github.com/WATonomous/infra-config/blob/121af9af1dbe78e187670163545fa6537a26757f/.github/workflows/push-images.yml#L62) is where we push the cache, and [here](https://github.com/WATonomous/infra-config/blob/121af9af1dbe78e187670163545fa6537a26757f/docker-compose.yml#L5-L6) is where we use it. The cache lives [here](https://github.com/WATonomous/infra-config/pkgs/container/infra-config). [^build-time-with-cache]: At the time of writing (2024-09-16), the build time with cache is about 30 seconds on a single core (Docker immediately recognizes that every layer can be cached, and downloads the image from the cache). The build time without cache is about 3 minutes and 40 seconds on 8 cores. -[^git-docker-permission-difference]: Git [only preserves the executable bit](https://stackoverflow.com/a/3211396/4527337) of files, and uses the umask ([defaults to `022`](https://man7.org/linux/man-pages/man2/umask.2.html) on most systems) to determine the permissions of the files it creates. Docker, on the other hand, [uses all permission bits](https://docs.docker.com/engine/reference/builder/#copy) when using `COPY` or `ADD`. +[^git-docker-permission-difference]: Git and Docker handle file permissions differently. Git [only preserves the executable bit](https://stackoverflow.com/a/3211396/4527337) of files, and uses the umask ([defaults to `022`](https://man7.org/linux/man-pages/man2/umask.2.html) on most systems) to determine the permissions of the files it creates. Docker, on the other hand, [uses all permission bits](https://docs.docker.com/engine/reference/builder/#copy) when using `COPY` or `ADD`. ### Port forwarding