Merge/Extend/Include multiple compose files to avoid redundancy

[yzx9]

Currently, we have three Docker Compose files that share most of their configurations. Docker offers options like Merge, Extend, and Include (2.20.3) to efficiently manage multiple Compose files. By using these methods, we can reduce redundancy and ensure that shared configurations are maintained centrally, while still allowing for environment-specific customizations.

Key Outcomes:

• Reduced redundancy, making maintenance less burdensome
• An additional file for users, but with the advantage of clearer separation of concerns

---

Difference between docker-compose.certbot.yml and docker-compose.yml

› diff docker-compose.certbot.yml docker-compose.yml 
12,13d11
<       simple-certbot:
<         condition: service_started
16c14
<       - 443:443
---
>       - 80:80
20d17
<       - simple-certbot
54,55c51,53
<       SHARELATEX_SECURE_COOKIE: "true"
<       SHARELATEX_BEHIND_PROXY: "true"
---
>       # Uncomment the following line to enable secure cookies if you are using SSL
>       # SHARELATEX_SECURE_COOKIE: "true"
>       # SHARELATEX_BEHIND_PROXY: "true"
157,174d154
< 
<   simple-certbot:
<     restart: always
<     image: certbot/certbot
<     container_name: simple-certbot
<     ports:
<       - 80:80
<     volumes:
<       - ${MYDATA}/letsencrypt:/etc/letsencrypt
<     # a bit hacky but this docker image uses very little disk-space
<     # best practices for ssl and nginx are set in the ldap-overleaf-sl Dockerfile
<     entrypoint:
<       - "/bin/sh"
<       - -c
<       - |
<         trap exit TERM;\
<         certbot certonly --standalone -d ${MYDOMAIN} --agree-tos -m ${MYMAIL} -n ; \
<         while :; do certbot renew; sleep 240h & wait $${!}; done;

Difference between docker-compose.traefik.yml and docker-compose.yml

› diff docker-compose.traefik.yml docker-compose.yml 
3,60d2
<   traefik:
<     image: traefik:latest
<     container_name: traefik
<     restart: unless-stopped
<     security_opt:
<       - no-new-privileges:true
<     networks:
<       - web
<     ports:
<       - 80:80
<       - 443:443
<       - 8443:8443
<       # - 8080:8080
<       # - 27017:27017
<     volumes:
<       - ${MYDATA}/letsencrypt:/letsencrypt
<       - /etc/localtime:/etc/localtime:ro
<       - /var/run/docker.sock:/var/run/docker.sock:ro
<       - ./traefik/dynamic_conf.yml:/dynamic_conf.yml
<       - ./traefik/users.htpasswd:/users.htpasswd
<     command:
<       - "--api=true"
<       - "--api.dashboard=true"
<       #- "--api.insecure=true" # provides the dashboard on http://IPADRESS:8080
<       - "--providers.docker=true"
<       - "--ping"
<       - "--providers.docker.network=web"
<       - "--providers.docker.exposedbydefault=false"
<       - "--providers.file.filename=/dynamic_conf.yml"
<       - "--entrypoints.web.address=:80"
<       - "--entrypoints.web-secure.address=:443"
<       - "--entrypoints.web-admin.address=:8443"
<       - "--certificatesresolvers.myhttpchallenge.acme.httpchallenge=true"
<       - "--certificatesresolvers.myhttpchallenge.acme.httpchallenge.entrypoint=web"
<       - "--certificatesresolvers.myhttpchallenge.acme.email=${MYMAIL}"
<       - "--certificatesresolvers.myhttpchallenge.acme.storage=/letsencrypt/acme.json"
<       - "--entrypoints.mongo.address=:27017"
<       #- --certificatesresolvers.myhttpchallenge.acme.caserver=https://acme-v02.api.letsencrypt.org/directory
<     labels:
<       - "traefik.enable=true"
<       # To Fix enable dashboard on port 8443
<       - "traefik.http.routers.dashboard.entrypoints=web-admin"
<       - "traefik.http.routers.dashboard.rule=Host(`${MYDOMAIN}`)"
<       # - "traefik.http.routers.dashboard.rule=Host(`traefik.${MYDOMAIN}`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
<       - "traefik.http.routers.dashboard.tls=true"
<       - "traefik.http.routers.dashboard.middlewares=auth"
<       - "traefik.http.middlewares.auth.basicauth.usersfile=/users.htpasswd"
<       - "traefik.http.routers.dashboard.service=api@internal"
<       - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
<       - "traefik.http.routers.proxy-https.entrypoints=web-secure"
<       - "traefik.http.routers.proxy-https.rule=Host(`${MYDOMAIN}`)"
< 
<   logging:
<     driver: "json-file"
<     options:
<       max-size: "10m"
<       max-file: "1"
< 
63c5,6
<     image: ldap-overleaf-sl:latest
---
>     image: ldap-overleaf-sl
>     container_name: ldap-overleaf-sl
69,72d11
<       traefik:
<         condition: service_started
<       #simple-certbot:
<       #    condition: service_started
74,78c13,14
<     networks:
<       - web
<     expose:
<       - 80
<       - 443
---
>     ports:
>       - 80:80
84,107c20,21
<       - ${MYDATA}/letsencrypt:/etc/letsencrypt:ro
<       # - ${MYDATA}/letsencrypt/live/${MYDOMAIN}/:/etc/letsencrypt/certs/domain
<     labels:
<       - "traefik.enable=true"
<       # global redirect to https
<       - "traefik.http.routers.http-catchall.rule=hostregexp(`${MYDOMAIN}`)"
<       - "traefik.http.routers.http-catchall.entrypoints=web"
<       - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
<       - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
<       # handle https traffic
<       - "traefik.http.routers.sharel-secured.rule=Host(`${MYDOMAIN}`)"
<       - "traefik.http.routers.sharel-secured.tls=true"
<       - "traefik.http.routers.sharel-secured.tls.certresolver=myhttpchallenge"
<       - "traefik.http.routers.sharel-secured.entrypoints=web-secure"
<       - "traefik.http.middlewares.sharel-secured.forwardauth.trustForwardHeader=true"
<       # Docker loadbalance
<       - "traefik.http.services.sharel.loadbalancer.server.port=80"
<       - "traefik.http.services.sharel.loadbalancer.server.scheme=http"
<       - "traefik.http.services.sharel.loadbalancer.sticky.cookie=true"
<       - "traefik.http.services.sharel.loadbalancer.sticky.cookie.name=io"
<       - "traefik.http.services.sharel.loadbalancer.sticky.cookie.httponly=true"
<       - "traefik.http.services.sharel.loadbalancer.sticky.cookie.secure=true"
<       - "traefik.http.services.sharel.loadbalancer.sticky.cookie.samesite=io"
< 
---
>       - ${MYDATA}/letsencrypt:/etc/letsencrypt
>       - ${MYDATA}/letsencrypt/live/${MYDOMAIN}/:/etc/letsencrypt/certs/domain
117a32,33
>       # SHARELATEX_EMAIL_AWS_SES_ACCESS_KEY_ID:
>       # SHARELATEX_EMAIL_AWS_SES_SECRET_KEY:
135,136c51,53
<       SHARELATEX_SECURE_COOKIE: "true"
<       SHARELATEX_BEHIND_PROXY: "true"
---
>       # Uncomment the following line to enable secure cookies if you are using SSL
>       # SHARELATEX_SECURE_COOKIE: "true"
>       # SHARELATEX_BEHIND_PROXY: "true"
145c62
<       # Tries to bind with login-user (as uid) to LDAP_BINDDN
---
>       # Tries directly to bind with the login user (as uid)
148c65
<       ## Using a LDAP_BIND_USER/PW
---
>       ## Or you can use ai global LDAP_BIND_USER
206,213d122
<     labels:
<       - "traefik.enable=true"
<       - "traefik.tcp.routers.mongodb.rule=HostSNI(`*`)"
<       - "traefik.tcp.services.mongodb.loadbalancer.server.port=27017"
<       - "traefik.tcp.routers.mongodb.tls=true"
<       - "traefik.tcp.routers.mongodb.entrypoints=mongo"
<     networks:
<       - web
237,244d145
<     # modify to get rid of the redis issue #35 and #19 with a better solution
<     # WARNING: /proc/sys/net/core/somaxconn is set to the lower value of 128.
<     # for vm overcommit: enable first on host system
<     # sysctl vm.overcommit_memory=1 (and add it to rc.local)
<     # then you do not need it in the redis container
<     sysctls:
<       - net.core.somaxconn=65535
<       # - vm.overcommit_memory=1
254,259d154
<     networks:
<       - web
< 
< networks:
<   web:
<     external: true