Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

netfilter-mailcow restart loop #5849

Closed
5 tasks done
Lulalaby opened this issue Apr 19, 2024 · 7 comments
Closed
5 tasks done

netfilter-mailcow restart loop #5849

Lulalaby opened this issue Apr 19, 2024 · 7 comments
Labels
bug stale Please update the issue with current status, unclear if it's still open/needed.

Comments

@Lulalaby
Copy link

Lulalaby commented Apr 19, 2024 •
edited
Loading

Contribution guidelines

I've found a bug and checked that ...

  • ... I understand that not following the below instructions will result in immediate closure and/or deletion of my issue.
  • ... I have understood that this bug report is dedicated for bugs, and not for support-related inquiries.
  • ... I have understood that answers are voluntary and community-driven, and not commercial support.
  • ... I have verified that my issue has not been already answered in the past. I also checked previous issues.

Description

Apparently something broke with the netfilter.
Mailcow is trying to fixing the container by restarting, over and over again.
This impacts the mail functionality.

Additionally: `WARN[0000] /opt/mailcow-dockerized/docker-compose.yml: `version` is obsolete`

Logs:

netfilter-mailcow-1  | MAILCOW target is in position 7 in the ip forward table, restarting container to fix it...
netfilter-mailcow-1  | # Warning: table ip6 nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip6 filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip mangle is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | Using NFTables backend
netfilter-mailcow-1  | Clearing all bans
netfilter-mailcow-1  | Clear completed: ip6
netfilter-mailcow-1  | Initializing mailcow netfilter chain
netfilter-mailcow-1  | MAILCOW ip6 chain created successfully.
netfilter-mailcow-1  | Setting MAILCOW isolation
netfilter-mailcow-1  | Watching Redis channel F2B_CHANNEL
netfilter-mailcow-1  | Blacklist was changed, it has 2 entries
netfilter-mailcow-1  | Added host/network 45.146.165.37 to blacklist
netfilter-mailcow-1  | Whitelist was changed, it has 2 entries
netfilter-mailcow-1  | Added host/network 5.34.207.156 to blacklist
watchdog-mailcow-1   | Fri Apr 19 15:44:38 CEST 2024 ACME health level: 100% (1/1), health trend: 0

Steps to reproduce:

1. Update to latest master
2. Observe logs

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Linux mail.aitsys.dev 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux (bullseye)

Server/VM specifications:

8GB RAM, 4 Cores AMD EPYC 7282 16-Core Processor

Is Apparmor, SELinux or similar active?

no

Virtualization technology:

KVM

Docker version:

26.0.2

docker-compose version or docker compose version:

v2.26.1

mailcow version:

2024-04

Reverse proxy:

None (Using mailcow directly)

Logs of git diff:

Logs of iptables -L -vn:

Chain INPUT (policy ACCEPT 46040 packets, 15M bytes)
 pkts bytes target     prot opt in     out     source               destination
70260   22M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
24299 6472K AS0_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    2   140 AS0_ACCEPT  all  --  lo     *       0.0.0.0/0            0.0.0.0/0
    0     0 AS0_IN_PRE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x2000000/0x2000000
    1    52 AS0_ACCEPT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:914
    0     0 AS0_ACCEPT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:915
    0     0 AS0_ACCEPT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:916
    0     0 AS0_ACCEPT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:917
    0     0 AS0_ACCEPT  udp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW udp dpt:918
    0     0 AS0_ACCEPT  udp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW udp dpt:919
    0     0 AS0_ACCEPT  udp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW udp dpt:920
    0     0 AS0_ACCEPT  udp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW udp dpt:921
    0     0 AS0_WEBACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 AS0_WEBACCEPT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:943
    0     0 AS0_APIACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
46040   15M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
46040   15M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
87253   56M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
87253   56M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
74656   33M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 3503  222K DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
 9094   23M ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
 3279  208K ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0
 122K   58M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
 102K   56M AS0_ACCEPT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 AS0_IN_PRE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x2000000/0x2000000
    0     0 AS0_OUT_S2C  all  --  *      as0t+   0.0.0.0/0            0.0.0.0/0
14158  918K MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
14158  918K MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */

Chain OUTPUT (policy ACCEPT 33348 packets, 8934K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 AS0_OUT_LOCAL  all  --  *      as0t+   0.0.0.0/0            0.0.0.0/0

Chain AS0_ACCEPT (11 references)
 pkts bytes target     prot opt in     out     source               destination
 126K   63M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_APIACCEPT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_IN (4 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            172.27.224.1
    0     0 AS0_IN_POST  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_IN_NAT (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000000
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_IN_POST (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            172.17.0.0/16
    0     0 AS0_OUT    all  --  *      as0t+   0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_IN_PRE (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 AS0_IN     all  --  *      *       0.0.0.0/0            169.254.0.0/16
    0     0 AS0_IN     all  --  *      *       0.0.0.0/0            192.168.0.0/16
    0     0 AS0_IN     all  --  *      *       0.0.0.0/0            172.16.0.0/12
    0     0 AS0_IN     all  --  *      *       0.0.0.0/0            10.0.0.0/8
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_IN_ROUTE (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000000
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_OUT (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 AS0_OUT_POST  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_OUT_LOCAL (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       icmp --  *      *       0.0.0.0/0            0.0.0.0/0            icmptype 5
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_OUT_POST (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x2000000/0x2000000
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_OUT_S2C (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       172.17.0.0/16        0.0.0.0/0
    0     0 AS0_OUT    all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_WEBACCEPT (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.5           tcp dpt:8983
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.6           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
  113  7408 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:443
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.8           tcp dpt:80
   17  1020 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587
   91  5460 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
    3   180 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
 9094   23M DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
 203K  114M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
13355   24M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
 203K  114M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MAILCOW (6 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
 6141  367K DROP       tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */
    0     0 DROP       all  --  *      *       45.146.165.37        0.0.0.0/0
    0     0 DROP       all  --  *      *       5.34.207.156         0.0.0.0/0
    0     0 DROP       tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */

Logs of ip6tables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
   11   894 MAILCOW    all      *      *       ::/0                 ::/0                 /* mailcow */
15815   14M DOCKER-USER  all      *      *       ::/0                 ::/0
15815   14M DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0
11499   13M ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
 3733  259K DOCKER     all      *      br-mailcow  ::/0                 ::/0
  583 57186 ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0
 3733  259K ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0
    0     0 ACCEPT     all      *      docker0  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all      *      docker0  ::/0                 ::/0
    0     0 ACCEPT     all      docker0 !docker0  ::/0                 ::/0
    0     0 ACCEPT     all      docker0 docker0  ::/0                 ::/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:587
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:25

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
  583 57186 DOCKER-ISOLATION-STAGE-2  all      br-mailcow !br-mailcow  ::/0                 ::/0
    0     0 DOCKER-ISOLATION-STAGE-2  all      docker0 !docker0  ::/0                 ::/0
56407   63M RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all      *      br-mailcow  ::/0                 ::/0
    0     0 DROP       all      *      docker0  ::/0                 ::/0
 2307  572K RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  17M   15G RETURN     all      *      *       ::/0                 ::/0

Chain MAILCOW (1 references)
 pkts bytes target     prot opt in     out     source               destination

Logs of iptables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 21574 packets, 1386K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 AS0_NAT_PRE_REL_EST  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 AS0_DPFWD_TCP  tcp  --  *      *       0.0.0.0/0            172.17.0.1           tcp dpt:1194 state NEW
    1    52 AS0_DPFWD_TCP  tcp  --  *      *       0.0.0.0/0            176.57.188.251       tcp dpt:1194 state NEW
    0     0 AS0_DPFWD_UDP  udp  --  *      *       0.0.0.0/0            172.17.0.1           udp dpt:1194 state NEW
    0     0 AS0_DPFWD_UDP  udp  --  *      *       0.0.0.0/0            176.57.188.251       udp dpt:1194 state NEW
 6902  403K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 2687 packets, 151K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 611 packets, 38887 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 15982 packets, 991K bytes)
 pkts bytes target     prot opt in     out     source               destination
 1366  105K MASQUERADE  all  --  *      !br-mailcow  172.22.1.0/24        0.0.0.0/0
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
    0     0 AS0_NAT_POST_REL_EST  all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
    0     0 AS0_NAT_PRE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x2000000/0x2000000
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.249         172.22.1.249         tcp dpt:6379
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.5           172.22.1.5           tcp dpt:8983
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.6           172.22.1.6           tcp dpt:3306
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:12345
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:4190
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.8           172.22.1.8           tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:995
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.8           172.22.1.8           tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:993
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:143
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:110
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:587
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:465
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:25

Chain AS0_DPFWD_TCP (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            to:176.57.188.251:914
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_DPFWD_UDP (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            to:176.57.188.251:918
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_NAT (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 SNAT       all  --  *      eth0    0.0.0.0/0            0.0.0.0/0            to:176.57.188.251
    0     0 SNAT       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            to:172.17.0.1
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_NAT_POST_REL_EST (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_NAT_PRE (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 AS0_NAT    all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x8000000/0x8000000
    0     0 AS0_NAT_TEST  all  --  *      *       0.0.0.0/0            169.254.0.0/16
    0     0 AS0_NAT_TEST  all  --  *      *       0.0.0.0/0            192.168.0.0/16
    0     0 AS0_NAT_TEST  all  --  *      *       0.0.0.0/0            172.16.0.0/12
    0     0 AS0_NAT_TEST  all  --  *      *       0.0.0.0/0            10.0.0.0/8
    0     0 AS0_NAT    all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_NAT_PRE_REL_EST (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain AS0_NAT_TEST (4 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      as0t+   0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x4000000/0x4000000
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            172.27.224.0/20
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            172.17.0.0/16
    0     0 AS0_NAT    all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    9   540 RETURN     all  --  br-mailcow *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:7654 to:172.22.1.249:6379
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:18983 to:172.22.1.5:8983
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:13306 to:172.22.1.6:3306
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:19991 to:172.22.1.250:12345
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4190 to:172.22.1.250:4190
  117  7728 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:172.22.1.8:443
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995 to:172.22.1.250:995
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:172.22.1.8:80
   18  1080 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:172.22.1.250:993
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143 to:172.22.1.250:143
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110 to:172.22.1.250:110
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587 to:172.22.1.253:587
   96  5760 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:465 to:172.22.1.253:465
    3   180 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.22.1.253:25

Logs of ip6tables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  174 14194 DOCKER     all      *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all      *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  566 53400 MASQUERADE  all      *      !br-mailcow  fd4d:6169:6c63:6f77::/64  ::/0
    0     0 MASQUERADE  all      *      !docker0  fd00:dead:beef:c0::/80  ::/0
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:995
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::c  fd4d:6169:6c63:6f77::c  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::b  fd4d:6169:6c63:6f77::b  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:25

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
   45  3600 RETURN     all      br-mailcow *       ::/0                 ::/0
    0     0 RETURN     all      docker0 *       ::/0                 ::/0
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:4190 to:[fd4d:6169:6c63:6f77::b]:4190
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:443 to:[fd4d:6169:6c63:6f77::c]:443
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:995 to:[fd4d:6169:6c63:6f77::b]:995
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:80 to:[fd4d:6169:6c63:6f77::c]:80
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:993 to:[fd4d:6169:6c63:6f77::b]:993
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:143 to:[fd4d:6169:6c63:6f77::b]:143
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:110 to:[fd4d:6169:6c63:6f77::b]:110
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:587 to:[fd4d:6169:6c63:6f77::f]:587
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:465 to:[fd4d:6169:6c63:6f77::f]:465
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:25 to:[fd4d:6169:6c63:6f77::f]:25

DNS check:

22.1.254
104.18.32.7
172.64.155.249
@Lulalaby Lulalaby added the bug label Apr 19, 2024
@Lulalaby
Copy link
Author

Oop, seems also related to #5798

@Lulalaby
Copy link
Author

Clearing all iptables rules with iptables -F and iptables -X, stopping mailcow with docker compose down, restarting docker with service docker restart and starting mailcow again with ./update.sh seems to fix the issue.

I did not manually touch the iptables, so I assume mailcow messed something up at some point

@MaxXor
Copy link
Contributor

MaxXor commented Apr 20, 2024

Yes, there is definitely something wrong. I noticed this too!

@Lulalaby
Copy link
Author

Yeah, I was lucky that I really use my vps just for the mailserver, lazy moment, so it wasn't that much of a problem to reset all.
But I can imagine on other infra setups it might be a real problem.

@Reggae13
Copy link

If I clear the iptables and restart the Mailcow as mentioned here, it works for less than a day before crashing again.

@MaxXor
Copy link
Contributor

MaxXor commented May 21, 2024 •
edited
Loading

I applied the work-around mentioned here: #5735 (comment)

and added this to /etc/nftables.conf (make sure that you use nftables and not iptables!):

table ip filter {
	chain DOCKER-USER {
		iifname != "br-mailcow" oifname "br-mailcow" tcp dport { 3306, 6379, 8983, 12345 } counter packets 0 bytes 0 drop
		counter packets 0 bytes 0 return
	}
}

This workaround should fix the vulnerability and fix the netfilter restart loop. Reboot after adding this. the docker service will automatically add other necessary DOCKER* chains to nftables and keep this DOCKER-USER chain untouched. You can list all rules with nft list ruleset.

@milkmaker
Copy link
Collaborator

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label Jul 20, 2024
@milkmaker milkmaker closed this as not planned Won't fix, can't repro, duplicate, stale Jul 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug stale Please update the issue with current status, unclear if it's still open/needed.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MaxXor @Reggae13 @Lulalaby @milkmaker