Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compatibility Issue Between iptables-nft, nftables and netfilter-mailcow #5847

Open
5 tasks done
othiman opened this issue Apr 18, 2024 · 8 comments
Open
5 tasks done

Compatibility Issue Between iptables-nft, nftables and netfilter-mailcow #5847

othiman opened this issue Apr 18, 2024 · 8 comments
Labels
bug stale Please update the issue with current status, unclear if it's still open/needed.

Comments

@othiman
Copy link

othiman commented Apr 18, 2024

Contribution guidelines

I've found a bug and checked that ...

  • ... I understand that not following the below instructions will result in immediate closure and/or deletion of my issue.
  • ... I have understood that this bug report is dedicated for bugs, and not for support-related inquiries.
  • ... I have understood that answers are voluntary and community-driven, and not commercial support.
  • ... I have verified that my issue has not been already answered in the past. I also checked previous issues.

Description

Hi,

I use iptables-nft for firewall rules on the mailcow system because libvirt, which I use for different VMs, doesn't support nftables (yet). However, netfilter-mailcow detects via data/Dockerfiles/netfilter/docker-entrypoint.sh both iptables and nftables and defaults to using nftables. The issue arises when nftables modifies the nat table, making it unreadable for iptables-nft (iptables v1.8.7 (nf_tables): table `nat' is incompatible, use 'nft' tool.) and causing libvirt to fail when creating new networks or restarting libvirtd.

The logs below indicate warnings about not altering firewall tables (with nft). My suggestion is for these warnings to trigger a fallback to iptables(-nft) backend. Alternatively, a config parameter could be added to override the netfilter backend.

Logs:

`docker compose logs|grep netfilter`

netfilter-mailcow-1  | # Warning: table ip filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip6 filter is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | # Warning: table ip6 nat is managed by iptables-nft, do not touch!
netfilter-mailcow-1  | Using NFTables backend
netfilter-mailcow-1  | Clearing all bans
netfilter-mailcow-1  | Clear completed: ip
netfilter-mailcow-1  | Clear completed: ip6
netfilter-mailcow-1  | Initializing mailcow netfilter chain
netfilter-mailcow-1  | MAILCOW ip chain created successfully.
netfilter-mailcow-1  | MAILCOW ip6 chain created successfully.
netfilter-mailcow-1  | Setting MAILCOW isolation
...

Steps to reproduce:

1. iptables -t nat -L
  command shows full firewall nat table
2. Startup mailcow while nftables tool nft is installed
3. iptables -t nat -L
  command only shows: `iptables v1.8.7 (nf_tables): table `nat' is incompatible, use 'nft' tool.`

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Ubuntu 22.04.4 LTS

Server/VM specifications:

64GB Ram, 8 cores (Intel(R) Core(TM) i7-7700 CPU)

Is Apparmor, SELinux or similar active?

no

Virtualization technology:

mailcow with docker direct on host, other VMs using KVM

Docker version:

26.0.1

docker-compose version or docker compose version:

v2.16.1

mailcow version:

2024-04

Reverse proxy:

Traefik

Logs of git diff:

Just certificates and this:


diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index 572300db..1fb637c8 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -173,3 +173,36 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks
 
 # DO NOT EDIT ANYTHING BELOW #
 # Overrides #
+
+postscreen_dnsbl_sites = wl.mailspike.net=127.0.0.[18;19;20]*-2
+  hostkarma.junkemailfilter.com=127.0.0.1*-2
+  list.dnswl.org=127.0.[0..255].0*-2
+  list.dnswl.org=127.0.[0..255].1*-4
+  list.dnswl.org=127.0.[0..255].2*-6
+  list.dnswl.org=127.0.[0..255].3*-8
+  ix.dnsbl.manitu.net*2
+  bl.spamcop.net*2
+  bl.suomispam.net*2
+  hostkarma.junkemailfilter.com=127.0.0.2*3
+  hostkarma.junkemailfilter.com=127.0.0.4*2
+  hostkarma.junkemailfilter.com=127.0.1.2*1
+  backscatter.spameatingmonkey.net*2
+  bl.ipv6.spameatingmonkey.net*2
+  bl.spameatingmonkey.net*2
+  b.barracudacentral.org=127.0.0.2*7
+  bl.mailspike.net=127.0.0.2*5
+  bl.mailspike.net=127.0.0.[10;11;12]*4
+  dnsbl.sorbs.net=127.0.0.10*8
+  dnsbl.sorbs.net=127.0.0.5*6
+  dnsbl.sorbs.net=127.0.0.7*3
+  dnsbl.sorbs.net=127.0.0.8*2
+  dnsbl.sorbs.net=127.0.0.6*2
+  dnsbl.sorbs.net=127.0.0.9*2
+  zen.spamhaus.org=127.0.0.[10;11]*8
+  zen.spamhaus.org=127.0.0.[4..7]*6
+  zen.spamhaus.org=127.0.0.3*4
+  zen.spamhaus.org=127.0.0.2*3
+
+# User Overrides
+myhostname = mail.mailhost.de
+



### Logs of iptables -L -vn:

```plain text
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
1233K 9320M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
1420M 6142G LIBVIRT_INP  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
4592K 4679M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* mailcow */
4590K 4679M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
4590K 4679M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
18214   10M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
  121  7216 DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
17254 2134K ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            192.168.122.4        tcp dpt:443
    0     0 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            192.168.122.4        tcp dpt:22
    0     0 ACCEPT     all  --  *      br-207bdca914e4  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-207bdca914e4  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-207bdca914e4 !br-207bdca914e4  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-207bdca914e4 br-207bdca914e4  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-d69bcf815d29 !br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-d69bcf815d29 br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0           
2292M 4036G ACCEPT     all  --  *      br-126410685ea9  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
  14M  842M DOCKER     all  --  *      br-126410685ea9  0.0.0.0/0            0.0.0.0/0           
1139M   86G ACCEPT     all  --  br-126410685ea9 !br-126410685ea9  0.0.0.0/0            0.0.0.0/0           
  14M  832M ACCEPT     all  --  br-126410685ea9 br-126410685ea9  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            213.133.100.25       udp spts:30000:44999
    7   443 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            192.168.122.5        udp dpts:30000:44999
  151  6240 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            192.168.122.5        tcp dpt:1443
19441 1190K ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            192.168.122.5        tcp dpt:443
  131  5888 ACCEPT     tcp  --  *      virbr0  0.0.0.0/0            192.168.122.5        tcp dpt:22
  11G 7888G LIBVIRT_FWX  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  11G 7888G LIBVIRT_FWI  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
6027M 4081G LIBVIRT_FWO  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
1241M 1273G LIBVIRT_OUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER (5 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.2           tcp dpt:8983
  105  6144 ACCEPT     tcp  --  !br-126410685ea9 br-126410685ea9  0.0.0.0/0            172.18.0.3           tcp dpt:443
   25  1268 ACCEPT     tcp  --  !br-126410685ea9 br-126410685ea9  0.0.0.0/0            172.18.0.3           tcp dpt:80
    0     0 ACCEPT     udp  --  !br-d69bcf815d29 br-d69bcf815d29  0.0.0.0/0            172.19.0.3           udp dpt:1935
   20  1200 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587
   80  4800 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
    2   120 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25
   31  1836 ACCEPT     tcp  --  !br-126410685ea9 br-126410685ea9  0.0.0.0/0            172.18.0.5           tcp dpt:8000
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
    1    40 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
    2   104 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    2   112 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.9           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.13          tcp dpt:443
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.13          tcp dpt:80

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
16457 2015K DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-207bdca914e4 !br-207bdca914e4  0.0.0.0/0            0.0.0.0/0           
1604K  146M DOCKER-ISOLATION-STAGE-2  all  --  br-126410685ea9 !br-126410685ea9  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-d69bcf815d29 !br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0           
4590K 4679M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (5 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-207bdca914e4  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-126410685ea9  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-d69bcf815d29  0.0.0.0/0            0.0.0.0/0           
1620K  148M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  15G   12T RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain LIBVIRT_FWI (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_FWO (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_FWX (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_INP (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_OUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  377 22540 DROP       all  --  *      *       194.169.175.10       0.0.0.0/0           
  468 29172 DROP       all  --  *      *       194.169.175.17       0.0.0.0/0           
    0     0 DROP       tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0            /* mailcow isolation */

Logs of ip6tables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 1982  164K MAILCOW    all      *      *       ::/0                 ::/0                 /* mailcow */
6993K 8773M LIBVIRT_INP  all      *      *       ::/0                 ::/0                

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 7027 2129K MAILCOW    all      *      *       ::/0                 ::/0                 /* mailcow */
 7052 2138K DOCKER-USER  all      *      *       ::/0                 ::/0                
 7052 2138K DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0                
    0     0 ACCEPT     all      *      docker0  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all      *      docker0  ::/0                 ::/0                
    0     0 ACCEPT     all      docker0 !docker0  ::/0                 ::/0                
    0     0 ACCEPT     all      docker0 docker0  ::/0                 ::/0                
 1630  622K ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all      *      br-mailcow  ::/0                 ::/0                
 1671  161K ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0                
    0     0 ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0                
 208K   37M LIBVIRT_FWX  all      *      *       ::/0                 ::/0                
 208K   37M LIBVIRT_FWI  all      *      *       ::/0                 ::/0                
 208K   37M LIBVIRT_FWO  all      *      *       ::/0                 ::/0                
78149   27M ACCEPT     all      br0    br0     ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
3812K   68G LIBVIRT_OUT  all      *      *       ::/0                 ::/0                

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::4  tcp dpt:587
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::4  tcp dpt:465
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::4  tcp dpt:25
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:4190
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:995
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:993
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:143
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:110

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all      docker0 !docker0  ::/0                 ::/0                
 1422  138K DOCKER-ISOLATION-STAGE-2  all      br-mailcow !br-mailcow  ::/0                 ::/0                
    0     0 DOCKER-ISOLATION-STAGE-2  all      br-207bdca914e4 !br-207bdca914e4  ::/0                 ::/0                
    0     0 DOCKER-ISOLATION-STAGE-2  all      br-126410685ea9 !br-126410685ea9  ::/0                 ::/0                
    0     0 DOCKER-ISOLATION-STAGE-2  all      br-d69bcf815d29 !br-d69bcf815d29  ::/0                 ::/0                
 7068 2142K RETURN     all      *      *       ::/0                 ::/0                

Chain DOCKER-ISOLATION-STAGE-2 (5 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all      *      docker0  ::/0                 ::/0                
    0     0 DROP       all      *      br-mailcow  ::/0                 ::/0                
    0     0 DROP       all      *      br-207bdca914e4  ::/0                 ::/0                
    0     0 DROP       all      *      br-126410685ea9  ::/0                 ::/0                
    0     0 DROP       all      *      br-d69bcf815d29  ::/0                 ::/0                
 1422  138K RETURN     all      *      *       ::/0                 ::/0                

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  26M   37G RETURN     all      *      *       ::/0                 ::/0                

Chain LIBVIRT_FWI (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_FWO (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_FWX (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_INP (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain LIBVIRT_OUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination

Logs of iptables -L -vn -t nat:

iptables v1.8.7 (nf_tables): table `nat' is incompatible, use 'nft' tool.

Logs of ip6tables -L -vn -t nat:

ip6tables v1.8.7 (nf_tables): table `nat' is incompatible, use 'nft' tool.

DNS check:

104.18.32.7
172.64.155.249
@othiman othiman added the bug label Apr 18, 2024
@milkmaker
Copy link
Collaborator

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label Jun 17, 2024
@othiman
Copy link
Author

othiman commented Jun 18, 2024

I think this issue is still valid and can cause a lot of problems. E.g., in our system, where we are running libvirt/KVM in parallel, libvirt is not able to change the firewall rules and therefore refuses to start.

@milkmaker milkmaker removed the stale Please update the issue with current status, unclear if it's still open/needed. label Jun 18, 2024
@djdomi
Copy link

djdomi commented Jun 23, 2024

i also copy and say i am affected for this imho, having the similar warning

@JohnPBee
Copy link

JohnPBee commented Jul 2, 2024

same here

@wmtech-1
Copy link

Same here. I agree with the suggestions in the OP.

@milkmaker
Copy link
Collaborator

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label Oct 21, 2024
@othiman
Copy link
Author

othiman commented Oct 25, 2024

I think the bug report is still valid, isn't it? Still no official answer to now. 🥲

@milkmaker milkmaker removed the stale Please update the issue with current status, unclear if it's still open/needed. label Oct 25, 2024
@milkmaker
Copy link
Collaborator

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug stale Please update the issue with current status, unclear if it's still open/needed.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@djdomi @othiman @wmtech-1 @milkmaker @JohnPBee