Mark a packets with iptables in order to make a source ip routing

Question

I have a linux machine, which I use as a gateway and firewall.

The linux machine (GW/FIREWALL) is connected to a device which allow internet connection (Router Aggregator), it has 2 IP addresses 172.16.61.1 and 172.16.62.254.

                                    +---------+    +-----------------+       /
                                    |         |    |     Router      |      /
                                    |   GW    |    |   Aggregator    |     /     
                                    |         |    |                 |    /
                 +------------------| Firewall+----+-> 172.16.61.1   +---/
                 |                  |         |    |                 |   |
                 |                  |         +----+-> 172.16.62.254 |   |
        __       |                  |         |    |                 |   |
    ___/  \_     |                  +---------+    +-----------------+   |
  _/        \__  |                                                       |
 /             \ |                                                       |Internet
| Local network  |                                                       |
| 192.168.1.0/24 +                                                       |
| 192.168.3.0/24 |                                                       |
\_           __/                                                         \
   \__     __/                                                            \
      \___/

The Router Aggregator acts as a gateway to allow access to the internet, but it has a peculiarity: if it is accessed through the ip 172.16.62.254 it guarantees a bandwidth of 1Gb/s, if it is accessed through the address 172.16.61.1 the bandwidth is limited to 300Mb/s .

To access the Router Aggregator through the ip 172.16.61.1 it is necessary that the source ip belongs to the network 172.16.61.0, while to access through the ip 172.16.62.254 it is necessary that the source ip is 172.16.62.100. For this reason, the GW / Firewall must perform a translation of the source ip addresses coming from the local network.

The ip addresses of the linux machine are :

$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:14:5e:08:49:06 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.150/24 brd 192.168.1.255 scope global eth3
    inet 192.168.168.1/24 brd 192.168.168.255 scope global eth3:0
    inet 192.168.8.1/24 brd 192.168.8.255 scope global eth3:2
    inet6 fe80::214:5eff:fe08:4906/64 scope link 
       valid_lft forever preferred_lft forever
3: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:13:49:24:41:6a brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.1/24 brd 192.168.3.255 scope global eth5
    inet6 fe80::213:49ff:fe24:416a/64 scope link 
       valid_lft forever preferred_lft forever
4: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:13:49:24:3e:e0 brd ff:ff:ff:ff:ff:ff
    inet 172.16.61.2/24 brd 172.16.61.255 scope global eth4
    inet 172.16.62.100/24 brd 172.16.62.255 scope global eth4:1
    inet 172.16.61.3/24 brd 172.16.61.255 scope global secondary eth4:2
    inet 172.16.61.4/24 brd 172.16.61.255 scope global secondary eth4:3
    inet 172.16.61.5/24 brd 172.16.61.255 scope global secondary eth4:4
    inet 172.16.61.6/24 brd 172.16.61.255 scope global secondary eth4:5
    inet 172.16.61.7/24 brd 172.16.61.255 scope global secondary eth4:6
     inet6 fe80::213:49ff:fe24:3ee0/64 scope link 
      valid_lft forever preferred_lft forever
5: sit0: <NOARP> mtu 1480 qdisc noop state DOWN 
    link/sit 0.0.0.0 brd 0.0.0.0

When a packet as a source address translation to network 172.16.61.0/24 must use 172.16.61.1 as gateway, when a packet as source address translation to ip 172.16.62.100 must use 172.16.62.254 as gateway.

With reference to network 172.16.61.0/24 only, the address translation are done by iptables using:

iptables -t nat -A POSTROUTING  -s 192.168.1.0/24 -j SNAT --to-source 172.16.61.2
iptables -A OUTPUT -s 172.16.61.2 -j ACCEPT

To route the packets via 172.16.61.1 I have tried to mark them using:

iptables -A OUTPUT -t mangle -s 172.16.61.2 -j MARK --set-mark 2
iptables -A POSTROUTING -t mangle -s 192.168.1.0/255.255.255.0 -j MARK --set-mark 2

then I use iproute2 to route the packet, with this configuration:

$ cat /etc/iproute2/rt_tables
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
1       route61
2       route62

$ ip route show table route61
172.16.61.0/24 via 172.16.61.1 dev eth4 
default via 172.16.61.1 dev eth4 

$ ip rule show
0:      from all lookup local 
32764:  from all fwmark 0x2 lookup route61 
32765:  from 172.16.61.0/24 lookup route61 
32766:  from all lookup main 
32767:  from all lookup default

but it doesn't work if, I try

$ nc -v 216.58.205.78 443

I have

nc: connect to 216.58.205.78 port 443 (tcp) failed: Network is unreachable

What did I do wrong?

Update:

I tried what A.B suggested in the post at the url marked packets not detected by routing policy database

I've added the following line

iptables -t mangle -I OUTPUT -m connmark ! --mark 0 -j CONNMARK --restore-mark
iptables -A OUTPUT -t mangle -s 172.16.61.2 -j MARK --set-mark 2
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j CONNMARK --save-mark

the mangle table turns out to be

$ iptables -t mangle -L  -v -n --line-numbers
Chain PREROUTING (policy ACCEPT 31455 packets, 11M bytes)
num   pkts bytes target     prot opt in     out     source               destination    
1        1    98 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           MARK match !0x0 CONNMARK save  

Chain INPUT (policy ACCEPT 2800 packets, 199K bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 28028 packets, 11M bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1860 packets, 433K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        1    70 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0           CONNMARK match !0x0 CONNMARK restore 
2       13   818 MARK       all  --  *      *       172.16.61.2          0.0.0.0/0           MARK set 0x2

without getting results.

The following instructions :

sysctl -w net.ipv4.conf.eth0.rp_filter=2
sysctl -w net.ipv4.fwmark_reflect=1

report that they are unknown keys.

iptables-save -c output purged of some lines not relevant to the post

# Generated by iptables-save v1.3.8 on Wed Aug 11 13:11:07 2021
*filter
:INPUT DROP [79:12622]
:FORWARD DROP [18:1104]
:OUTPUT DROP [0:0]
[633:37905] -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A INPUT -p icmp -m state --state ESTABLISHED -j ACCEPT 
[2:152] -A INPUT -s 192.168.1.0/255.255.255.0 -p tcp -j ACCEPT 
[0:0] -A INPUT -s 192.168.1.0/255.255.255.0 -p udp -j ACCEPT 
[10:1936] -A INPUT -s 192.168.1.0/255.255.255.0 -p icmp -j ACCEPT 
[8:696] -A FORWARD -p udp -m state --state ESTABLISHED -j ACCEPT 
[129:20225] -A FORWARD -p tcp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A FORWARD -p icmp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A FORWARD -s 192.168.1.0/255.255.255.0 -p icmp -j ACCEPT 
[33:3014] -A FORWARD -s 192.168.1.0/255.255.255.0 -p tcp -j ACCEPT 
[8:532] -A FORWARD -s 192.168.1.0/255.255.255.0 -p udp -j ACCEPT 
[1026:179208] -A OUTPUT -p tcp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A OUTPUT -p udp -m state --state ESTABLISHED -j ACCEPT 
[0:0] -A OUTPUT -p icmp -m state --state ESTABLISHED -j ACCEPT 
[10:1656] -A OUTPUT -s 192.168.1.0/255.255.255.0 -p tcp -j ACCEPT 
[2:140] -A OUTPUT -s 192.168.1.0/255.255.255.0 -p udp -j ACCEPT 
[0:0] -A OUTPUT -s 192.168.1.0/255.255.255.0 -p icmp -j ACCEPT 
[0:0] -A OUTPUT -s 172.16.61.2 -j ACCEPT 
COMMIT   
*mangle
:PREROUTING ACCEPT [920:78186]
:INPUT ACCEPT [724:52615]
:FORWARD ACCEPT [196:25571]
:OUTPUT ACCEPT [1040:181200]
:POSTROUTING ACCEPT [1218:205667]
[2:196] -A PREROUTING -m mark ! --mark 0x0 -j CONNMARK --save-mark 
[2:168] -A OUTPUT -m connmark ! --mark 0x0 -j CONNMARK --restore-mark 
[2:196] -A OUTPUT -s 172.16.61.2 -j MARK --set-mark 0x2 
COMMIT
*nat
:PREROUTING ACCEPT [127:16611]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [2:242]
[30:2978] -A POSTROUTING -s 192.168.1.0/255.255.255.0 -j SNAT --to-source 172.16.61.2 
COMMIT

Update:

I have tried what A.B suggested in the post at the url IPTable mangle rule to mark traffic for route table :

"Adding any default route in table main (even using a non-existing router as long as it's a valid syntax) would allow the intended flow:"
$ ip route add default via 192.168.168.150

$ ip route show table main
192.168.3.0/24 dev eth5  proto kernel  scope link  src 192.168.3.1 
192.168.1.0/24 dev eth3  proto kernel  scope link  src 192.168.1.150 
172.16.62.0/24 dev eth4  proto kernel  scope link  src 172.16.62.100 
172.16.61.0/24 dev eth4  proto kernel  scope link  src 172.16.61.2 
192.168.168.0/24 dev eth3  proto kernel  scope link  src 192.168.168.1 
192.168.8.0/24 dev eth3  proto kernel  scope link  src 192.168.8.1 
169.254.0.0/16 dev eth5  scope link 
10.200.0.0/16 via 192.168.1.185 dev eth3 
default via 192.168.168.150 dev eth3

Now on GW/Firewall I can connect to the internet as shown below

nc -v 216.58.205.78 443
Connection to 216.58.205.78 443 port [tcp/https] succeeded!

I am not yet able to connect to the internet from a PC belonging to the local network 192.168.1.0 and using the GW/Firewall as Gateway, I probably believe that this is due to the fact that the instruction of

iptables -t nat -A POSTROUTING  -s 192.168.1.0/24 -j SNAT --to-source 172.16.61.2

is performed after

iptables -A OUTPUT -t mangle -s 172.16.61.2 -j MARK --set-mark 2

and therefore the route61 routing table is not used.

The output of tcpdump -ni any '(host 192.168.1.5 or host 172.16.61.2)' and port 443 when i run nc -v 216.58.205.78 443 from pc 192.168.1.5 on the local network is

12:15:51.184032 IP 192.168.1.5.58870 > 216.58.205.78.https: S 4157294472:4157294472(0) win 29200 <mss 1460,sackOK,timestamp 14470157 0,nop,wscale 7>
12:15:52.186303 IP 192.168.1.5.58870 > 216.58.205.78.https: S 4157294472:4157294472(0) win 29200 <mss 1460,sackOK,timestamp 14471160 0,nop,wscale 7>
12:15:54.190008 IP 192.168.1.5.58870 > 216.58.205.78.https: S 4157294472:4157294472(0) win 29200 <mss 1460,sackOK,timestamp 14473164 0,nop,wscale 7>

and we see that the IP translation from 192.168.1.5 to 172.16.61.2 is not performed

if on GW/Firewall I run route add -net 0.0.0.0 gw 172.16.61.1 netmask 0.0.0.0 dev eth4 the output of tcpdump becomes :

13:00:53.194649 IP 192.168.1.5.59188 > 216.58.205.78.https: S 3448270320:3448270320(0) win 29200 <mss 1460,sackOK,timestamp 17172595 0,nop,wscale 7>
13:00:53.194879 IP 172.16.61.2.59188 > 216.58.205.78.https: S 3448270320:3448270320(0) win 29200 <mss 1460,sackOK,timestamp 17172595 0,nop,wscale 7>
13:00:53.223644 IP 216.58.205.78.https > 172.16.61.2.59188: S 3227185810:3227185810(0) ack 3448270321 win 65535 <mss 1430,sackOK,timestamp 3513192655 17172595,nop,wscale 8>
13:00:53.223691 IP 216.58.205.78.https > 192.168.1.5.59188: S 3227185810:3227185810(0) ack 3448270321 win 65535 <mss 1430,sackOK,timestamp 3513192655 17172595,nop,wscale 8>
13:00:53.223785 IP 192.168.1.5.59188 > 216.58.205.78.https: . ack 1 win 229 <nop,nop,timestamp 17172625 3513192655>
13:00:53.223819 IP 172.16.61.2.59188 > 216.58.205.78.https: . ack 1 win 229 <nop,nop,timestamp 17172625 3513192655>

Update : Now I am trying iptables -A PREROUTING -t mangle -s 192.168.1.0/24 ! -d 192.168.1.0/255.255.255.0 -j MARK --set-mark 0x2 which seems to give some good results. It allows me to connect to the internet from the pc 192.168.1.5, but does not allow me to connect to the pc 192.168.3.5 within the same local network.

Update :

I have made the following changes to the mangle table and now it seems to work :

iptables -t mangle -I OUTPUT -m connmark ! --mark 0 -j CONNMARK --restore-mark
iptables -A OUTPUT -t mangle -s 172.16.62.100 -j MARK --set-mark 1
iptables -A OUTPUT -t mangle -s 172.16.61.2   -j MARK --set-mark 2
iptables -A PREROUTING -t mangle -s 192.168.1.0/24 -d 192.168.3.0/24 -j ACCEPT
iptables -A PREROUTING -t mangle -s 192.168.3.0/24 -d 172.16.61.2    -j ACCEPT
iptables -A PREROUTING -t mangle -s 192.168.1.0/24 ! -d 192.168.1.0/24 -j MARK --set-mark 0x2
iptables -A PREROUTING -t mangle -s 192.168.3.5    ! -d 192.168.3.0/24 -j MARK --set-mark 0x1
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j CONNMARK --save-mark

@A.B I updated the post, but I had to remove a few lines as otherwise it would have been too long — famedoro
– famedoro, Commented Aug 11, 2021 at 11:23
@A.B It is a: Fedora release 8 (Werewolf) Linux 2.6.23.1-42.fc8 #1 SMP Tue Oct 30 13:18:33 EDT 2007 x86_64 GNU/Linux Linux runs directly on the physical machine — famedoro
– famedoro, Commented Aug 11, 2021 at 11:47
If I use a default gateway with route add -net 0.0.0.0 gw 172.16.61.1 netmask 0.0.0.0 dev eth4 the connection for the network 171.16.61.0 works. — famedoro
– famedoro, Commented Aug 11, 2021 at 11:50
Linux 2.6.23? honestly I don't think an answer will work as expected for a so old kernel — A.B
– A.B, Commented Aug 11, 2021 at 11:55

Stack Exchange Network

Mark a packets with iptables in order to make a source ip routing

0

You must log in to answer this question.

Linked

Hot Network Questions

Mark a packets with iptables in order to make a source ip routing

0

You must log in to answer this question.

Linked

Related

Hot Network Questions