19

Can a multihomed Linux machine implement a true Strong ES Model?

###Specific Use Case

I have a system with five different interfaces, each connected to the same subnet, thus the same gateway to the Internet.

  • I would like to listen on each interface separately on the same port, and ensure that packets always go out the same interface they came in, and ensure that packets trying to come in the "wrong" interface are discarded.
  • I would like to be able to bind to each interface and make outgoing connections to Internet destinations that always originate from the same source IP I bound to. For example,
curl --interface interface_ip http://ipecho.net/plain

should always show the same IP address I bound to with --interface.

  • Static routes may be problematic due to using DHCP on one of these interfaces.

RFC 1122

From RFC 1122 - Requirements for Internet Hosts - Communication Layers, Section 3.3.4.2 – Multihoming Requirements:

Internet host implementors have used two different conceptual models for multihoming, briefly summarized in the following discussion.  This document takes no stand on which model is preferred; each seems to have a place.  This ambivalence is reflected in the issues (A) and (B) being optional.

  •   Strong ES Model
      The Strong ES (End System, i.e., host) model emphasizes the host/gateway (ES/IS) distinction, and would therefore substitute MUST for MAY in issues (A) and (B) above.  It tends to model a multihomed host as a set of logical hosts within the same physical host.

      With respect to (A), proponents of the Strong ES model note that automatic Internet routing mechanisms could not route a datagram to a physical interface that did not correspond to the destination address.

      Under the Strong ES model, the route computation for an outgoing datagram is the mapping: 
             route(src IP addr, dest IP addr, TOS) -> gateway

    Here the source address is included as a parameter in order to select a gateway that is directly reachable on the corresponding physical interface.  Note that this model logically requires that in general there be at least one default gateway, and preferably multiple defaults, for each IP source address.


  •   Weak ES Model
      This view de-emphasizes the ES/IS distinction, and would therefore substitute MUST NOT for MAY in issues (A) and (B).  This model may be the more natural one for hosts that wiretap gateway routing protocols, and is necessary for hosts that have embedded gateway functionality.

      The Weak ES Model may cause the Redirect mechanism to fail.  If a datagram is sent out a physical interface that does not correspond to the destination address, the first-hop gateway will not realize when it needs to send a Redirect.  On the other hand, if the host has embedded gateway functionality, then it has routing information without listening to Redirects.

      In the Weak ES model, the route computation for an outgoing datagram is the mapping: 
             route(dest IP addr, TOS) -> gateway, interface

Linux is a Weak ES model by default, whereas FreeBSD and other Unix varieties act as Strong ES systems. Is there any way to make it behave more like a Strong ES system?

What sysctl or compile-time config would need to be set to make it behave like a Strong ES by default, without adding specific routing rules for any new interface you add? I know we can do strict source-route filtering via net.ipv4.conf.default.rp_filter = 1, but there seems to be much more to it than that. How can I do Source-Based Routing by default?

Improve this question
1
  • 2
    For the close-voters, why? This seems to be directly on-topic to me. If not, where would it be on-topic? Commented Jan 31, 2016 at 5:07

3 Answers 3

Reset to default
13

Just adding firewall rules won't be enough for this one. You want the system to route traffic as if it was two independent systems that just happen to share the same hardware and processes: that's what the Strong ES Model effectively is.

When aiming for Strong ES Model in Linux, you'll first need these sysctl settings:

net.ipv4.conf.all.arp_filter=1 
net.ipv4.conf.all.arp_ignore=1 # or even 2
net.ipv4.conf.all.arp_announce=2

These settings will make ARP behave as appropriate with the Strong ES Model, i.e. when an ARP request is received, only the interface with the exact requested address will answer, and only if traffic to the originating address would in fact be sent out through that specific interface.

Then, since you have five interfaces which you want to be behave differently in terms of routing, you'll need to set up five custom routing tables. You could use numbers to identify them, but generally it is clearer to specify names for them. So, pick numbers for each of them between 1 and 252, and some suitable names. (Numbers 0, 253, 254 and 255 are reserved.)

For example, let's pick 100 = rtable0, 101 = rtable1, 102 = rtable2, 103 = rtable3 and 104 = rtable4. Add these numbers and names to the end of /etc/iproute2/rt_tables file:

# ...default stuff above...
100    rtable0
101    rtable1
102    rtable2
103    rtable3
104    rtable4

Then, populate each custom routing table with a minimal set of route entries appropriate for each interface. (I'm replacing actual values with hopefully descriptively named environment variable names.)

ip route add $ETH0_NET dev eth0 proto static src $ETH0_IP table rtable0
ip route add default via $ETH0_GW dev eth0 proto static src $ETH0_IP table rtable0

ip route add $ETH1_NET dev eth1 proto static src $ETH1_IP table rtable1
ip route add default via $ETH1_GW dev eth1 proto static src $ETH1_IP table rtable1

# ... and so on, for all 5 interfaces

Finally, add advanced routing rules that will check the source address of each package and choose the routing table to be used accordingly:

ip rule add from $ETH0_IP table rtable0
ip rule add from $ETH1_IP table rtable1
#...

To make all this configuration persistent over reboots, you may have to write custom start-up scripts (or possibly ifup-pre or ifup-post scripts) to suit your Linux distribution's conventions.

For an extra insurance, you might add per-interface iptables rules to silently drop any incoming packets that may get received on the wrong interface. If all is well, the packet counts for these should remain zeroes: if they start increasing, you may have missed something in the configuration.

iptables -A INPUT -m addrtype --dst-type UNICAST -i eth0 ! -d $ETH0_IP -j DROP
iptables -A INPUT -m addrtype --dst-type UNICAST -i eth1 ! -d $ETH1_IP -j DROP
# ... and so on for each interface

Note: I've once implemented a set-up like this based on an old internet discussion by Rick Jones and other networking gurus. They said, paraphrased, "while all this is clearly necessary to achieve Strong Host Model behavior in Linux, I cannot guarantee that it is sufficient for all possible use cases". It worked flawlessly for me; it may or may not be enough for you, depending on exactly what you are going to use it for.

Warning: ensure that you have some sort of local or remote console access to the system when setting this configuration up. This set-up is extremely likely to completely mess up your network access while it's only half-done.

While it would be possible to set up N interfaces with only (N-1) custom routing tables, my personal preference is to move all the routing configuration to custom tables when using advanced routing. Having route -n or ip route show come up essentially empty while the system is clearly having network connections will be a very big clue that something very special is going on. Nevertheless, back when I set up a system like this, I also set up a permanent notice in /etc/motd about advanced routing being in effect, and the locations of the actual scripts that set it up.

Improve this answer
3
  • Would it be possible and/or useful to use network namespaces to either replace (some of) this or make it "stronger"? Commented Jan 22, 2018 at 20:41
  • 1
    That would be a subtly different thing. A strong host model essentially just means "no short cuts": a response to an incoming connection will be sent out the same interface the connection came in, even though the routing table might indicate there could be a more direct route using another interface - and a connection would be accepted in only on the interface that actually has the destination IP address. Network namespaces might achieve the same thing, but at the cost of having to duplicate service processes for each namespace. Commented Mar 12, 2018 at 17:23
  • 1
    Instead of issuing multiple iptables command one can use this magic nftables one liner nft add rule inet filter input fib daddr . iif type != { local, broadcast, multicast } drop. This is't really the subject of original question thought it was about the gateway, i.e. output chain Commented Jun 15, 2019 at 13:52
2

There is another option which allows more control about the ARP handling. Have a look at arp_ignore.

Improve this answer
7
  • Thanks! But this wouldn't have much to do with the actual source-routing of packets, right? Commented Jan 31, 2016 at 5:06
  • @Will arp_filter is about source routing; arp_ignore is about source and target addresses only. Commented Jan 31, 2016 at 5:11
  • Ah, got it! So a combination of arp_filter, arp_ignore, and rp_filter could do the trick? Ideally, I'm wanting each interface to behave as if they're located on different machines, and only ever respond from the same interface a packet came in on. Commented Jan 31, 2016 at 5:17
  • @Will Have you tried arp_filter, doesn't it do what you want? If so then add some details about the situation to your question. In your last comment you mix up the perspective: ARP is about where a packets comed in. Routing is about where it goes out. What you probably mean is you want packets accepted only on those interfaces over whoich the response would be sent. But arp_filter should do that. Commented Jan 31, 2016 at 5:26
  • 1
    @Will Oh, I misread your question. I saw arp_filter but you mentioned rp_filter. Commented Jan 31, 2016 at 5:52
0

A single iptables rule using the addrtype module is sufficient to enforce the strong host model:

iptables -I INPUT -m addrtype ! --dst-type LOCAL,BROADCAST --limit-iface-in -j DROP

From the horse’s mouth:

Find address type as if only "dev" was present in the system.

There is no need to set up multiple routing tables.

Tweaking ARP may be beneficial as well, but it certainly isn’t foolproof and does not apply to point-to-point interfaces.

Improve this answer
2
  • 1
    Are you sure that's enough? If there are two interfaces with different adddresses and different next-hop routers to a particular destination (or default gateways), how does just an input firewall rule affect which one is chosen? Commented Jan 15 at 6:48
  • 1
    You are right, this doesn’t address 3.3.4.2 (B), which is source-based routing. Commented Jan 15 at 15:34

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.