Setting Multi-Home Servers with Rocky 9

A multi-home server is a server that has more than one interface and each interface is capable of routing. This sounds very exciting, especially if you are into low-budget high-availability. As sexy as it sounds, multi-homed systems are not as easy and transparent to configure.

Where is the catch? In Linux, multi-homed routers are possible, however, it is not what people think. Most may think a packet entered through eth0 will go out through the same interface. This is a false assumption. Each time a packet enters the interface, it is evaluated against rules and routing tables, and each time a packet (even from the same connection) is going out, it is re-evaluated against rules and routing tables.

That is the way it is. So here is how I did it on a RHEL

Ah, I forgot, in my example:

• ens18 faces the LAN,
• ens19 faces one ISP, and
• ens20 faces to another ISP.

Rocky 9 Multihomed Server

Linux systems can be multi-home systems thanks to advanced routing. This means a Linux system may have more than one routing table. The selection of the table is based on rules. The first thing I did was the definition of the new routing table. Edit the /etc/iproute2/rt_tables file and add a new line.

# 
# reserved values 
# 
255     local 
254     main 
253     default 
0       unspec 
# 
# local 
# 
#1      inr.ruhep 
200 voip
201 voip2

I will skip here on how you configure your interfaces. Just as a reminder, I did this to create a connection on the second interface: nmcli con add type ethernet con-name ens19 ifname ens19.

The last and most important thing is to set up all the rules and routes on booting time. Happily for us, the Network Manager has a way to do it. Network Manager will execute any script inside the /etc/NetworkManager/dispatcher.d/ directory. So, here is what I placed as /etc/NetworkManager/dispatcher.d/ha.sh. 

#!/bin/sh
IF="$1" # your current interface name such as eth0, wlp4s0 and so on
STATUS="$2" # status such as UP or DOWN

logger "$0 called for interface named $IF with $STATUS ..."

case "$IF" in
 ens19)
 if [ "$STATUS" = "up" ]; then
 /usr/sbin/ip route add y.y.y.0/24 dev ens20 proto kernel scope link table voip
 /usr/sbin/ip route add 192.168.86.0/24 dev ens18 proto kernel scope link metric 100 table voip
 /usr/sbin/ip route add x.x.x.0/28 dev ens19 proto kernel scope link table voip
 /usr/sbin/ip route add default x.x.x.1 dev ens19 proto static metric 100 table voip
 /usr/sbin/ip rule add to x.x.x.x table voip prio 1
 /usr/sbin/ip rule add from x.x.x.x table voip prio 2
 fi
 if [ "$STATUS" = "down" ]; then
 /usr/sbin/ip route del default via x.x.x.1 dev ens19 proto static metric 100 table voip
 /usr/sbin/ip route del x.x.x.0/28 dev ens19 proto kernel scope link table voip
 /usr/sbin/ip route del 192.168.86.0/24 dev ens18 proto kernel scope link metric 100 table voip 
 /usr/sbin/ip rule del to x.x.x.x table voip
 /usr/sbin/ip rule del from x.x.x.x table voip
 fi
 ;;
 ens20)
 if [ "$STATUS" = "up" ]; then
 /usr/sbin/ip route add x.x.x.0/28 dev ens19 proto kernel scope link table voip2
 /usr/sbin/ip route add 192.168.86.0/24 dev ens18 proto kernel scope link metric 100 table voip2
 /usr/sbin/ip route add y.y.y.0/28 dev ens19 proto kernel scope link table voip2
 /usr/sbin/ip route add default y.y.y.1 dev ens19 proto static metric 100 table voip2
 /usr/sbin/ip rule add to y.y.y.y table voip2 prio 1
 /usr/sbin/ip rule add from y.y.y.y table voip2 prio 2
 fi
 if [ "$STATUS" = "down" ]; then
 /usr/sbin/ip route del default via y.y.y.1 dev ens19 proto static metric 100 table voip2
 /usr/sbin/ip route del y.y.y.0/28 dev ens19 proto kernel scope link table voip2
 /usr/sbin/ip route del 192.168.86.0/24 dev ens18 proto kernel scope link metric 100 table voip2 
 /usr/sbin/ip rule del to y.y.y.y table voip2
 /usr/sbin/ip rule del from y.y.y.y table voip2
 fi
 ;;
esac
exit 0

Change it to fit your needs. Don't forget to make it executable.

Notes:

• x.x.x.x and y.y.y.y are the static IP your ISP gives you. If you do not have a static IP you may enter a network CIDR notation as well.
• x.x.x.0 and y.y.y.0 are the network names, we all know not all network names end with a zero, but just for the sake of clarity.
• x.x.x1 and y.y.y.1 are the gateways, and again, this is just for the sake of clarity.

If the interfaces are for incoming traffic mostly, you should be all set. But if you are more interested in the outgoing traffic, you have to include some IPTables rules (PREROUTING chain) to set up the egress IP and let the routing select what interface to use.

Good luck!