iptables and Multiple NICs (masq/fwd)
Hi everyone
Any Linux networking types around please? I've got a (probably easy!) question for you:
I've got a dual Xeon machine running Centos 5 (effectively RedHat) which I've been using for firewall duties with two of five network cards in use (eth0 - external internet and eth1 my trusted home network 192.168.0.1/24). I've set up my firewall using iptables and forwarding/masquerading and all of my internal machines can ping each other and machines on the internet - works fine.
I now need to use a third NIC in the machine to connect other "internal" machines to it, with the aim of all machines on both eth1 and eth2 being able to communicate with each other (and the internet). I had decided that the easiest approach was to set up eth2 as a separate subnet (192.168.1.1/24) and masq the machines in that subnet as I have been doing for those hanging off eth1.
I'm nearly there, but not quite...
- the server with the NICs in it can ping everything (int/int2/ext)
- the machines on int (existing network) can ping the server but NOT int2 ("new") machines
- the machines on int2 can ping the server but NOT int1 machines
Everyone on the two internal networks can get to the internet (external) so forwarding is working.
Is it some kind of routing issue (well, yes... but how/what?) and do I fix it by changing my firewall or by configuring the routing table, or something else?
I've included as much info as I can below. Any comments on whether the firewall looks sensible (from a security point of view)/and changes I need to make to finish the last part of this (both subnets working) would be much appreciated
Here is the output of ifconfig (ext IP blanked with x.x.x.x)
Code:
eth0 Link encap:Ethernet HWaddr 00:B0:D0:20:E5:4F
inet addr:x.x.x.x Bcast:255.255.255.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2169439 errors:0 dropped:0 overruns:0 frame:0
TX packets:1433083 errors:0 dropped:0 overruns:0 carrier:0
collisions:1790 txqueuelen:1000
RX bytes:3102962411 (2.8 GiB) TX bytes:88647610 (84.5 MiB)
eth1 Link encap:Ethernet HWaddr 00:90:27:FC:90:B0
inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5374956 errors:0 dropped:0 overruns:0 frame:0
TX packets:10010489 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:330970541 (315.6 MiB) TX bytes:596826515 (569.1 MiB)
eth2 Link encap:Ethernet HWaddr 00:90:27:FC:90:B1
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7912 errors:0 dropped:0 overruns:0 frame:0
TX packets:3669 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:929420 (907.6 KiB) TX bytes:215114 (210.0 KiB)
...and route:
Code:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 * 255.255.255.0 U 0 0 0 eth2
192.168.0.0 * 255.255.255.0 U 0 0 0 eth1
82.9.236.0 * 255.255.252.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth2
default abc2-mexch-0-0- 0.0.0.0 UG 0 0 0 eth0
Here's the new firewall script, based on (my understanding of) the Linux HOWTO example.
Code:
#!/bin/sh
#
# rc.firewall-iptables-stronger
#
FWVER=0.88s
IPTABLES=/sbin/iptables
LSMOD=/sbin/lsmod
DEPMOD=/sbin/depmod
MODPROBE=/sbin/modprobe
GREP=/bin/grep
AWK=/bin/awk
IFCONFIG=/sbin/ifconfig
# For this example, "eth0" is external and "eth1+2" are internal
EXTIF="eth0"
INTIF="eth1"
INTIF2="eth2"
echo " External Interface: $EXTIF"
echo " Internal Interface: $INTIF"
echo " 2nd Intl Interface: $INTIF2"
echo " ---"
# Determine the external IP automatically:
EXTIP="`$IFCONFIG $EXTIF | $AWK \
/$EXTIF/'{next}//{split($0,a,":");split(a[2],a," ");print a[1];exit}'`"
echo " External IP: $EXTIP"
echo " ---"
# Assign the internal TCP/IP network and IP address
INTNET="192.168.0.0/24"
INTNET2="192.168.1.0/24"
INTIP="192.168.0.1/32"
INTIP2="192.168.1.1/32"
echo " Internal Network: $INTNET"
echo " Internal IP: $INTIP"
echo " Internal Network2: $INTNET2"
echo " Internal IP2: $INTIP2"
echo " ---"
# Setting a few other local variables
#
UNIVERSE="0.0.0.0/0"
# Need to verify that all modules have all required dependencies
#
echo " - Verifying that all kernel modules are ok"
$DEPMOD -a
echo -en " Loading kernel modules: "
# With the new IPTABLES code, the core MASQ functionality is now either
# modular or compiled into the kernel. This HOWTO shows ALL IPTABLES
# options as MODULES. If your kernel is compiled correctly, there is
# NO need to load the kernel modules manually.
#Load the main body of the IPTABLES module - "ip_tables"
# - Loaded automatically when the "iptables" command is invoked
#
# - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_tables, "
#
#Verify the module isn't loaded. If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_tables | $AWK {'print $1'} `" ]; then
$MODPROBE ip_tables
fi
#Load the IPTABLES filtering module - "iptable_filter"
#
# - Loaded automatically when filter policies are activated
#Load the stateful connection tracking framework - "ip_conntrack"
#
# The conntrack module in itself does nothing without other specific
# conntrack modules being loaded afterwards such as the "ip_conntrack_ftp"
# module
#
# - This module is loaded automatically when MASQ functionality is
# enabled
#
# - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_conntrack, "
#Verify the module isn't loaded. If it is, skip it
if [ -z "` $LSMOD | $GREP ip_conntrack | $AWK {'print $1'} `" ]; then
$MODPROBE ip_conntrack
fi
#Load the FTP tracking mechanism for full FTP tracking
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -e "ip_conntrack_ftp, "
#Verify the module isn't loaded. If it is, skip it
if [ -z "` $LSMOD | $GREP ip_conntrack_ftp | $AWK {'print $1'} `" ]; then
$MODPROBE ip_conntrack_ftp
fi
#Load the general IPTABLES NAT code - "iptable_nat"
# - Loaded automatically when MASQ functionality is turned on
#
# - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "iptable_nat, "
#Verify the module isn't loaded. If it is, skip it
if [ -z "` $LSMOD | $GREP iptable_nat | $AWK {'print $1'} `" ]; then
$MODPROBE iptable_nat
fi
#Loads the FTP NAT functionality into the core IPTABLES code
# Required to support non-PASV FTP.
echo -e "ip_nat_ftp"
#Verify the module isn't loaded. If it is, skip it
if [ -z "` $LSMOD | $GREP ip_nat_ftp | $AWK {'print $1'} `" ]; then
$MODPROBE ip_nat_ftp
fi
echo " ---"
#CRITICAL: Enable IP forwarding since it is disabled by default since
echo " Enabling forwarding.."
echo "1" > /proc/sys/net/ipv4/ip_forward
echo " Enabling DynamicAddr.."
echo "1" > /proc/sys/net/ipv4/ip_dynaddr
echo " ---"
#Clearing any previous configuration
#
# Unless specified, the defaults for INPUT, OUTPUT, and FORWARD to DROP
#
# You CANNOT change this to REJECT as it isn't a vaild policy setting.
# If you want REJECT, you must explictly REJECT at the end of a giving
# INPUT, OUTPUT, or FORWARD chain
#
echo " Clearing any existing rules and setting default policy to DROP.."
$IPTABLES -P INPUT DROP
$IPTABLES -F INPUT
$IPTABLES -P OUTPUT DROP
$IPTABLES -F OUTPUT
$IPTABLES -P FORWARD DROP
$IPTABLES -F FORWARD
$IPTABLES -F -t nat
#Not needed and it will only load the unneeded kernel module
#
#$IPTABLES -F -t mangle
# Delete all User-specified chains
$IPTABLES -X
# Reset all IPTABLES counters
$IPTABLES -Z
#Configuring specific CHAINS for later use in the ruleset
#
# NOTE: Some users prefer to have their firewall silently
# "DROP" packets while others prefer to use "REJECT"
# to send ICMP error messages back to the remote
# machine. The default is "REJECT" but feel free to
# change this below.
#
# NOTE: Without the --log-level set to "info", every single
# firewall hit will goto ALL vtys. This is a very big
# pain.
#
echo " Creating a DROP chain.."
$IPTABLES -N reject-and-log-it
$IPTABLES -A reject-and-log-it -j LOG --log-level info
$IPTABLES -A reject-and-log-it -j REJECT
echo -e "\n - Loading INPUT rulesets"
#######################################################################
# INPUT: Incoming traffic from various interfaces. All rulesets are
# already flushed and set to a default policy of DROP.
# loopback interfaces are valid.
#
$IPTABLES -A INPUT -i lo -s $UNIVERSE -d $UNIVERSE -j ACCEPT
# local interface, local machines, going anywhere is valid
#
$IPTABLES -A INPUT -i $INTIF -s $INTNET -d $UNIVERSE -j ACCEPT
#Oaf: second NIC:
$IPTABLES -A INPUT -i $INTIF2 -s $INTNET2 -d $UNIVERSE -j ACCEPT
# remote interface, claiming to be local machines, IP spoofing, get lost
$IPTABLES -A INPUT -i $EXTIF -s $INTNET -d $UNIVERSE -j reject-and-log-it
# external interface, from any source, for ICMP traffic is valid
$IPTABLES -A INPUT -i $EXTIF -p ICMP -s $UNIVERSE -d $EXTIP -j ACCEPT
# Allow any related traffic coming back to the MASQ server in.
# STATEFULLY TRACKED
$IPTABLES -A INPUT -i $EXTIF -s $UNIVERSE -d $EXTIP -m state --state \
ESTABLISHED,RELATED -j ACCEPT
#
# ----- End OPTIONAL INPUT Section -----
# Catch all rule, all other incoming is denied and logged.
#
$IPTABLES -A INPUT -s $UNIVERSE -d $UNIVERSE -j reject-and-log-it
# ---------------------------------------------------------------------
echo -e " - Loading OUTPUT rulesets"
#######################################################################
# OUTPUT: Outgoing traffic from various interfaces. All rulesets are
# already flushed and set to a default policy of DROP.
# Workaround bug in netfilter
# See http://www.netfilter.org/security/2002-04-02-icmp-dnat.html
#
$IPTABLES -A OUTPUT -m state -p icmp --state INVALID -j DROP
# loopback interface is valid.
#
$IPTABLES -A OUTPUT -o lo -s $UNIVERSE -d $UNIVERSE -j ACCEPT
# local interfaces, any source going to local net is valid
#
$IPTABLES -A OUTPUT -o $INTIF -s $EXTIP -d $INTNET -j ACCEPT
# Oaf eth2 interface
$IPTABLES -A OUTPUT -o $INTIF2 -s $EXTIP -d $INTNET2 -j ACCEPT
$IPTABLES -A OUTPUT -o $INTIF -s $EXTIP -d $INTNET2 -j ACCEPT
$IPTABLES -A OUTPUT -o $INTIF2 -s $EXTIP -d $INTNET -j ACCEPT
# local interface, MASQ server source going to the local net is valid
#
$IPTABLES -A OUTPUT -o $INTIF -s $INTIP -d $INTNET -j ACCEPT
# Oaf eth2 interface
$IPTABLES -A OUTPUT -o $INTIF2 -s $INTIP2 -d $INTNET2 -j ACCEPT
# outgoing to local net on remote interface, stuffed routing, deny
#
$IPTABLES -A OUTPUT -o $EXTIF -s $UNIVERSE -d $INTNET -j reject-and-log-it
# anything else outgoing on remote interface is valid
#
$IPTABLES -A OUTPUT -o $EXTIF -s $EXTIP -d $UNIVERSE -j ACCEPT
# ----- Begin OPTIONAL OUTPUT Section -----
#
# DHCPd - Enable the following lines if you run an INTERNAL DHCPd server
# - Remove BOTH #s all the #s if you need this functionality.
#
#$IPTABLES -A OUTPUT -o $INTIF -p tcp -s $INTIP --sport 67 \
# -d 255.255.255.255 --dport 68 -j ACCEPT
#$IPTABLES -A OUTPUT -o $INTIF -p udp -s $INTIP --sport 67 \
# -d 255.255.255.255 --dport 68 -j ACCEPT
#
# ----- End OPTIONAL OUTPUT Section -----
# Catch all rule, all other outgoing is denied and logged.
#
$IPTABLES -A OUTPUT -s $UNIVERSE -d $UNIVERSE -j reject-and-log-it
echo -e " - Loading FORWARD rulesets"
#######################################################################
# FORWARD: Enable Forwarding and thus IPMASQ
#
# ----- Begin OPTIONAL FORWARD Section -----
#
# Put PORTFW commands here
#
# ----- End OPTIONAL FORWARD Section -----
echo " - FWD: Allow all connections OUT and only existing/related IN"
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED \
-j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
#Oaf eth2
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF2 -m state --state ESTABLISHED,RELATED \
-j ACCEPT
$IPTABLES -A FORWARD -i $INTIF2 -o $EXTIF -j ACCEPT
# Catch all rule, all other forwarding is denied and logged.
#
$IPTABLES -A FORWARD -j reject-and-log-it
echo " - NAT: Enabling SNAT (MASQUERADE) functionality on $EXTIF"
#
#More liberal form
#$IPTABLES -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
#
#Stricter form
$IPTABLES -t nat -A POSTROUTING -o $EXTIF -j SNAT --to $EXTIP
#################################################################
Any ideas please?
I just want to be able to get the two subnets talking to each other :telephone:
Thanks for reading this far :stupid:
Oaf
Re: iptables and Multiple NICs (masq/fwd)
Not a direct answer - but if I read you correctly, wouldn't it have been easier to setup the second NIC to be on the same network is your eth1? This way there should be no iptables routing. All machines on the new eth2 will just have to route through that NIC to hit eth1 and its routing. Or have I read this wrong?
Re: iptables and Multiple NICs (masq/fwd)
Quote:
Originally Posted by
alexkoon
...wouldn't it have been easier to setup the second NIC to be on the same network is your eth1? This way there should be no iptables routing. All machines on the new eth2 will just have to route through that NIC to hit eth1 and its routing. Or have I read this wrong?
Hi
Thanks for your reply. That had been my original idea (I guess really it's the third NIC) but then I thought separate subnets for eth1 and eth2 would mean I didn't have to worry about masks... beginning to wonder now :rolleyes:
I'll have another look later and see if I can progress it further. Networking isn't really my thing but I have a suspicion that I'm not THAT far from sorting it!
Cheers
Oaf
Re: iptables and Multiple NICs (masq/fwd)
alexkoon is right about it being easier to simply set up a better subnet mask than 255.255.255.0 - the catch-all solution being 255.255.0.0 which means 192.168.*.* can talk to each other
Re: iptables and Multiple NICs (masq/fwd)
Ah that sounds like a good plan. Will try it later today and let you know how I get on.
Thanks :)
Oaf
