[afnog] Packet Forwarding Issue with Linux

[Gerald Begumisa]

Hi all,

I'm having difficulty understanding why packet forwarding in Linux is not
behaving as expected in the scenario described below. The confusing thing is
that this configuration works on an older version of Linux (Kernel:
2.6.11.4-20a-default; Distro: SuSE Linux 9.3), and not on a newer one
(CentOS 5.5). We're trying to replace the old server with a newer one.

Description follows:

1. I have two servers, let's call them server A and server B

2. Server A is being used as a gateway, and server B accesses the Internet
through server A

3. Our ISP gave us a block of addresses namely: 1.2.2.192/28

4. Our ISP told us to configure our "router" (Linux) with the following
information on the interface interconnected with the ISP's router:
    IP address: 1.2.3.42
    Netmask: 255.255.255.224
    Default Gateway: 1.2.3.33

5. Our ISP notified us that the IP address 1.2.3.42 is not routable on the
Internet and therefore, if we intend to have our "router" masquerade for our
LAN, we must give our router an IP in the range 1.2.2.192/28.  Needless to
say, they further clarified that for any machine on our network which needs
to be accessible from the Internet should be assigned an IP in the range
1.2.2.192/28.

6. Now, server A has two interfaces, eth0 and eth2, with eth2 connected to
the ISP's infrastructure
   The configuration of eth2 is as follows: IP address: 1.2.3.42; netmask:
255.255.255.224
   The configuration of eth0 is as follows: IP address: 1.2.2.206; netmask:
255.255.255.240
   Server A's default gateway was duly configured as 1.2.3.33 as directed by
the ISP

7. In order for server A itself to access the Internet, we made the
following configuration:
        /usr/sbin/iptables -s 1.2.3.42 --table nat --append POSTROUTING
--out-interface eth2 -j SNAT --to-source 1.2.2.206
    The above was because any packets originating from server A would not be
Internet-routable since the source would be set to 1.2.3.33 (see point [5]
above).

8. Now, server B's interface, eth0 is configured as follows: IP address:
1.2.2.205; Netmask: 255.255.255.240

9. Server B's default gateway is configured as 1.2.2.206

10. IP forwarding has been turned on, on server A, as below:

server-A:~ # sysctl -a|grep ipv4|grep forwarding
net.ipv4.conf.eth2.mc_forwarding = 0
net.ipv4.conf.eth2.forwarding = 1
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.forwarding = 1


Now, the problem is as follows:

Server B cannot access the Internet as per the explanations below:

The path of ping requests is traced as serverB:eth0 -> serverA:eth0 ->
serverA:eth2 -> remote_server -> serverA:eth2 and that's where it stops, the
"last mile" for inbound packets from the Internet destined for server B stop
at serverA:eth2. Longer explanation below:

1. When I ping 4.2.2.2 (or any pingable address on the "far Internet") from
server A, I get replies. This is good.

2. When I ping 4.2.2.2 from server B, I do not get any replies :-(

3. Using tcpdump on server B, I can see the ping requests leaving eth0 on
server B with source addr 1.2.2.205 and destination addr 4.2.2.2

4. Using tcpdump on server A, I can see the ping requests coming in through
eth0 with source addr 1.2.2.205 and destination addr 4.2.2.2

5. Using tcpdump on server A, I can see the ping requests leaving eth2 with
source addr 1.2.2.205 and destination addr 4.2.2.2

6. Using tcpdump on server A, I can see the ping replies coming in through
eth2 with source addr 4.2.2.2 and destination addr 1.2.2.205

And that's where the issue is, the packets are not forwarded through eth0 on
server A back to server B, and therefore server B never gets the replies.


Any clues on what could be going on will be highly appreciated.

Regards,
Gerald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://afnog.org/pipermail/afnog/attachments/20110411/95d63d3f/attachment.html>