On Sat, Jul 29, 2006 at 12:45:50PM -0400, Prof J C Nash wrote:
> Is there something between computer and Scotiabank that is giving
> troubles?
Yes, most likely DSL lines with improperly configured MTU and MSS
settings. I think this has been the general consensus in past
questions about the Scotiabank system, and certainly in many similar
situations.
Ever since consumer DSL services started using the "Point-to-Point
Protocol over Ethernet" (PPPoE) system, they have suffered from a
slightly lower Maximum Transmission Unit (MTU). MTU is the maximum
number of bytes per packet. Ethernet generally has an MTU of 1500
bytes. So does PPP. But PPPoE is sending packets via Ethernet with
real data plus some overhead. Because of this overhead, PPPoE MTU
will never exceed 1492.
Now, when you have a connection to a remote system, there's a "path
MTU", which is basically the *lowest* MTU of any link between you and
the target system. It's like you have two big pipes linked by a
little junction -- you'll never fit more water through than the little
junction can fit. Normally, the path MTU is 1500, same as the basic
Ethernet MTU. But when you use PPPoE, the path MTU is 1492 or lower.
Yet the computers on either end of the link don't know that. All they
know is the MTU for the first hop -- such as from the Scotiabank
server to the Internet, or from your computer to your DSL router, both
of which are probably 1500. As soon as one or the other tries to send
a full-sized packet (according to their definition of full-sized,
typically 1500 bytes), it "jams" in the pipe and is thrown away.
Normally, this is not a problem. As soon as a packet from your
machine hits your router, your router "realises" that it can't forward
the packet on (too big), and sends back an Internet Control Message
Protocol (ICMP) message saying, basically, "your packet is too big, I
can only support 1492 bytes". Your computer receives this and now
knows that it must send smaller packets for that connection.
The same thing *should* happen when Scotiabank sends a 1500-byte
packet to you. It will hit the computer that sits just beyond your
DSL modem link, who again will send an ICMP message to the Scotiabank
server saying, "too big, send smaller packets".
However, there are some misguided firewall operators out there who
don't understand the importance of ICMP. Certain ICMP packets have
been abused for Denial of Service attacks (DoS), and some admins
overreact and deny all ICMP as a result. Certain ICMP packets can be
used to see if a server is "up", and some admins misguidedly think
that it's better if people not know that. Or maybe they're just doing
a "block everything, allow what we need" approach, and not realising
they need ICMP.
Whatever the reason, if that "too big" ICMP packet never makes it to
the sender of the big packet, then the sender just thinks the Internet
is having a temporary hiccup. So it sends again. And again. And
again, and again, and again, with longer and longer delays, until the
connection times out.
Whenever connections seem to work until a certain point -- like an SSH
session that freezes if you try to refresh the screen, or web browsing
that works until you download a big document, or HTTPS (secure web
browsing) where it seems to connect and negotiate okay but then
freezes -- MTU should be your first suspect.
So how does one deal with an MTU issue? Well, that depends on which
side is having the problem. Generally, it's the remote side, unless
*you* are the firewall admin who messed up by denying ICMP. :)
If the problem is on your end, you can reduce the MTU of the system in
question. To do this ad hoc:
ifconfig eth0 mtu 1400
where eth0 is the interface on the system you're trying to "fix". If
this works, either put that command in a bootup script somewhere, or
find a way to have your distribution's network configuration scripts
set the MTU of the interface.
However, the problem is usually on the remote side, assuming your
network is set up correctly. For that, things get a little more
complicated.
For problems with the remote side, there's something in TCP called the
Maximum Send Size (MSS). Basically, this is a little note you stick
in your *outgoing* TCP connect packet that says to the *remote* host,
"please don't try to send packets bigger than this".
You want that to agree with your DSL's MTU limit. You can do that
with iptables. Here's a quote from the manpage for "iptables":
TCPMSS:
This target allows to alter the MSS value of TCP SYN packets,
to control the maximum size for that connection (usually
limiting it to your outgoing interface's MTU minus 40). Of
course, it can only be used in conjunction with -p tcp.
This target is used to overcome criminally braindead ISPs or
servers which block ICMP Fragmentation Needed packets. The
symptoms of this problem are that everything works fine from
your Linux firewall/router, but machines behind it can never
exchange large packets:
1) Web browsers connect, then hang with no data received.
2) Small mail works fine, but large emails hang.
3) ssh works fine, but scp hangs after initial handshaking.
(The "target" is the "TCPMSS" rule that sets the MSS; "SYN" packets
are "connecting" packets; "-p tcp" means only for the TCP protocol;
"ICMP Fragmentation Needed" packets are those "too big" packets I
talked about.)
The manpage recommends the "--clamp-mss-to-pmtu" option, which tries to
automatically guess the appropriate value. I'm not a fan of guessing.
Try something like this:
iptables -A OUTPUT -p tcp --syn -j TCPMSS --set-mss 1400
If your Linux system is a router for other systems, you should do this
as well as the above:
iptables -A FORWARD -p tcp --syn -j TCPMSS --set-mss 1400
An MSS of 1400 leaves a decent margin just in case.
If this solves your problem, then throw it into a bootup script and
you're set. "Criminally braindead" servers shouldn't hang any more.
Good luck.
Attachment:
signature.asc
Description: Digital signature