Common Firewall False Positives
By: Lawrence
Baldwin
I'm tired of hearing unexplainable firewall events as: "normal",
"background noise", "random probes", "people mis-typing IP
addresses", "Internet radiation", etc. These are all vague
assertions to explain away something this is admittedly difficult
to analyze and explain. When you hear words like "random"
and "noise" those are really synonyms for "I don't know"!
Well I DO know. As the operator of myNetWatchman, one of
the largest distributed intrusion detection systems (dIDS)
in the world I've spent thousands of hours analyzing the 100,000+
firewall events every day. Though I often feel like I've just
scratched the surface of understanding, I believe the following
scenarios explain about 80% of the false positives that are
logged by firewalls.
a) Slow server responses
Firewalls block and generate alerts when traffic is received
for which there isn't an associated and RECENT outbound request.
This means that if you make a request, but the server responds
too slowly (e.g. > 1 or 2 seconds), your firewall may consider
that response as unsolicited and block/log it.
The destination port of all server responses will be in
the range of 1025-65535. If you see your firewall log "probes"
against these ports AND the source is a server you are communicating
with, they are probably just slow server responses.
To complicate matters, your attempt to surf to ONE web site,
can actually result in communication with dozens of different
hosts--often hosts that would appear to be unaffiliated with
the site in question. Consider cnn.com...some of their content
is distributed across Akamai caching servers....so firewall
events sourced from XXXX.akamai.net while surfing to cnn.com
are probably just due to a slow cache server.
Example: mNW 2518142 (note:
this is a case of an Akamai server providing Real audio content)
If you see UDP events targeting high numbered ports, sourced
from the DNS port (udp/53) AND the source IP is your DNS server...then
these are probably slow DNS responses. Slow DNS responses
is probably the most frequent cause of false positives.
b) Proximity probes
Larger web sites maintain mirrored content on many distributed
web servers, often in multiple countries. When you do a DNS
lookup of a web site (e.g. www.windowsupdate.com ) the site's
load-balancing servers will send "proximity probes" from every
location to your IP address. PC's that don't have a firewall
will send back a reject packet (ICMP port unreachable) in
response to these probes. Information in the ICMP response
packets allow the load balancers to determine which one is
"closest" to the user, allowing it to provide the user the
IP address of the nearest web server.
Users running firewalls will block and log proximity probes
(they often target port tcp/53) because they come from IP
addresses that they did not make any outbound request to.
There is even one content hosting company that provides this
capability as part of its hosting services (mirror-image.com)...when
you surf to ANY website hosted by this company, you will be
immediately "probed" by over 10 load-balancers in 6 countries!
Example: mNW 305387 Note:
This incident is generated by the F5 Network's 3DNS
product.
c) Open proxy tests
If you are an IRC user you will likely be probed on several
ports every time you attempt to connect to an IRC server.
This to prevent anonymous IRC access through other user's
PC's that are unknowingly configured to allow proxying.
Example: mNW 2466138
d) Stale IP caches
If you have a dynamic IP address, you will often find that you
receive a lot of unsolicited probes when you first obtain a
new IP address. This often because the previous user of that
IP address was running some application which has cached their
IP address somewhere, and it's unaware that the owner of that
IP has changed.
Often the involved applications are Internet game servers,
peer-to-peer file/music software (e.g. Gnutella, Napster,
Kazaa, audiogalaxy, etc..).
Some of these applications are poorly written to handle
this situation and will incessantly pound an IP address thousands
of times for many hours. As much as this may seem like an
targeted attack, it is really just a function of poorly written
code that gives no consideration to how many firewall false
positives it generates.
Example: mNW 2529363
e) Search-engine bots
Similarly, people often post web content with URLs that contain
dynamic IP addresses (e.g. 123.123.123.123/index.htm ). Days,
weeks, or months later web search bots may encounter this reference
and then attempt to index that site. If you are the unfortunate
person to have IP address 123.123.123.123 you now get a few
dozen probes from abc.googlebot.com.
Example: mNW 2482968
f) Netbios name lookup from IIS servers
If an IIS server is also has Netbios over TCP/IP enabled,
the server will often send Netbios name lookups directly at
the users that surf to that site. This is due to IIS's attempt
to associate a host name with every IP address that accesses
it.
Although this is technically not hostile, it's probably
not advisable for webmasters to enable Netbios on their Internet
facing network adapter...nevertheless, this is a common scenario
Example: mNW 2470765
g) Victims of Spoofed Denial-of-service (DoS) attack
One or more attackers Syn-flood a victim web site, sending
each TCP connect request with a different randomly spoofed
IP address. The victim host sends a response (SYN/ACK) back
to each of the spoofed IPs. If the DoS attack is over a long
period of time, potentially millions of spoofed IPs may be
sent a response packet. Users running firewalls on any of
these IPs will log this response packet as a probe.
Example: mNW 2542636 - livejournal.com
DoS Attack
I spoke to the owner of the web site above and he confirmed
that he indeed was has been under DoS attack in the last day
or so.
The link above doesn't show it, but all these "probes" had
a *source* TCP port = 80...showing these these were really
response packets from the web server. Also notice, that the
4 sensors that picked up this activity all got hit within
a VERY short time-frame (2.5 hours). That tells me that whoever
was launching this DoS attack must have been generating a
boat load of connect attempts at an extremely high rate!
h) Broadcasted DHCP Offer packets
Here's an example firewall log documenting this issue: (thanks
to Gazer)
The firewall has blocked an Internet broadcast to
your computer (UDP Port 68) from 10.119.192.1 (DHCP). Occurred:
12 times between 3/28/2002 11:29:20 AM and 3/28/2002 12:22:54
PM
Credit: SYNACK
If your ISP assigns IP addresses using the Dynamic Host Control
Protocol (DHCP), then your firewall may log events when other
subscribers on the same shared network request an IP address.
According to the DHCP standards (RFC
1541, Sect 4.1) some DHCP clients can not receive a Unicast
packet until after IP address assignment has been completed.
Such clients set the 'Broadcast' bit in their DHCP Discover
packets, requesting that the DHCP server send their IP address
assignment (a DHCP Offer) as an IP broadcast (e.g. to destination
address '255.255.255.255'). Such a broadcast packet will be
received by ALL hosts connected to the same shared network
as the host making the DHCP request.
i) ICMP Unreachable DoS backscatter
If a web site is under a denial-of-service (Dos) attack,
they system owner will often contact their ISP and have a
"null route" added to their routing tables. The null route
causes all traffic destined for the customer's web site to
be discarded by ISP's routers--thus defeating the DoS attack.
However, as the attacker continues to send randomly spoofed
packets at the victim, the ISP's router will begin sending
ICMP unreachable packets packets to the *perceived* source
of the packets. As the attacker randomly picks YOUR IP address,
this can result in an ICMP packet being sent to you and thus
logged by your firewall.
Some more advanced ISP's (e.g. UUnet) also induce ICMP unreachables
when attempting to trace the source of spoofed DoS attacks.
See: Tracing DoS ingress
source using ICMP backscatter
|
