annotate html/rn01re01.html @ 2:bb3f804f13a0

add random unsynchronization to hourly timer, trust prefix only for same origin AS, ignore self adjacency
author Carl Byington <carl@five-ten-sg.com>
date Mon, 19 May 2008 21:45:45 -0700
parents 48d06780cf77
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
1 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
3 <html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>routeflapper</title><meta name="generator" content="DocBook XSL Stylesheets V1.65.1" /><link rel="home" href="index.html" title="routeflapper" /><link rel="up" href="index.html" title="routeflapper" /><link rel="previous" href="index.html" title="routeflapper" /><link rel="next" href="rn01re02.html" title="routeflapper.conf" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">routeflapper</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="index.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="rn01re02.html">Next</a></td></tr></table><hr /></div><div class="refentry" lang="en" xml:lang="en"><a id="routeflapper.1"></a><div class="titlepage"><div></div><div></div></div><div class="refnamediv"><a id="name.1"></a><h2>Name</h2><p>routeflapper — detects suspicious routes</p></div><div class="refsynopsisdiv"><a id="synopsis.1"></a><h2>Synopsis</h2><div class="cmdsynopsis"><p><tt class="command">routeflapper</tt> [<tt class="option">-c</tt>] [<tt class="option">-d <i class="replaceable"><tt>n</tt></i></tt>]</p></div></div><div class="refsect1" lang="en" xml:lang="en"><a id="description.1"></a><h2>Description</h2><p><span><b class="command">routeflapper</b></span> is a daemon that monitors BGP
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
4 updates and SMTP connections to discover whether SMTP connections are
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
5 coming from ip addresses whose best route is suspicious. </p><p>The <span class="citerefentry"><span class="refentrytitle">routeflapper.conf</span>(5)</span> file specifies the syslog files
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
6 to be monitored, and the regular expressions (<span class="citerefentry"><span class="refentrytitle">regex</span>(7)</span>) to be applied to new lines in those files. </p><p>The discussion has focused on syslog files, but any ascii text
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
7 file can be used, so long as some other process appends lines to that
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
8 file, and those lines containing bgp updates can be matched
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
9 with some regular expression.</p><p>Considering syslog files in particular, these are normally rotated
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
10 via logrotate. <span><b class="command">routeflapper</b></span> properly detects and
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
11 handles this case by closing the old file, and reopening the newly
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
12 created file.</p></div><div class="refsect1" lang="en" xml:lang="en"><a id="options.1"></a><h2>Options</h2><div class="variablelist"><dl><dt><span class="term">-c</span></dt><dd><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
13 Load the configuration file, print a cannonical form
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
14 of the configuration on stdout, and exit.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
15 </p></dd><dt><span class="term">-d <i class="replaceable"><tt>n</tt></i></span></dt><dd><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
16 Set the debug level to <i class="replaceable"><tt>n</tt></i>.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
17 </p></dd></dl></div></div><div class="refsect1" lang="en" xml:lang="en"><a id="usage.1"></a><h2>Usage</h2><p><span><b class="command">routeflapper</b></span> -d 2</p></div><div class="refsect1" lang="en" xml:lang="en"><a id="configuration.1"></a><h2>Configuration</h2><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
18 The configuration file is documented in <span class="citerefentry"><span class="refentrytitle">routeflapper.conf</span>(5)</span>. Any change to the config file will cause it to be
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
19 reloaded within three minutes.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
20 </p></div><div class="refsect1" lang="en" xml:lang="en"><a id="introduction.1"></a><h2>Introduction</h2><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
21 Consider the hypothetical case of a spammer who is connected via a
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
22 provider that does not filter BGP routing announcements. The spammer
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
23 then has some options to announce ip address space to be used for
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
24 sending spam. Note that we only consider cases where the spammer
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
25 simply wants to anonymously use some ip address space. This is very
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
26 different from the case where the attacker wants to use some specific
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
27 address space belonging to another organization in order to impersonate
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
28 some service provided by that other organization.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
29 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
30 They can announce a more specific route, for example a /24, inside a
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
31 larger block. For example, consider 169.232.0.0/16. If the spammer
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
32 pokes around, they can probably find an unused /24 in there. So they
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
33 announce 169.232.240.0/24 and then send spam from that block. There
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
34 are two problems with this scheme. First, the announcement of such a
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
35 smaller block may be filtered out by many BGP routers, reducing their
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
36 reachability to their spam targets. Second, they may have made a
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
37 mistake, and that /24 is actually in use by some UCLA service that
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
38 will notice their hijack.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
39 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
40 They can announce a less specific route, for example a /16, covering
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
41 some individual smaller blocks. For example, they could announce
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
42 52.129.0.0/16. The spammer could then avoid the four existing
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
43 announcements inside that block, and instead spam from
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
44 52.129.128.0/17. That gives them 32K ip addresses to work with. The
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
45 advantage here is that their announcement of a large block won't be
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
46 filtered out by as many (if any) BGP routers, giving them better reachability
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
47 to their spam targets. And they know they won't interfere with any
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
48 existing use of that address space, since there was no previous BGP
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
49 announcement of that /17 or any subset of it.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
50 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
51 Or they can simply announce a prefix that is not assigned to anyone.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
52 For example, they could simply start announcing 185.10.0.0/16. This
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
53 has many of the same advantages as the previous scheme, but some BGP
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
54 routers may be configured to drop such bogon announcements.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
55 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
56 In each of these cases, the spammer can use BGP to announce some
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
57 address space, then send spam from those addresses, and then withdraw
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
58 the route annoucement. This would make it difficult for the recipient of
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
59 such spam to determine who actually sent it.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
60 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
61 In a paper from 2006 published at <a href="http://www-static.cc.gatech.edu/~feamster/publications/p396-ramachandran.pdf" target="_top">
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
62 http://www-static.cc.gatech.edu/~feamster/publications/p396-ramachandran.pdf
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
63 </a>, Ramachandran and Feamster claim evidence for the statement
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
64 that spammers are using such short-lived bogus BGP route announcements
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
65 to send spam from hijacked parts of the IPv4 address space.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
66 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
67 The question is, are spammers actually doing this today, or is this
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
68 just a hypothetical spam tactic that they could use in the future? To
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
69 help answer that question, this package monitors BGP annoucements,
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
70 classifies some of them as suspicious, and logs instances of SMTP
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
71 connections from suspicious prefixes.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
72 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
73 We track the history of the AS adjacency graph, by computing the union
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
74 of all AS adjacent pairs over all the announced prefixes. For example,
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
75 137.169.0.0/16 is currently announced here with an AS path of '22298
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
76 19080 3549 6517 14981', so we add (22298,19080) (19080,3549)
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
77 (3549,6517) and (6517,14981) as valid adjacent AS pairs.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
78 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
79 We track the history of the origin AS for each announced prefix. Both
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
80 the origin AS and AS adjacency pairs are tracked over a timescale of
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
81 100 hours, with an exponential decay half-life of 100 hours.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
82 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
83 A prefix announcement is suspicious if the origin AS is not in the
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
84 historical AS set for that prefix at least 20% of the time, or if the
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
85 AS path contains any adjacent AS pair that is not in the historical AS
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
86 adjacency graph at least 40% of the time.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
87 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
88 <a href="http://phas.netsec.colostate.edu/" target="_top">PHAS</a> is another
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
89 system that attempts to detect address space hijacking, but it is not
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
90 correlated with SMTP connections or spam attempts.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
91 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
92 <a href="http://cs.unm.edu/~karlinjf/IAR/index.php" target="_top">IAR</a> is
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
93 another system that attempts to detect address space hijacking, but it
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
94 is not correlated with SMTP connections or spam attempts. IAR uses
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
95 methods detailed in <a href="http://www.cs.unm.edu/~treport/tr/06-06/pgbgp3.pdf" target="_top">PGBGP</a>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
96 to detect suspicious routes. One problem with PGBGP as applied to our
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
97 hypothetical spammer problem, is that PGBGP is primarily looking for
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
98 hijacks where the attacker actually wants some specific ip address
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
99 space, either for a denial of service, or to impersonate the actual
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
100 owner. Our hypothetical spammer does not care about that - they only
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
101 care about sending spam anonymously. In particular, PGBGP ignores
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
102 super-prefix hijacks, but it seems likely that that is the preferred
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
103 method for our hypothetical spammer. However, the PGBGP paper does provide
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
104 useful data on the required timescale to avoid most of the normal AS
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
105 origin changes.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
106 </p></div><div class="refsect1" lang="en" xml:lang="en"><a id="todo.1"></a><h2>TODO</h2><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
107 None.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
108 </p></div><div class="refsect1" lang="en" xml:lang="en"><a id="copyright.1"></a><h2>Copyright</h2><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
109 Copyright (C) 2008 by 510 Software Group &lt;carl@five-ten-sg.com&gt;
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
110 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
111 This program is free software; you can redistribute it and/or modify it
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
112 under the terms of the GNU General Public License as published by the
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
113 Free Software Foundation; either version 3, or (at your option) any
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
114 later version.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
115 </p><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
116 You should have received a copy of the GNU General Public License along
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
117 with this program; see the file COPYING. If not, please write to the
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
118 Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
119 </p></div><div class="refsect1" lang="en" xml:lang="en"><a id="version.1"></a><h2>Version</h2><p>
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
120 1.0.1
48d06780cf77 initial version
Carl Byington <carl@five-ten-sg.com>
parents:
diff changeset
121 </p></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="index.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="index.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="rn01re02.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">routeflapper </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> routeflapper.conf</td></tr></table></div></body></html>