Mercurial > routeflapper
annotate xml/routeflapper.in @ 8:69a5dcf953df default tip
routeflapper runs as root to read the log files
| author | Carl Byington <carl@five-ten-sg.com> |
|---|---|
| date | Thu, 04 Sep 2014 08:57:50 -0700 |
| parents | 180d26aa2a17 |
| children |
| rev | line source |
|---|---|
| 0 | 1 <reference> |
| 2 <title>@PACKAGE@</title> | |
| 3 <partintro> | |
| 4 <title>Packages</title> | |
| 5 | |
| 6 <para>The various source and binary packages are available at <ulink | |
| 7 url="http://www.five-ten-sg.com/@PACKAGE@/packages/">http://www.five-ten-sg.com/@PACKAGE@/packages/</ulink> | |
| 8 The most recent documentation is available at <ulink | |
| 9 url="http://www.five-ten-sg.com/@PACKAGE@/">http://www.five-ten-sg.com/@PACKAGE@/</ulink> | |
| 10 </para> | |
| 11 | |
| 12 <para>A <ulink | |
| 13 url="http://www.selenic.com/mercurial/wiki/">Mercurial</ulink> source | |
| 14 code repository for this project is available at <ulink | |
| 15 url="http://hg.five-ten-sg.com/@PACKAGE@/">http://hg.five-ten-sg.com/@PACKAGE@/</ulink>. | |
| 16 </para> | |
| 17 | |
| 18 </partintro> | |
| 19 | |
| 20 <refentry id="@PACKAGE@.1"> | |
| 21 <refentryinfo> | |
|
4
180d26aa2a17
Fedora 9 compile and const correctness.
Carl Byington <carl@five-ten-sg.com>
parents:
1
diff
changeset
|
22 <date>2008-06-12</date> |
| 0 | 23 </refentryinfo> |
| 24 | |
| 25 <refmeta> | |
| 26 <refentrytitle>@PACKAGE@</refentrytitle> | |
| 27 <manvolnum>1</manvolnum> | |
| 28 <refmiscinfo>@PACKAGE@ @VERSION@</refmiscinfo> | |
| 29 </refmeta> | |
| 30 | |
| 31 <refnamediv id='name.1'> | |
| 32 <refname>@PACKAGE@</refname> | |
| 33 <refpurpose>detects suspicious routes</refpurpose> | |
| 34 </refnamediv> | |
| 35 | |
| 36 <refsynopsisdiv id='synopsis.1'> | |
| 37 <title>Synopsis</title> | |
| 38 <cmdsynopsis> | |
| 39 <command>@PACKAGE@</command> | |
| 40 <arg><option>-c</option></arg> | |
| 41 <arg><option>-d <replaceable class="parameter">n</replaceable></option></arg> | |
| 42 </cmdsynopsis> | |
| 43 </refsynopsisdiv> | |
| 44 | |
| 45 <refsect1 id='description.1'> | |
| 46 <title>Description</title> | |
| 47 | |
| 48 <para><command>@PACKAGE@</command> is a daemon that monitors BGP | |
| 49 updates and SMTP connections to discover whether SMTP connections are | |
| 50 coming from ip addresses whose best route is suspicious. </para> | |
| 51 | |
| 52 <para>The <citerefentry> <refentrytitle>@PACKAGE@.conf</refentrytitle> | |
| 53 <manvolnum>5</manvolnum> </citerefentry> file specifies the syslog files | |
| 54 to be monitored, and the regular expressions (<citerefentry> | |
| 55 <refentrytitle>regex</refentrytitle> <manvolnum>7</manvolnum> | |
| 56 </citerefentry>) to be applied to new lines in those files. </para> | |
| 57 | |
| 58 <para>The discussion has focused on syslog files, but any ascii text | |
| 59 file can be used, so long as some other process appends lines to that | |
| 60 file, and those lines containing bgp updates can be matched | |
| 61 with some regular expression.</para> | |
| 62 | |
| 63 <para>Considering syslog files in particular, these are normally rotated | |
| 64 via logrotate. <command>@PACKAGE@</command> properly detects and | |
| 65 handles this case by closing the old file, and reopening the newly | |
| 66 created file.</para> | |
| 67 </refsect1> | |
| 68 | |
| 69 <refsect1 id='options.1'> | |
| 70 <title>Options</title> | |
| 71 <variablelist> | |
| 72 <varlistentry> | |
| 73 <term>-c</term> | |
| 74 <listitem> | |
| 75 <para> | |
| 76 Load the configuration file, print a cannonical form | |
| 77 of the configuration on stdout, and exit. | |
| 78 </para> | |
| 79 </listitem> | |
| 80 </varlistentry> | |
| 81 <varlistentry> | |
| 82 <term>-d <replaceable class="parameter">n</replaceable></term> | |
| 83 <listitem> | |
| 84 <para> | |
| 85 Set the debug level to <replaceable class="parameter">n</replaceable>. | |
| 86 </para> | |
| 87 </listitem> | |
| 88 </varlistentry> | |
| 89 </variablelist> | |
| 90 </refsect1> | |
| 91 | |
| 92 <refsect1 id='usage.1'> | |
| 93 <title>Usage</title> | |
| 94 <para><command>@PACKAGE@</command> -d 2</para> | |
| 95 </refsect1> | |
| 96 | |
| 97 <refsect1 id='configuration.1'> | |
| 98 <title>Configuration</title> | |
| 99 <para> | |
| 100 The configuration file is documented in <citerefentry> | |
| 101 <refentrytitle>@PACKAGE@.conf</refentrytitle> <manvolnum>5</manvolnum> | |
| 102 </citerefentry>. Any change to the config file will cause it to be | |
| 103 reloaded within three minutes. | |
| 104 </para> | |
| 105 </refsect1> | |
| 106 | |
| 107 <refsect1 id='introduction.1'> | |
| 108 <title>Introduction</title> | |
| 109 <para> | |
| 110 Consider the hypothetical case of a spammer who is connected via a | |
| 111 provider that does not filter BGP routing announcements. The spammer | |
| 112 then has some options to announce ip address space to be used for | |
| 113 sending spam. Note that we only consider cases where the spammer | |
| 114 simply wants to anonymously use some ip address space. This is very | |
| 115 different from the case where the attacker wants to use some specific | |
| 116 address space belonging to another organization in order to impersonate | |
| 117 some service provided by that other organization. | |
| 118 </para> | |
| 119 | |
| 120 <para> | |
| 121 They can announce a more specific route, for example a /24, inside a | |
| 122 larger block. For example, consider 169.232.0.0/16. If the spammer | |
| 123 pokes around, they can probably find an unused /24 in there. So they | |
| 124 announce 169.232.240.0/24 and then send spam from that block. There | |
| 125 are two problems with this scheme. First, the announcement of such a | |
| 126 smaller block may be filtered out by many BGP routers, reducing their | |
| 127 reachability to their spam targets. Second, they may have made a | |
| 128 mistake, and that /24 is actually in use by some UCLA service that | |
| 129 will notice their hijack. | |
| 130 </para> | |
| 131 | |
| 132 <para> | |
| 133 They can announce a less specific route, for example a /16, covering | |
| 134 some individual smaller blocks. For example, they could announce | |
| 135 52.129.0.0/16. The spammer could then avoid the four existing | |
| 136 announcements inside that block, and instead spam from | |
| 137 52.129.128.0/17. That gives them 32K ip addresses to work with. The | |
| 138 advantage here is that their announcement of a large block won't be | |
| 139 filtered out by as many (if any) BGP routers, giving them better reachability | |
| 140 to their spam targets. And they know they won't interfere with any | |
| 141 existing use of that address space, since there was no previous BGP | |
| 142 announcement of that /17 or any subset of it. | |
| 143 </para> | |
| 144 | |
| 145 <para> | |
| 146 Or they can simply announce a prefix that is not assigned to anyone. | |
| 147 For example, they could simply start announcing 185.10.0.0/16. This | |
| 148 has many of the same advantages as the previous scheme, but some BGP | |
| 149 routers may be configured to drop such bogon announcements, again | |
| 150 potentially reducing their reachability to their spam targets. | |
| 151 </para> | |
| 152 | |
| 153 <para> | |
| 154 In each of these cases, the spammer can use BGP to announce some | |
| 155 address space, then send spam from those addresses, and then withdraw | |
| 156 the route annoucement. This would make it difficult for the recipient of | |
| 157 such spam to determine who actually sent it. | |
| 158 </para> | |
| 159 | |
| 160 <para> | |
| 161 In a paper from 2006 published at <ulink | |
| 162 url="http://www-static.cc.gatech.edu/~feamster/publications/p396-ramachandran.pdf"> | |
| 163 http://www-static.cc.gatech.edu/~feamster/publications/p396-ramachandran.pdf | |
| 164 </ulink>, Ramachandran and Feamster claim evidence for the statement | |
| 165 that spammers are using such short-lived bogus BGP route announcements | |
| 166 to send spam from hijacked parts of the IPv4 address space. | |
| 167 </para> | |
| 168 | |
| 169 <para> | |
| 170 The question is, are spammers actually doing this today, or is this | |
| 171 just a hypothetical spam tactic that they could use in the future? To | |
| 172 help answer that question, this package monitors BGP annoucements, | |
| 173 classifies some of them as suspicious, and logs instances of SMTP | |
| 174 connections from suspicious prefixes. | |
| 175 </para> | |
| 176 | |
| 177 <para> | |
| 178 We track the history of the AS adjacency graph, by computing the union | |
| 179 of all AS adjacent pairs over all the announced prefixes. For example, | |
| 180 137.169.0.0/16 is currently announced here with an AS path of '22298 | |
| 181 19080 3549 6517 14981', so we add (22298,19080) (19080,3549) | |
| 182 (3549,6517) and (6517,14981) as valid adjacent AS pairs. | |
| 183 We also track the history of the origin AS for each announced prefix. Both | |
| 184 the origin AS and the AS adjacency pairs are tracked via the following | |
| 185 algorithm that runs every hour. | |
| 186 </para> | |
| 187 | |
| 188 <para> | |
| 189 For each prefix, (prefix[*] *= 0.99) to exponentially decay the current | |
| 190 prefix origin counts. Then, for each prefix, if the prefix is announced, | |
| 191 (prefix[current.origin]++) increments the hourly count for the current origin. | |
| 192 The decay factor of 0.99 gives the counts a half life of about 69 hours. | |
| 193 The same is done with the hourly counts for each observed AS adjacent pair. | |
| 194 </para> | |
| 195 | |
| 196 <para> | |
| 197 A prefix announcement is suspicious if the prefix[origin] count is less | |
| 198 than 3.0, or if the AS path contains any adjacent AS pair with a count | |
| 199 less than 3.0. | |
| 200 </para> | |
| 201 | |
| 202 <para> | |
| 203 <ulink url="http://phas.netsec.colostate.edu/">PHAS</ulink> is another | |
| 204 system that attempts to detect address space hijacking, but it is not | |
| 205 correlated with SMTP connections or spam attempts. | |
| 206 </para> | |
| 207 | |
| 208 <para> | |
| 209 <ulink url="http://cs.unm.edu/~karlinjf/IAR/index.php">IAR</ulink> is | |
| 210 another system that attempts to detect address space hijacking, but it | |
| 211 is not correlated with SMTP connections or spam attempts. IAR uses | |
| 212 methods detailed in <ulink | |
| 213 url="http://www.cs.unm.edu/~treport/tr/06-06/pgbgp3.pdf">PGBGP</ulink> | |
| 214 to detect suspicious routes. One problem with PGBGP as applied to our | |
| 215 hypothetical spammer problem, is that PGBGP is primarily looking for | |
| 216 hijacks where the attacker actually wants some specific ip address | |
| 217 space, either for a denial of service, or to impersonate the actual | |
| 218 owner. Our hypothetical spammer does not care about that - they only | |
| 219 care about sending spam anonymously. In particular, PGBGP ignores | |
| 220 super-prefix hijacks, but it seems likely that that is the preferred | |
| 221 method for our hypothetical spammer. However, the PGBGP paper does provide | |
| 222 useful data on the required timescale to avoid most of the normal AS | |
| 223 origin changes. | |
| 224 </para> | |
| 225 </refsect1> | |
| 226 | |
| 227 <refsect1 id='todo.1'> | |
| 228 <title>TODO</title> | |
| 229 <para> | |
| 230 None. | |
| 231 </para> | |
| 232 </refsect1> | |
| 233 | |
| 234 <refsect1 id='copyright.1'> | |
| 235 <title>Copyright</title> | |
| 236 <para> | |
| 237 Copyright (C) 2008 by 510 Software Group <carl@five-ten-sg.com> | |
| 238 </para> | |
| 239 <para> | |
| 240 This program is free software; you can redistribute it and/or modify it | |
| 241 under the terms of the GNU General Public License as published by the | |
| 242 Free Software Foundation; either version 3, or (at your option) any | |
| 243 later version. | |
| 244 </para> | |
| 245 <para> | |
| 246 You should have received a copy of the GNU General Public License along | |
| 247 with this program; see the file COPYING. If not, please write to the | |
| 248 Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA. | |
| 249 </para> | |
| 250 </refsect1> | |
| 251 | |
| 252 <refsect1 id='version.1'> | |
| 253 <title>Version</title> | |
| 254 <para> | |
| 255 @VERSION@ | |
| 256 </para> | |
| 257 </refsect1> | |
| 258 </refentry> | |
| 259 | |
| 260 | |
| 261 <refentry id="@PACKAGE@.conf.5"> | |
| 262 <refentryinfo> | |
|
4
180d26aa2a17
Fedora 9 compile and const correctness.
Carl Byington <carl@five-ten-sg.com>
parents:
1
diff
changeset
|
263 <date>2008-06-12</date> |
| 0 | 264 </refentryinfo> |
| 265 | |
| 266 <refmeta> | |
| 267 <refentrytitle>@PACKAGE@.conf</refentrytitle> | |
| 268 <manvolnum>5</manvolnum> | |
| 269 <refmiscinfo>@PACKAGE@ @VERSION@</refmiscinfo> | |
| 270 </refmeta> | |
| 271 | |
| 272 <refnamediv id='name.5'> | |
| 273 <refname>@PACKAGE@.conf</refname> | |
| 274 <refpurpose>configuration file for @PACKAGE@</refpurpose> | |
| 275 </refnamediv> | |
| 276 | |
| 277 <refsynopsisdiv id='synopsis.5'> | |
| 278 <title>Synopsis</title> | |
| 279 <cmdsynopsis> | |
| 280 <command>@PACKAGE@.conf</command> | |
| 281 </cmdsynopsis> | |
| 282 </refsynopsisdiv> | |
| 283 | |
| 284 <refsect1 id='description.5'> | |
| 285 <title>Description</title> | |
| 286 <para>The <command>@PACKAGE@.conf</command> configuration file is | |
| 287 specified by this partial bnf description. The entire config file | |
| 288 is case sensitive. All the keywords are lower case. | |
| 289 </para> | |
| 290 | |
| 291 <literallayout class="monospaced"><![CDATA[ | |
| 292 CONFIG := {FILE}+ | |
| 293 FILE := "file" FILENAME "{" PATTERN+ "};" | |
|
1
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
294 PATTERN := RESET | PATH | ANNOUNCE | WITHDRAW | IP |
|
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
295 RESET := "reset" REGEX "{" "}" ";" |
|
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
296 PATH := "path" REGEX "{" INDEXPATH "}" ";" |
|
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
297 ANNOUNCE := "announce" REGEX "{" INDEXVAL INDEXLEN "}" ";" |
|
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
298 WITHDRAW := "withdraw" REGEX "{" INDEXVAL INDEXLEN "}" ";" |
|
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
299 IP := "ip" REGEX "{" INDEXIP "}" ";" |
| 0 | 300 INDEXPATH := "index_path" REGEX-INTEGER-VALUE ";" |
| 301 INDEXVAL := "index_value" REGEX-INTEGER-VALUE ";" | |
| 302 INDEXLEN := "index_length" REGEX-INTEGER-VALUE ";" | |
| 303 INDEXIP := "index_ip" REGEX-INTEGER-VALUE ";"]]></literallayout> | |
| 304 </refsect1> | |
| 305 | |
| 306 <refsect1 id='sample.5'> | |
| 307 <title>Sample</title> | |
| 308 <literallayout class="monospaced"><![CDATA[ | |
| 309 file "/var/log/bgp" { | |
|
1
47f787af96c1
update documentation to match code
Carl Byington <carl@five-ten-sg.com>
parents:
0
diff
changeset
|
310 reset "ADJCHANGE: neighbor .* Up" {}; |
| 0 | 311 path " rcvd UPDATE w.* path (([0-9]| )*[0-9])" { |
| 312 index_path 1; | |
| 313 }; | |
| 314 announce " rcvd (([0-9]|\.)*)/([0-9]*)$" { | |
| 315 index_value 1; | |
| 316 index_length 3; | |
| 317 }; | |
| 318 withdraw " rcvd UPDATE about (([0-9]|\.)*)/([0-9]*) -- withdrawn" { | |
| 319 index_value 1; | |
| 320 index_length 3; | |
| 321 }; | |
| 322 }; | |
| 323 | |
| 324 file "/var/log/maillog" { | |
| 325 ip "NOQUEUE: connect from.* \[(.*)\]" { | |
| 326 index_ip 1; | |
| 327 }; | |
| 328 };]]></literallayout> | |
| 329 </refsect1> | |
| 330 | |
| 331 <refsect1 id='version.5'> | |
| 332 <title>Version</title> | |
| 333 <para> | |
| 334 @VERSION@ | |
| 335 </para> | |
| 336 </refsect1> | |
| 337 | |
| 338 </refentry> | |
| 339 </reference> |