Mercurial > dnsbl
annotate xml/dnsbl.in @ 75:1142e46be550
start coding on new config syntax
| author | carl |
|---|---|
| date | Wed, 13 Jul 2005 23:04:14 -0700 |
| parents | fb8afa205293 |
| children | 81f1e400e8ab |
| rev | line source |
|---|---|
| 0 | 1 <html> |
| 2 | |
| 3 <head> | |
| 4 <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> | |
| 75 | 5 <title>DNSBL Sendmail milter - Version 5.0</title> |
| 0 | 6 </head> |
| 7 | |
| 12 | 8 <center>Introduction</center> |
| 0 | 9 <p>This milter is released under the GPL license version 2 included in |
| 10 the LICENSE file in the distribution, and also available at | |
| 11 <a href="http://www.gnu.org/licenses/gpl.html">http://www.gnu.org/licenses/gpl.html</a> | |
| 12 | |
| 12 | 13 <p>Consider the case of a mail server that is acting as secondary MX for |
| 14 a collection of clients, each of which has a collection of mail domains. | |
| 15 Each client may use their own collection of DNSBLs on their primary mail | |
| 16 server. We present here a mechanism whereby the backup mail server can | |
| 17 use the correct set of DNSBLs for each recipient for each message. As a | |
| 0 | 18 side-effect, it gives us the ability to customize the set of DNSBLs on a |
| 19 per-recipient basis, so that fred@example.com could use SPEWS and the | |
| 20 SBL, where all other users @example.com use only the SBL. | |
| 21 | |
| 68 | 22 <p>This milter will also decode (uuencode, base64, mime, html entity, |
| 23 url encodings) and scan for HTTP and HTTPS URLs and bare hostnames in | |
| 24 the body of the mail. If any of those host names have A or NS records | |
| 25 on the SBL (or a single configurable DNSBL), the mail will be rejected | |
| 34 | 26 unless previously whitelisted. This milter also counts the number of |
| 27 invalid HTML tags, and can reject mail if that count exceeds your | |
| 28 specified limit. | |
| 11 | 29 |
| 6 | 30 <p>The DNSBL milter reads a text configuration file (dnsbl.conf) on |
| 31 startup, and whenever the config file (or any of the referenced include | |
| 32 files) is changed. The entire configuration file is case insensitive. | |
| 0 | 33 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
34 <hr> <center>DCC Issues</center> |
| 0 | 35 <p>If you are also using the <a |
| 36 href="http://www.rhyolite.com/anti-spam/dcc/">DCC</a> milter, there are | |
| 37 a few considerations. You may need to whitelist senders from the DCC | |
| 38 bulk detector, or from the DNS based lists. Those are two very | |
| 39 different reasons for whitelisting. The former is done thru the DCC | |
| 40 whiteclnt config file, the later is done thru the DNSBL milter config | |
| 5 | 41 file. |
| 0 | 42 |
| 43 <p>You may want to blacklist some specific senders or sending domains. | |
| 44 This could be done thru either the DCC (on a global basis, or for a | |
| 45 specific single recipient). We prefer to do such blacklisting via the | |
| 13 | 46 DNSBL milter config, since it can be done for a collection of recipient |
| 47 mail domains. The DCC approach has the feature that you can capture the | |
| 0 | 48 entire message in the DCC log files. The DNSBL milter approach has the |
| 49 feature that the mail is rejected earlier (at RCPT TO time), and the | |
| 50 sending machine just gets a generic "550 5.7.1 no such user" message. | |
| 51 | |
| 75 | 52 <p>The DCC whiteclnt file can be included in the DNSBL milter config by |
| 53 the dcc_to and dcc_from statements. This will import the (env_to, | |
| 54 env_from, and substitute mail_host) entries from the DCC config into the | |
| 55 DNSBL config. This allows using the DCC config as the single point for | |
| 56 white/blacklisting. | |
| 5 | 57 |
| 58 <p>Consider the case where you have multiple clients, each with their | |
| 59 own mail servers, and each running their own DCC milters. Each client | |
| 60 is using the DCC facilities for envelope from/to white/blacklisting. | |
| 6 | 61 Presumably you can use rsync or scp to fetch copies of your clients DCC |
| 5 | 62 whiteclnt files on a regular basis. Your mail server, acting as a |
| 63 backup MX for your clients, can use the DNSBL milter, and include those | |
| 75 | 64 client DCC config files. The envelope from/to white/blacklisting will |
| 65 be appropriately tagged and used only for the domains controlled by each | |
| 66 of those clients. | |
| 5 | 67 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
68 <hr> <center>Definitions</center> |
| 75 | 69 |
| 70 <p>CONTEXT - a collection of parameters that defines the filtering | |
| 71 context to be used for a collection of envelope recipient addresses. | |
| 72 The context includes such things as the list of DNSBLs to be used, and | |
| 73 the various content filtering parameters. | |
| 74 | |
| 0 | 75 <p>DNSBL - a named DNS based blocking list is defined by a dns suffix |
| 76 (e.g. sbl-xbl.spamhaus.org) and a message string that is used to | |
| 77 generate the "550 5.7.1" smtp error return code. The names of these | |
| 78 DNSBLs will be used to define the DNSBL-LISTs. | |
| 79 | |
| 80 <p>DNSBL-LIST - a named list of DNSBLs that will be used for specific | |
| 81 recipients or recipient domains. | |
| 82 | |
| 75 | 83 <p>The envelope to email address is used to find an initial filtering context. |
| 84 That context then uses the envelope from email address to find the final | |
| 85 filtering context. The envelope from email address is checked in that context | |
| 86 to see if we should whitelist or blacklist the message | |
| 0 | 87 two names (a named DNSBL-LIST, and a named ENVELOPE-FROM-MAP). If the |
| 88 recipient is not found in the configuration, the named DEFAULT | |
| 89 dnsbl-list and DEFAULT envelope-from-map will be used. When mail is | |
| 90 received for that recipient, | |
| 91 | |
| 92 <ol> | |
| 93 | |
| 75 | 94 <li>If the client has authenticated with sendmail, the mail is accepted, |
| 95 the dns lists are not checked, and the body content is not scanned. | |
| 96 | |
| 97 <li>The envelope to email address is used to find an initial filtering | |
| 98 context. We first look for a context that specified the full email address | |
| 99 in the env_to statement. If that is not found, we look for a context that | |
| 100 specified the entire domain name of the envelope recipient in the env_to | |
| 101 statement. If that is not found, we look for a context that specified the | |
| 102 user@ part of the envelope recipient in the env_to statement. If that is not | |
| 103 found, we use the first top level context defined in the config file. | |
| 0 | 104 |
| 75 | 105 <li>The initial filtering context may redirect to a child context based |
| 106 on the values in the initial context's env_from statement. We look for | |
| 107 [1) the full envelope from email address, 2) the domain name part of the | |
| 108 envelope from address, 3) the user@ part of the envelope from address] | |
| 109 in that context's env_from statement, with values that point to a child | |
| 110 context. If such an entry is found, we switch to that filtering | |
| 111 context. | |
| 112 | |
| 113 <li>We lookup [1) the full envelope from email address, 2) the domain | |
| 114 name part of the envelope from address, 3) the user@ part of the | |
| 115 envelope from address] in the filtering context env_from statement. | |
| 116 That results in one of (white, black, unknown, inherit). | |
| 117 | |
| 118 <li>If the answer is black, mail to this recipient is rejected with "no | |
| 0 | 119 such user", and the dns lists are not checked. |
| 120 | |
| 75 | 121 <li>If the answer is white, mail to this recipient is accepted and the |
| 122 dns lists are not checked. | |
| 123 | |
| 124 <li>If the answer is unknown, we don't reject yet, but the dns lists | |
| 125 will be checked, and the content may be scanned. | |
| 0 | 126 |
| 75 | 127 <li>If the answer is inherit, we repeat the envelope from search in the |
| 128 parent context. | |
| 0 | 129 |
| 75 | 130 <li>The dns lists specified in the filtering context are checked and the |
| 131 mail is rejected if any list has an A record for the standard dns based | |
| 0 | 132 lookup scheme (reversed octets of the client followed by the dns |
| 133 suffix). | |
| 134 | |
| 11 | 135 <li>If the mail has not been accepted or rejected yet, the body content |
|
56
57607387263d
updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents:
53
diff
changeset
|
136 is optionally scanned for HTTP URLs (after base64, mime and html entity |
|
57607387263d
updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents:
53
diff
changeset
|
137 decoding), and the first <configurable> host names are checked for |
| 57 | 138 their presence on the SBL. If any host name is on the SBL, and it is |
| 139 not on the "ignore" list, the mail is rejected. If we are doing body | |
| 140 content scanning, we also scan for excessive bad html tags, and if a | |
| 141 <configurable> limit is exceeded, the mail is rejected. | |
| 11 | 142 |
| 0 | 143 </ol> |
| 144 | |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
145 <hr> <center>Sendmail access vs. DNSBL</center> |
| 12 | 146 <p>With the standard sendmail.mc dnsbl FEATURE, the dnsbl checks may be |
| 147 suppressed by entries in the /etc/mail/access database. For example, | |
| 148 suppose you control a /18 of address space, and have allocated some /24s | |
| 149 to some clients. You have access entries like | |
| 0 | 150 |
| 12 | 151 <pre> |
| 152 192.168.4 OK | |
| 153 192.168.17 OK | |
| 154 </pre> | |
| 155 | |
| 156 <p>to allow those clients to smarthost thru your mail server. Now if | |
| 13 | 157 one of those clients happens get infected with a virus that turns a |
| 158 machine into an open proxy, and their 192.168.4.45 lands on the SBL-XBL, | |
| 159 you will still wind up allowing that infected machine to smarthost thru | |
| 160 your mail servers. | |
| 12 | 161 |
| 162 <p>With this DNSBL milter, the sendmail access database cannot override | |
| 163 the dnsbl checks, so that machine won't be able to send mail to or thru | |
| 15 | 164 your smarthost mail server (unless the virus/proxy can use smtp-auth). |
| 165 | |
| 166 <p>Using the standard sendmail features, you would add access entries to | |
| 167 allow hosts on your local network to relay thru your mail server. Those | |
| 168 OK entries in the sendmail access database will override all the dnsbl | |
| 169 checks. With this DNSBL milter, you will need to have the local users | |
| 170 authenticate with smtp-auth to get the same effect. You might find <a | |
| 75 | 171 href="http://www.lists.dartmouth.edu/IRIA/knowledge_base/linuxinfo/sendmail-ssl-how-to.htm"> |
| 15 | 172 these directions</a> helpful for setting up smtp-auth if you are on RH |
| 173 Linux. | |
| 12 | 174 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
175 <hr> <center>Installation and configuration</center> |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
176 <p>Usage: Note that this has ONLY been tested on Linux, specifically |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
177 RedHat Linux. In particular, this milter makes no attempt to understand |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
178 IPv6. Your mileage will vary. You will need at a minimum a C++ |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
179 compiler with a minimally thread safe STL implementation. The |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
180 distribution includes a test.cpp program. If it fails this milter won't |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
181 work. If it passes, this milter might work. |
| 0 | 182 |
| 183 Fetch <a href="http://www.five-ten-sg.com/util/dnsbl.tar.gz">dnsbl.tar.gz</a> | |
| 184 and | |
| 185 | |
| 186 <pre> | |
| 187 tar xfvz dnsbl.tar.gz | |
| 188 bash install.bash | |
| 189 </pre> | |
| 190 | |
| 191 Read and understand the contents of that install.bash script before you | |
| 192 run it. It may not be suitable for your system. Modify your | |
| 193 sendmail.mc by removing all the "FEATURE(dnsbl" lines, add the following | |
| 194 line in your sendmail.mc and rebuild the .cf file | |
| 195 | |
| 196 <pre> | |
| 50 | 197 INPUT_MAIL_FILTER(`dnsbl', `S=local:/var/run/dnsbl/dnsbl.sock, F=T, T=C:30s;S:5m;R:5m;E:5m') |
| 0 | 198 </pre> |
| 199 | |
| 200 Read the sample <a | |
| 44 | 201 href="http://www.five-ten-sg.com/dnsbl.conf">/etc/dnsbl/dnsbl.conf</a> |
| 6 | 202 file and modify it to fit your configuration. You can test your |
| 13 | 203 configuration files, and see a readable internal dump of them on stdout |
| 6 | 204 with |
| 205 | |
| 206 <pre> | |
| 44 | 207 cd /etc/dnsbl |
| 208 /usr/sbin/dnsbl -c | |
| 6 | 209 </pre> |
| 210 | |
| 75 | 211 You can check a specific envelope from/to pair with |
| 212 | |
| 213 <pre> | |
| 214 cd /etc/dnsbl | |
| 215 from="$1" # or your from address | |
| 216 to="$2" # or your to address | |
| 217 /usr/sbin/dnsbl -e "$from"'|'"$to" | |
| 218 </pre> | |
| 219 | |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
220 <hr> <center>Performance issues</center> |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
221 |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
222 <p>Consider a high volume high performance machine running sendmail. |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
223 Each sendmail process can do its own dns resolution. Typically, such |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
224 dns resolver libraries are not thread safe, and so must be protected by |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
225 some sort of mutex in a threaded environment. When we add a milter to |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
226 sendmail, we now have a collection of sendmail processes, and a |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
227 collection of milter threads. |
| 0 | 228 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
229 <p>We will be doing a lot of dns lookups per mail message, and at least |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
230 some of those will take many tens of seconds. If all this dns work is |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
231 serialized inside the milter, we have an upper limit of about 25K mail |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
232 messages per day. That is clearly not sufficient for many sites. |
| 0 | 233 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
234 <p>Since we want to do parallel dns resolution across those milter |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
235 threads, we add another collection of dns resolver processes. Each |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
236 sendmail process is talking to a milter thread over a socket, and each |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
237 milter thread is talking to a dns resolver process over another socket. |
| 6 | 238 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
239 <p>Suppose we are processing 20 messages per second, and each message |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
240 requires 20 seconds of dns work. Then we will have 400 sendmail |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
241 processes, 400 milter threads, and 400 dns resolver processes. Of |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
242 course that steady state is very unlikely to happen. |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
243 |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
244 <pre> |
| 2 | 245 $Id$ |
| 4 | 246 </pre> |
| 0 | 247 </body> |
| 248 </html> |