Mercurial > dnsbl
annotate xml/dnsbl.in @ 65:1ab70970c8c8
need to cleanup zombie children
| author | carl |
|---|---|
| date | Sun, 16 Jan 2005 17:40:22 -0800 |
| parents | 7ad7d8b100bf |
| children | 14f0b04e2e15 |
| rev | line source |
|---|---|
| 0 | 1 <html> |
| 2 | |
| 3 <head> | |
| 4 <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> | |
|
62
7ad7d8b100bf
only keep 20% of the open resolver sockets in the ready pool.
carl
parents:
61
diff
changeset
|
5 <title>DNSBL Sendmail milter - Version 4.3</title> |
| 0 | 6 </head> |
| 7 | |
| 12 | 8 <center>Introduction</center> |
| 0 | 9 <p>This milter is released under the GPL license version 2 included in |
| 10 the LICENSE file in the distribution, and also available at | |
| 11 <a href="http://www.gnu.org/licenses/gpl.html">http://www.gnu.org/licenses/gpl.html</a> | |
| 12 | |
| 12 | 13 <p>Consider the case of a mail server that is acting as secondary MX for |
| 14 a collection of clients, each of which has a collection of mail domains. | |
| 15 Each client may use their own collection of DNSBLs on their primary mail | |
| 16 server. We present here a mechanism whereby the backup mail server can | |
| 17 use the correct set of DNSBLs for each recipient for each message. As a | |
| 0 | 18 side-effect, it gives us the ability to customize the set of DNSBLs on a |
| 19 per-recipient basis, so that fred@example.com could use SPEWS and the | |
| 20 SBL, where all other users @example.com use only the SBL. | |
| 21 | |
|
27
43a4f6b3e668
add configurable host name limit and bad html tag limits.
carl
parents:
24
diff
changeset
|
22 <p>This milter will also decode (base64, mime, html entity, url |
|
43a4f6b3e668
add configurable host name limit and bad html tag limits.
carl
parents:
24
diff
changeset
|
23 encodings) and scan for HTTP and HTTPS URLs and bare hostnames in the |
| 34 | 24 body of the mail. If any of those host names have A or NS records on |
| 25 the SBL (or a single configurable DNSBL), the mail will be rejected | |
| 26 unless previously whitelisted. This milter also counts the number of | |
| 27 invalid HTML tags, and can reject mail if that count exceeds your | |
| 28 specified limit. | |
| 11 | 29 |
| 6 | 30 <p>The DNSBL milter reads a text configuration file (dnsbl.conf) on |
| 31 startup, and whenever the config file (or any of the referenced include | |
| 32 files) is changed. The entire configuration file is case insensitive. | |
| 0 | 33 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
34 <hr> <center>DCC Issues</center> |
| 0 | 35 <p>If you are also using the <a |
| 36 href="http://www.rhyolite.com/anti-spam/dcc/">DCC</a> milter, there are | |
| 37 a few considerations. You may need to whitelist senders from the DCC | |
| 38 bulk detector, or from the DNS based lists. Those are two very | |
| 39 different reasons for whitelisting. The former is done thru the DCC | |
| 40 whiteclnt config file, the later is done thru the DNSBL milter config | |
| 5 | 41 file. |
| 0 | 42 |
| 43 <p>You may want to blacklist some specific senders or sending domains. | |
| 44 This could be done thru either the DCC (on a global basis, or for a | |
| 45 specific single recipient). We prefer to do such blacklisting via the | |
| 13 | 46 DNSBL milter config, since it can be done for a collection of recipient |
| 47 mail domains. The DCC approach has the feature that you can capture the | |
| 0 | 48 entire message in the DCC log files. The DNSBL milter approach has the |
| 49 feature that the mail is rejected earlier (at RCPT TO time), and the | |
| 50 sending machine just gets a generic "550 5.7.1 no such user" message. | |
| 51 | |
| 5 | 52 <p>There is an option to reference the DCC whiteclnt file (via an |
| 53 include_dcc line) in the DNSBL milter config. This will import the | |
| 54 (env_to, env_from, and substitute mail_host) entries from the DCC config | |
| 55 into the DNSBL config. This allows using the DCC config as the single | |
| 13 | 56 point for white/blacklisting. When used in this manner, the whitelist |
| 57 env_to entries from the DCC config become global whitelist entries in | |
| 58 the DNSBL config. | |
| 5 | 59 |
| 60 <p>Consider the case where you have multiple clients, each with their | |
| 61 own mail servers, and each running their own DCC milters. Each client | |
| 62 is using the DCC facilities for envelope from/to white/blacklisting. | |
| 6 | 63 Presumably you can use rsync or scp to fetch copies of your clients DCC |
| 5 | 64 whiteclnt files on a regular basis. Your mail server, acting as a |
| 65 backup MX for your clients, can use the DNSBL milter, and include those | |
| 66 client DCC config files. The envelope to white/blacklisting will be | |
| 67 global for your system, but the envelope from white/blacklisting will be | |
| 68 appropriately tagged and used only for the domains controlled by each of | |
| 69 those clients. | |
| 70 | |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
71 <hr> <center>Definitions</center> |
| 0 | 72 <p>DNSBL - a named DNS based blocking list is defined by a dns suffix |
| 73 (e.g. sbl-xbl.spamhaus.org) and a message string that is used to | |
| 74 generate the "550 5.7.1" smtp error return code. The names of these | |
| 75 DNSBLs will be used to define the DNSBL-LISTs. | |
| 76 | |
| 77 <p>DNSBL-LIST - a named list of DNSBLs that will be used for specific | |
| 78 recipients or recipient domains. | |
| 79 | |
| 80 <p>ENVELOPE-FROM-MAP - a named collection of mappings (key->value pairs) | |
| 81 from envelope-from values to the WHITE, BLACK, or DEFAULT keywords. The | |
| 82 names of these maps will be used for specific recipients or recipient | |
| 83 domains. | |
| 84 | |
| 85 <p>The configuration file maps each recipient (or recipient domain) to | |
| 86 two names (a named DNSBL-LIST, and a named ENVELOPE-FROM-MAP). If the | |
| 87 recipient is not found in the configuration, the named DEFAULT | |
| 88 dnsbl-list and DEFAULT envelope-from-map will be used. When mail is | |
| 89 received for that recipient, | |
| 90 | |
| 91 <ol> | |
| 92 | |
| 93 <li>If the client has authenticated with sendmail, the mail is accepted | |
| 94 and the dns lists are not checked. | |
| 95 | |
| 96 <li>If either one is BLACK, mail to this recipient is rejected with "no | |
| 97 such user", and the dns lists are not checked. | |
| 98 | |
| 99 <li>If the envelope-from-map name is WHITE, mail to this recipient is | |
| 100 accepted and the dns lists are not checked. | |
| 101 | |
| 102 <li>If the envelope-from-map exists, the map is checked for the presence | |
| 103 of the sender. A WHITE or BLACK answer is definitive and the dns lists | |
| 104 are not checked. | |
| 105 | |
| 106 <li>If the dnsbl-list name is WHITE, the dns lists are not checked and | |
| 107 the mail is accepted. Otherwise, the dns lists are checked and the mail | |
| 108 is rejected if any list has an A record for the standard dns based | |
| 109 lookup scheme (reversed octets of the client followed by the dns | |
| 110 suffix). | |
| 111 | |
| 11 | 112 <li>If the mail has not been accepted or rejected yet, the body content |
|
56
57607387263d
updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents:
53
diff
changeset
|
113 is optionally scanned for HTTP URLs (after base64, mime and html entity |
|
57607387263d
updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents:
53
diff
changeset
|
114 decoding), and the first <configurable> host names are checked for |
| 57 | 115 their presence on the SBL. If any host name is on the SBL, and it is |
| 116 not on the "ignore" list, the mail is rejected. If we are doing body | |
| 117 content scanning, we also scan for excessive bad html tags, and if a | |
| 118 <configurable> limit is exceeded, the mail is rejected. | |
| 11 | 119 |
| 0 | 120 </ol> |
| 121 | |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
122 <hr> <center>Sendmail access vs. DNSBL</center> |
| 12 | 123 <p>With the standard sendmail.mc dnsbl FEATURE, the dnsbl checks may be |
| 124 suppressed by entries in the /etc/mail/access database. For example, | |
| 125 suppose you control a /18 of address space, and have allocated some /24s | |
| 126 to some clients. You have access entries like | |
| 0 | 127 |
| 12 | 128 <pre> |
| 129 192.168.4 OK | |
| 130 192.168.17 OK | |
| 131 </pre> | |
| 132 | |
| 133 <p>to allow those clients to smarthost thru your mail server. Now if | |
| 13 | 134 one of those clients happens get infected with a virus that turns a |
| 135 machine into an open proxy, and their 192.168.4.45 lands on the SBL-XBL, | |
| 136 you will still wind up allowing that infected machine to smarthost thru | |
| 137 your mail servers. | |
| 12 | 138 |
| 139 <p>With this DNSBL milter, the sendmail access database cannot override | |
| 140 the dnsbl checks, so that machine won't be able to send mail to or thru | |
| 15 | 141 your smarthost mail server (unless the virus/proxy can use smtp-auth). |
| 142 | |
| 143 <p>Using the standard sendmail features, you would add access entries to | |
| 144 allow hosts on your local network to relay thru your mail server. Those | |
| 145 OK entries in the sendmail access database will override all the dnsbl | |
| 146 checks. With this DNSBL milter, you will need to have the local users | |
| 147 authenticate with smtp-auth to get the same effect. You might find <a | |
| 148 href="http://www.ists.dartmouth.edu/IRIA/knowledge_base/linuxinfo/sendmail-ssh-how-to.htm"> | |
| 149 these directions</a> helpful for setting up smtp-auth if you are on RH | |
| 150 Linux. | |
| 12 | 151 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
152 <hr> <center>Installation and configuration</center> |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
153 <p>Usage: Note that this has ONLY been tested on Linux, specifically |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
154 RedHat Linux. In particular, this milter makes no attempt to understand |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
155 IPv6. Your mileage will vary. You will need at a minimum a C++ |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
156 compiler with a minimally thread safe STL implementation. The |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
157 distribution includes a test.cpp program. If it fails this milter won't |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
158 work. If it passes, this milter might work. |
| 0 | 159 |
| 160 Fetch <a href="http://www.five-ten-sg.com/util/dnsbl.tar.gz">dnsbl.tar.gz</a> | |
| 161 and | |
| 162 | |
| 163 <pre> | |
| 164 tar xfvz dnsbl.tar.gz | |
| 165 bash install.bash | |
| 166 </pre> | |
| 167 | |
| 168 Read and understand the contents of that install.bash script before you | |
| 169 run it. It may not be suitable for your system. Modify your | |
| 170 sendmail.mc by removing all the "FEATURE(dnsbl" lines, add the following | |
| 171 line in your sendmail.mc and rebuild the .cf file | |
| 172 | |
| 173 <pre> | |
| 50 | 174 INPUT_MAIL_FILTER(`dnsbl', `S=local:/var/run/dnsbl/dnsbl.sock, F=T, T=C:30s;S:5m;R:5m;E:5m') |
| 0 | 175 </pre> |
| 176 | |
| 177 Read the sample <a | |
| 44 | 178 href="http://www.five-ten-sg.com/dnsbl.conf">/etc/dnsbl/dnsbl.conf</a> |
| 6 | 179 file and modify it to fit your configuration. You can test your |
| 13 | 180 configuration files, and see a readable internal dump of them on stdout |
| 6 | 181 with |
| 182 | |
| 183 <pre> | |
| 44 | 184 cd /etc/dnsbl |
| 185 /usr/sbin/dnsbl -c | |
| 6 | 186 </pre> |
| 187 | |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
188 <hr> <center>Performance issues</center> |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
189 |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
190 <p>Consider a high volume high performance machine running sendmail. |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
191 Each sendmail process can do its own dns resolution. Typically, such |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
192 dns resolver libraries are not thread safe, and so must be protected by |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
193 some sort of mutex in a threaded environment. When we add a milter to |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
194 sendmail, we now have a collection of sendmail processes, and a |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
195 collection of milter threads. |
| 0 | 196 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
197 <p>We will be doing a lot of dns lookups per mail message, and at least |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
198 some of those will take many tens of seconds. If all this dns work is |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
199 serialized inside the milter, we have an upper limit of about 25K mail |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
200 messages per day. That is clearly not sufficient for many sites. |
| 0 | 201 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
202 <p>Since we want to do parallel dns resolution across those milter |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
203 threads, we add another collection of dns resolver processes. Each |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
204 sendmail process is talking to a milter thread over a socket, and each |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
205 milter thread is talking to a dns resolver process over another socket. |
| 6 | 206 |
|
59
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
207 <p>Suppose we are processing 20 messages per second, and each message |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
208 requires 20 seconds of dns work. Then we will have 400 sendmail |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
209 processes, 400 milter threads, and 400 dns resolver processes. Of |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
210 course that steady state is very unlikely to happen. |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
211 |
|
510a511ad554
Add resolver processes to allow better performance on busy machines
carl
parents:
57
diff
changeset
|
212 <pre> |
| 2 | 213 $Id$ |
| 4 | 214 </pre> |
| 0 | 215 </body> |
| 216 </html> |