annotate xml/dnsbl.in @ 56:57607387263d stable-3-6

updates for 3.6, better documentation on removing content filtering, missing some files in cvs
author carl
date Wed, 08 Sep 2004 14:46:45 -0700
parents c2371bb6cf84
children 419e00901570
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
96a9758165cd Initial revision
carl
parents:
diff changeset
1 <html>
96a9758165cd Initial revision
carl
parents:
diff changeset
2
96a9758165cd Initial revision
carl
parents:
diff changeset
3 <head>
96a9758165cd Initial revision
carl
parents:
diff changeset
4 <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
56
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
5 <title>DNSBL Sendmail milter - Version 3.6</title>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
6 </head>
96a9758165cd Initial revision
carl
parents:
diff changeset
7
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
8 <center>Introduction</center>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
9 <p>This milter is released under the GPL license version 2 included in
96a9758165cd Initial revision
carl
parents:
diff changeset
10 the LICENSE file in the distribution, and also available at
96a9758165cd Initial revision
carl
parents:
diff changeset
11 <a href="http://www.gnu.org/licenses/gpl.html">http://www.gnu.org/licenses/gpl.html</a>
96a9758165cd Initial revision
carl
parents:
diff changeset
12
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
13 <p>Consider the case of a mail server that is acting as secondary MX for
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
14 a collection of clients, each of which has a collection of mail domains.
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
15 Each client may use their own collection of DNSBLs on their primary mail
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
16 server. We present here a mechanism whereby the backup mail server can
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
17 use the correct set of DNSBLs for each recipient for each message. As a
0
96a9758165cd Initial revision
carl
parents:
diff changeset
18 side-effect, it gives us the ability to customize the set of DNSBLs on a
96a9758165cd Initial revision
carl
parents:
diff changeset
19 per-recipient basis, so that fred@example.com could use SPEWS and the
96a9758165cd Initial revision
carl
parents:
diff changeset
20 SBL, where all other users @example.com use only the SBL.
96a9758165cd Initial revision
carl
parents:
diff changeset
21
27
43a4f6b3e668 add configurable host name limit and bad html tag limits.
carl
parents: 24
diff changeset
22 <p>This milter will also decode (base64, mime, html entity, url
43a4f6b3e668 add configurable host name limit and bad html tag limits.
carl
parents: 24
diff changeset
23 encodings) and scan for HTTP and HTTPS URLs and bare hostnames in the
34
fc7f8f3ea90f look for NS records on the SBL also
carl
parents: 27
diff changeset
24 body of the mail. If any of those host names have A or NS records on
fc7f8f3ea90f look for NS records on the SBL also
carl
parents: 27
diff changeset
25 the SBL (or a single configurable DNSBL), the mail will be rejected
fc7f8f3ea90f look for NS records on the SBL also
carl
parents: 27
diff changeset
26 unless previously whitelisted. This milter also counts the number of
fc7f8f3ea90f look for NS records on the SBL also
carl
parents: 27
diff changeset
27 invalid HTML tags, and can reject mail if that count exceeds your
fc7f8f3ea90f look for NS records on the SBL also
carl
parents: 27
diff changeset
28 specified limit.
11
2c206836b4cc integration work on url scanner
carl
parents: 6
diff changeset
29
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
30 <p>The DNSBL milter reads a text configuration file (dnsbl.conf) on
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
31 startup, and whenever the config file (or any of the referenced include
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
32 files) is changed. The entire configuration file is case insensitive.
0
96a9758165cd Initial revision
carl
parents:
diff changeset
33
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
34 <hr>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
35 <center>DCC Issues</center>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
36 <p>If you are also using the <a
96a9758165cd Initial revision
carl
parents:
diff changeset
37 href="http://www.rhyolite.com/anti-spam/dcc/">DCC</a> milter, there are
96a9758165cd Initial revision
carl
parents:
diff changeset
38 a few considerations. You may need to whitelist senders from the DCC
96a9758165cd Initial revision
carl
parents:
diff changeset
39 bulk detector, or from the DNS based lists. Those are two very
96a9758165cd Initial revision
carl
parents:
diff changeset
40 different reasons for whitelisting. The former is done thru the DCC
96a9758165cd Initial revision
carl
parents:
diff changeset
41 whiteclnt config file, the later is done thru the DNSBL milter config
5
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
42 file.
0
96a9758165cd Initial revision
carl
parents:
diff changeset
43
96a9758165cd Initial revision
carl
parents:
diff changeset
44 <p>You may want to blacklist some specific senders or sending domains.
96a9758165cd Initial revision
carl
parents:
diff changeset
45 This could be done thru either the DCC (on a global basis, or for a
96a9758165cd Initial revision
carl
parents:
diff changeset
46 specific single recipient). We prefer to do such blacklisting via the
13
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
47 DNSBL milter config, since it can be done for a collection of recipient
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
48 mail domains. The DCC approach has the feature that you can capture the
0
96a9758165cd Initial revision
carl
parents:
diff changeset
49 entire message in the DCC log files. The DNSBL milter approach has the
96a9758165cd Initial revision
carl
parents:
diff changeset
50 feature that the mail is rejected earlier (at RCPT TO time), and the
96a9758165cd Initial revision
carl
parents:
diff changeset
51 sending machine just gets a generic "550 5.7.1 no such user" message.
96a9758165cd Initial revision
carl
parents:
diff changeset
52
5
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
53 <p>There is an option to reference the DCC whiteclnt file (via an
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
54 include_dcc line) in the DNSBL milter config. This will import the
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
55 (env_to, env_from, and substitute mail_host) entries from the DCC config
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
56 into the DNSBL config. This allows using the DCC config as the single
13
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
57 point for white/blacklisting. When used in this manner, the whitelist
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
58 env_to entries from the DCC config become global whitelist entries in
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
59 the DNSBL config.
5
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
60
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
61 <p>Consider the case where you have multiple clients, each with their
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
62 own mail servers, and each running their own DCC milters. Each client
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
63 is using the DCC facilities for envelope from/to white/blacklisting.
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
64 Presumably you can use rsync or scp to fetch copies of your clients DCC
5
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
65 whiteclnt files on a regular basis. Your mail server, acting as a
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
66 backup MX for your clients, can use the DNSBL milter, and include those
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
67 client DCC config files. The envelope to white/blacklisting will be
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
68 global for your system, but the envelope from white/blacklisting will be
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
69 appropriately tagged and used only for the domains controlled by each of
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
70 those clients.
793ac9cc114d updates to use dcc conf files
carl
parents: 4
diff changeset
71
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
72 <hr>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
73 <center>Definitions</center>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
74 <p>DNSBL - a named DNS based blocking list is defined by a dns suffix
96a9758165cd Initial revision
carl
parents:
diff changeset
75 (e.g. sbl-xbl.spamhaus.org) and a message string that is used to
96a9758165cd Initial revision
carl
parents:
diff changeset
76 generate the "550 5.7.1" smtp error return code. The names of these
96a9758165cd Initial revision
carl
parents:
diff changeset
77 DNSBLs will be used to define the DNSBL-LISTs.
96a9758165cd Initial revision
carl
parents:
diff changeset
78
96a9758165cd Initial revision
carl
parents:
diff changeset
79 <p>DNSBL-LIST - a named list of DNSBLs that will be used for specific
96a9758165cd Initial revision
carl
parents:
diff changeset
80 recipients or recipient domains.
96a9758165cd Initial revision
carl
parents:
diff changeset
81
96a9758165cd Initial revision
carl
parents:
diff changeset
82 <p>ENVELOPE-FROM-MAP - a named collection of mappings (key->value pairs)
96a9758165cd Initial revision
carl
parents:
diff changeset
83 from envelope-from values to the WHITE, BLACK, or DEFAULT keywords. The
96a9758165cd Initial revision
carl
parents:
diff changeset
84 names of these maps will be used for specific recipients or recipient
96a9758165cd Initial revision
carl
parents:
diff changeset
85 domains.
96a9758165cd Initial revision
carl
parents:
diff changeset
86
96a9758165cd Initial revision
carl
parents:
diff changeset
87 <p>The configuration file maps each recipient (or recipient domain) to
96a9758165cd Initial revision
carl
parents:
diff changeset
88 two names (a named DNSBL-LIST, and a named ENVELOPE-FROM-MAP). If the
96a9758165cd Initial revision
carl
parents:
diff changeset
89 recipient is not found in the configuration, the named DEFAULT
96a9758165cd Initial revision
carl
parents:
diff changeset
90 dnsbl-list and DEFAULT envelope-from-map will be used. When mail is
96a9758165cd Initial revision
carl
parents:
diff changeset
91 received for that recipient,
96a9758165cd Initial revision
carl
parents:
diff changeset
92
96a9758165cd Initial revision
carl
parents:
diff changeset
93 <ol>
96a9758165cd Initial revision
carl
parents:
diff changeset
94
96a9758165cd Initial revision
carl
parents:
diff changeset
95 <li>If the client has authenticated with sendmail, the mail is accepted
96a9758165cd Initial revision
carl
parents:
diff changeset
96 and the dns lists are not checked.
96a9758165cd Initial revision
carl
parents:
diff changeset
97
96a9758165cd Initial revision
carl
parents:
diff changeset
98 <li>If either one is BLACK, mail to this recipient is rejected with "no
96a9758165cd Initial revision
carl
parents:
diff changeset
99 such user", and the dns lists are not checked.
96a9758165cd Initial revision
carl
parents:
diff changeset
100
96a9758165cd Initial revision
carl
parents:
diff changeset
101 <li>If the envelope-from-map name is WHITE, mail to this recipient is
96a9758165cd Initial revision
carl
parents:
diff changeset
102 accepted and the dns lists are not checked.
96a9758165cd Initial revision
carl
parents:
diff changeset
103
96a9758165cd Initial revision
carl
parents:
diff changeset
104 <li>If the envelope-from-map exists, the map is checked for the presence
96a9758165cd Initial revision
carl
parents:
diff changeset
105 of the sender. A WHITE or BLACK answer is definitive and the dns lists
96a9758165cd Initial revision
carl
parents:
diff changeset
106 are not checked.
96a9758165cd Initial revision
carl
parents:
diff changeset
107
96a9758165cd Initial revision
carl
parents:
diff changeset
108 <li>If the dnsbl-list name is WHITE, the dns lists are not checked and
96a9758165cd Initial revision
carl
parents:
diff changeset
109 the mail is accepted. Otherwise, the dns lists are checked and the mail
96a9758165cd Initial revision
carl
parents:
diff changeset
110 is rejected if any list has an A record for the standard dns based
96a9758165cd Initial revision
carl
parents:
diff changeset
111 lookup scheme (reversed octets of the client followed by the dns
96a9758165cd Initial revision
carl
parents:
diff changeset
112 suffix).
96a9758165cd Initial revision
carl
parents:
diff changeset
113
11
2c206836b4cc integration work on url scanner
carl
parents: 6
diff changeset
114 <li>If the mail has not been accepted or rejected yet, the body content
56
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
115 is optionally scanned for HTTP URLs (after base64, mime and html entity
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
116 decoding), and the first &lt;configurable&gt; host names are checked for
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
117 their presence on the SBL. If any host name is on the SBL, the mail is
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
118 rejected. If we are doing body content scanning, we also scan for
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
119 excessive bad html tags, and if a &lt;configurable&gt; limit is
57607387263d updates for 3.6, better documentation on removing content filtering, missing some files in cvs
carl
parents: 53
diff changeset
120 exceeded, the mail is rejected.
11
2c206836b4cc integration work on url scanner
carl
parents: 6
diff changeset
121
0
96a9758165cd Initial revision
carl
parents:
diff changeset
122 </ol>
96a9758165cd Initial revision
carl
parents:
diff changeset
123
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
124 <hr>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
125 <center>Sendmail access vs. DNSBL</center>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
126 <p>With the standard sendmail.mc dnsbl FEATURE, the dnsbl checks may be
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
127 suppressed by entries in the /etc/mail/access database. For example,
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
128 suppose you control a /18 of address space, and have allocated some /24s
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
129 to some clients. You have access entries like
0
96a9758165cd Initial revision
carl
parents:
diff changeset
130
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
131 <pre>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
132 192.168.4 OK
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
133 192.168.17 OK
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
134 </pre>
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
135
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
136 <p>to allow those clients to smarthost thru your mail server. Now if
13
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
137 one of those clients happens get infected with a virus that turns a
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
138 machine into an open proxy, and their 192.168.4.45 lands on the SBL-XBL,
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
139 you will still wind up allowing that infected machine to smarthost thru
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
140 your mail servers.
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
141
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
142 <p>With this DNSBL milter, the sendmail access database cannot override
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
143 the dnsbl checks, so that machine won't be able to send mail to or thru
15
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
144 your smarthost mail server (unless the virus/proxy can use smtp-auth).
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
145
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
146 <p>Using the standard sendmail features, you would add access entries to
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
147 allow hosts on your local network to relay thru your mail server. Those
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
148 OK entries in the sendmail access database will override all the dnsbl
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
149 checks. With this DNSBL milter, you will need to have the local users
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
150 authenticate with smtp-auth to get the same effect. You might find <a
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
151 href="http://www.ists.dartmouth.edu/IRIA/knowledge_base/linuxinfo/sendmail-ssh-how-to.htm">
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
152 these directions</a> helpful for setting up smtp-auth if you are on RH
6a21f7a3b002 add reference to starttls directions for rh8
carl
parents: 14
diff changeset
153 Linux.
12
6ac6d6b822ce fix memory leak with duplicate url host names,
carl
parents: 11
diff changeset
154
13
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
155 <hr> <center>Installation and configuration</center> <p>Usage: Note
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
156 that this has ONLY been tested on Linux, specifically RedHat Linux. In
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
157 particular, this milter makes no attempt to understand IPv6. Your
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
158 mileage will vary. You will need at a minimum a C++ compiler with a
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
159 minimally thread safe STL implementation. The distribution includes a
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
160 test.cpp program. If it fails this milter won't work. If it passes,
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
161 this milter might work.
0
96a9758165cd Initial revision
carl
parents:
diff changeset
162
96a9758165cd Initial revision
carl
parents:
diff changeset
163 Fetch <a href="http://www.five-ten-sg.com/util/dnsbl.tar.gz">dnsbl.tar.gz</a>
96a9758165cd Initial revision
carl
parents:
diff changeset
164 and
96a9758165cd Initial revision
carl
parents:
diff changeset
165
96a9758165cd Initial revision
carl
parents:
diff changeset
166 <pre>
96a9758165cd Initial revision
carl
parents:
diff changeset
167 tar xfvz dnsbl.tar.gz
96a9758165cd Initial revision
carl
parents:
diff changeset
168 bash install.bash
96a9758165cd Initial revision
carl
parents:
diff changeset
169 </pre>
96a9758165cd Initial revision
carl
parents:
diff changeset
170
96a9758165cd Initial revision
carl
parents:
diff changeset
171 Read and understand the contents of that install.bash script before you
96a9758165cd Initial revision
carl
parents:
diff changeset
172 run it. It may not be suitable for your system. Modify your
96a9758165cd Initial revision
carl
parents:
diff changeset
173 sendmail.mc by removing all the "FEATURE(dnsbl" lines, add the following
96a9758165cd Initial revision
carl
parents:
diff changeset
174 line in your sendmail.mc and rebuild the .cf file
96a9758165cd Initial revision
carl
parents:
diff changeset
175
96a9758165cd Initial revision
carl
parents:
diff changeset
176 <pre>
50
05957b18b2e6 final changes for 3.3
carl
parents: 44
diff changeset
177 INPUT_MAIL_FILTER(`dnsbl', `S=local:/var/run/dnsbl/dnsbl.sock, F=T, T=C:30s;S:5m;R:5m;E:5m')
0
96a9758165cd Initial revision
carl
parents:
diff changeset
178 </pre>
96a9758165cd Initial revision
carl
parents:
diff changeset
179
96a9758165cd Initial revision
carl
parents:
diff changeset
180 Read the sample <a
44
6b79046b18c2 changes for 3.2
carl
parents: 34
diff changeset
181 href="http://www.five-ten-sg.com/dnsbl.conf">/etc/dnsbl/dnsbl.conf</a>
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
182 file and modify it to fit your configuration. You can test your
13
2752e512fd32 finish documentation
carl
parents: 12
diff changeset
183 configuration files, and see a readable internal dump of them on stdout
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
184 with
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
185
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
186 <pre>
44
6b79046b18c2 changes for 3.2
carl
parents: 34
diff changeset
187 cd /etc/dnsbl
6b79046b18c2 changes for 3.2
carl
parents: 34
diff changeset
188 /usr/sbin/dnsbl -c
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
189 </pre>
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
190
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
191 <pre>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
192
96a9758165cd Initial revision
carl
parents:
diff changeset
193
6
cea50d98a6cf start work on content url scanner
carl
parents: 5
diff changeset
194
2
9bcd5ef11279 no message
carl
parents: 0
diff changeset
195 $Id$
4
15a7e942adec updates to use dcc conf files
carl
parents: 2
diff changeset
196 </pre>
0
96a9758165cd Initial revision
carl
parents:
diff changeset
197 </body>
96a9758165cd Initial revision
carl
parents:
diff changeset
198 </html>