Mercurial > dnsbl

<html>

<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<title>DNSBL Sendmail milter - Version 4.2</title>
</head>

<center>Introduction</center>
<p>This milter is released under the GPL license version 2 included in
the LICENSE file in the distribution, and also available at
<a href="http://www.gnu.org/licenses/gpl.html">http://www.gnu.org/licenses/gpl.html</a>

<p>Consider the case of a mail server that is acting as secondary MX for
a collection of clients, each of which has a collection of mail domains.
Each client may use their own collection of DNSBLs on their primary mail
server.  We present here a mechanism whereby the backup mail server can
use the correct set of DNSBLs for each recipient for each message.  As a
side-effect, it gives us the ability to customize the set of DNSBLs on a
per-recipient basis, so that fred@example.com could use SPEWS and the
SBL, where all other users @example.com use only the SBL.

<p>This milter will also decode (base64, mime, html entity, url
encodings) and scan for HTTP and HTTPS URLs and bare hostnames in the
body of the mail.  If any of those host names have A or NS records on
the SBL (or a single configurable DNSBL), the mail will be rejected
unless previously whitelisted.  This milter also counts the number of
invalid HTML tags, and can reject mail if that count exceeds your
specified limit.

<p>The DNSBL milter reads a text configuration file (dnsbl.conf) on
startup, and whenever the config file (or any of the referenced include
files) is changed.  The entire configuration file is case insensitive.

<hr> <center>DCC Issues</center>
<p>If you are also using the <a
href="http://www.rhyolite.com/anti-spam/dcc/">DCC</a> milter, there are
a few considerations.  You may need to whitelist senders from the DCC
bulk detector, or from the DNS based lists.  Those are two very
different reasons for whitelisting.  The former is done thru the DCC
whiteclnt config file, the later is done thru the DNSBL milter config
file.

<p>You may want to blacklist some specific senders or sending domains.
This could be done thru either the DCC (on a global basis, or for a
specific single recipient).  We prefer to do such blacklisting via the
DNSBL milter config, since it can be done for a collection of recipient
mail domains.  The DCC approach has the feature that you can capture the
entire message in the DCC log files.  The DNSBL milter approach has the
feature that the mail is rejected earlier (at RCPT TO time), and the
sending machine just gets a generic "550 5.7.1 no such user" message.

<p>There is an option to reference the DCC whiteclnt file (via an
include_dcc line) in the DNSBL milter config.  This will import the
(env_to, env_from, and substitute mail_host) entries from the DCC config
into the DNSBL config.  This allows using the DCC config as the single
point for white/blacklisting.  When used in this manner, the whitelist
env_to entries from the DCC config become global whitelist entries in
the DNSBL config.

<p>Consider the case where you have multiple clients, each with their
own mail servers, and each running their own DCC milters.  Each client
is using the DCC facilities for envelope from/to white/blacklisting.
Presumably you can use rsync or scp to fetch copies of your clients DCC
whiteclnt files on a regular basis.  Your mail server, acting as a
backup MX for your clients, can use the DNSBL milter, and include those
client DCC config files.  The envelope to white/blacklisting will be
global for your system, but the envelope from white/blacklisting will be
appropriately tagged and used only for the domains controlled by each of
those clients.

<hr> <center>Definitions</center>
<p>DNSBL - a named DNS based blocking list is defined by a dns suffix
(e.g. sbl-xbl.spamhaus.org) and a message string that is used to
generate the "550 5.7.1" smtp error return code.  The names of these
DNSBLs will be used to define the DNSBL-LISTs.

<p>DNSBL-LIST - a named list of DNSBLs that will be used for specific
recipients or recipient domains.

<p>ENVELOPE-FROM-MAP - a named collection of mappings (key->value pairs)
from envelope-from values to the WHITE, BLACK, or DEFAULT keywords.  The
names of these maps will be used for specific recipients or recipient
domains.

<p>The configuration file maps each recipient (or recipient domain) to
two names (a named DNSBL-LIST, and a named ENVELOPE-FROM-MAP).  If the
recipient is not found in the configuration, the named DEFAULT
dnsbl-list and DEFAULT envelope-from-map will be used.  When mail is
received for that recipient,

<ol>

<li>If the client has authenticated with sendmail, the mail is accepted
and the dns lists are not checked.

<li>If either one is BLACK, mail to this recipient is rejected with "no
such user", and the dns lists are not checked.

<li>If the envelope-from-map name is WHITE, mail to this recipient is
accepted and the dns lists are not checked.

<li>If the envelope-from-map exists, the map is checked for the presence
of the sender.  A WHITE or BLACK answer is definitive and the dns lists
are not checked.

<li>If the dnsbl-list name is WHITE, the dns lists are not checked and
the mail is accepted.  Otherwise, the dns lists are checked and the mail
is rejected if any list has an A record for the standard dns based
lookup scheme (reversed octets of the client followed by the dns
suffix).

<li>If the mail has not been accepted or rejected yet, the body content
is optionally scanned for HTTP URLs (after base64, mime and html entity
decoding), and the first &lt;configurable&gt; host names are checked for
their presence on the SBL.  If any host name is on the SBL, and it is
not on the "ignore" list, the mail is rejected.  If we are doing body
content scanning, we also scan for excessive bad html tags, and if a
&lt;configurable&gt; limit is exceeded, the mail is rejected.

</ol>

<hr> <center>Sendmail access vs. DNSBL</center>
<p>With the standard sendmail.mc dnsbl FEATURE, the dnsbl checks may be
suppressed by entries in the /etc/mail/access database.  For example,
suppose you control a /18 of address space, and have allocated some /24s
to some clients.  You have access entries like

<pre>
192.168.4   OK
192.168.17  OK
</pre>

<p>to allow those clients to smarthost thru your mail server.  Now if
one of those clients happens get infected with a virus that turns a
machine into an open proxy, and their 192.168.4.45 lands on the SBL-XBL,
you will still wind up allowing that infected machine to smarthost thru
your mail servers.

<p>With this DNSBL milter, the sendmail access database cannot override
the dnsbl checks, so that machine won't be able to send mail to or thru
your smarthost mail server (unless the virus/proxy can use smtp-auth).

<p>Using the standard sendmail features, you would add access entries to
allow hosts on your local network to relay thru your mail server.  Those
OK entries in the sendmail access database will override all the dnsbl
checks.  With this DNSBL milter, you will need to have the local users
authenticate with smtp-auth to get the same effect.  You might find <a
href="http://www.ists.dartmouth.edu/IRIA/knowledge_base/linuxinfo/sendmail-ssh-how-to.htm">
these directions</a> helpful for setting up smtp-auth if you are on RH
Linux.

<hr> <center>Installation and configuration</center>
<p>Usage:  Note that this has ONLY been tested on Linux, specifically
RedHat Linux.  In particular, this milter makes no attempt to understand
IPv6.  Your mileage will vary.  You will need at a minimum a C++
compiler with a minimally thread safe STL implementation.  The
distribution includes a test.cpp program.  If it fails this milter won't
work.  If it passes, this milter might work.

Fetch <a href="http://www.five-ten-sg.com/util/dnsbl.tar.gz">dnsbl.tar.gz</a>
and

<pre>
tar xfvz dnsbl.tar.gz
bash install.bash
</pre>

Read and understand the contents of that install.bash script before you
run it.  It may not be suitable for your system.  Modify your
sendmail.mc by removing all the "FEATURE(dnsbl" lines, add the following
line in your sendmail.mc and rebuild the .cf file

<pre>
INPUT_MAIL_FILTER(`dnsbl', `S=local:/var/run/dnsbl/dnsbl.sock, F=T, T=C:30s;S:5m;R:5m;E:5m')
</pre>

Read the sample <a
href="http://www.five-ten-sg.com/dnsbl.conf">/etc/dnsbl/dnsbl.conf</a>
file and modify it to fit your configuration.  You can test your
configuration files, and see a readable internal dump of them on stdout
with

<pre>
cd /etc/dnsbl
/usr/sbin/dnsbl -c
</pre>

<hr> <center>Performance issues</center>

<p>Consider a high volume high performance machine running sendmail.
Each sendmail process can do its own dns resolution.  Typically, such
dns resolver libraries are not thread safe, and so must be protected by
some sort of mutex in a threaded environment.  When we add a milter to
sendmail, we now have a collection of sendmail processes, and a
collection of milter threads.

<p>We will be doing a lot of dns lookups per mail message, and at least
some of those will take many tens of seconds.  If all this dns work is
serialized inside the milter, we have an upper limit of about 25K mail
messages per day.  That is clearly not sufficient for many sites.

<p>Since we want to do parallel dns resolution across those milter
threads, we add another collection of dns resolver processes.  Each
sendmail process is talking to a milter thread over a socket, and each
milter thread is talking to a dns resolver process over another socket.

<p>Suppose we are processing 20 messages per second, and each message
requires 20 seconds of dns work.  Then we will have 400 sendmail
processes, 400 milter threads, and 400 dns resolver processes.  Of
course that steady state is very unlikely to happen.

<pre>
$Id$
</pre>
</body>
</html>
author	carl
date	Sat, 08 Jan 2005 12:26:30 -0800
parents	390ed250c5d2
children	7ad7d8b100bf