NORMAL OPERATIONS

The System Log

The system log is supported by the syslogd(8) program. All messages from sendmail are logged under the LOG_MAIL facility[7].

Format

Each line in the system log consists of a timestamp, the name of the machine that generated it (for logging from several machines over the local area network), the word sendmail:, and a message[8]. Most messages are a sequence of name = value pairs.

The two most common lines are logged when a message is processed. The first logs the receipt of a message; there will be exactly one of these per message. Some fields may be omitted if they do not contain interesting information. Fields are:

from
The envelope sender address.
size
The size of the message in bytes.
class
The class (i.e., numeric precedence) of the message.
pri
The initial message priority (used for queue sorting).
nrcpts
The number of envelope recipients for this message (after aliasing and forwarding).
msgid
The message id of the message (from the header).
proto
The protocol used to receive this message (e.g., ESMTP or UUCP)
daemon
The daemon name from the DaemonPortOptions setting.
relay
The machine from which it was received.

There is also one line logged per delivery attempt (so there can be several per message if delivery is deferred or there are multiple recipients). Fields are:

to
A comma-separated list of the recipients to this mailer.
ctladdr
The ``controlling user'', that is, the name of the user whose credentials we use for delivery.
delay
The total delay between the time this message was received and the current delivery attempt.
xdelay
The amount of time needed in this delivery attempt (normally indicative of the speed of the connection).
mailer
The name of the mailer used to deliver to this recipient.
relay
The name of the host that actually accepted (or rejected) this recipient.
dsn
The enhanced error code (RFC2034) if available.
stat
The delivery status.

Not all fields are present in all messages; for example, the relay is usually not listed for local deliveries.

Levels

If you have syslogd(8) or an equivalent installed, you will be able to do logging. There is a large amount of information that can be logged. The log is arranged as a succession of levels. At the lowest level only extremely strange situations are logged. At the highest level, even the most mundane and uninteresting events are recorded for posterity. As a convention, log levels under ten are considered generally useful; log levels above 64 are reserved for debugging purposes. Levels from 11-64 are reserved for verbose information that some sites might want.

A complete description of the log levels is given in section

Dumping State

You can ask sendmail to log a dump of the open files and the connection cache by sending it a SIGUSR1 signal. The results are logged at LOG_DEBUG priority.

The Mail Queues

Mail messages may either be delivered immediately or be held for later delivery. Held messages are placed into a holding directory called a mail queue.

A mail message may be queued for these reasons:

Queue Groups and Queue Directories

There are one or more mail queues. Each mail queue belongs to a queue group. There is always a default queue group that is called ``mqueue'' (which is where messages go by default unless otherwise specified). The directory or directories which comprise the default queue group are specified by the QueueDirectory option. There are zero or more additional named queue groups declared using the Q command in the configuration file.

By default, a queued message is placed in the queue group associated with the first recipient in the recipient list. A recipient address is mapped to a queue group as follows. First, if there is a ruleset called ``queuegroup'', and if this ruleset maps the address to a queue group name, then that queue group is chosen. That is, the argument for the ruleset is the recipient address and the result should be $# followed by the name of a queue group. Otherwise, if the mailer associated with the address specifies a queue group, then that queue group is chosen. Otherwise, the default queue group is chosen.

A message with multiple recipients will be split if different queue groups are chosen by the mapping of recipients to queue groups.

When a message is placed in a queue group, and the queue group has more than one queue, a queue is selected randomly.

If a message with multiple recipients is placed into a queue group with the 'r' option (maximum number of recipients per message) set to a positive value N, and if there are more than N recipients in the message, then the message will be split into multiple messages, each of which have at most N recipients.

Queue Runs

sendmail has two different ways to process the queue(s). The first one is to start queue runners after certain intervals (``normal'' queue runners), the second one is to keep queue runner processes around (``persistent'' queue runners). How to select either of these types is discussed in the appendix ``COMMAND LINE FLAGS''. Persistent queue runners have the advantage that no new processes need to be spawned at certain intervals; they just sleep for a specified time after they finished a queue run. Their disadvantage is that a new queue run is only started after all queue runners belonging to a group finished their tasks. In case one of the queue runners tries delivery to a slow recipient site at the end of a queue run, the next queue run may be substantially delayed. In general this should be smoothed out due to the distribution of those slow jobs, however, for sites with small number of queue entries this might introduce noticable delays. In general, persistent queue runners are only useful for sites with big queues.

Manual Intervention

Under normal conditions the mail queue will be processed transparently. However, you may find that manual intervention is sometimes necessary. For example, if a major host is down for a period of time the queue may become clogged. Although sendmail ought to recover gracefully when the host comes up, you may find performance unacceptably bad in the meantime. In that case you want to check the content of the queue and manipulate it as explained in the next two sections.

Printing the queue

The contents of the queue(s) can be printed using the mailq command (or by specifying the -bp flag to sendmail):

mailq
This will produce a listing of the queue id's, the size of the message, the date the message entered the queue, and the sender and recipients. If shared memory support is compiled in, the flag -bP can be used to print the number of entries in the queue(s), provided a process updates the data.

Forcing the queue

Sendmail should run the queue automatically at intervals. When using multiple queues, a separate process will by default be created to run each of the queues unless the queue run is initiated by a user with the verbose flag. The algorithm is to read and sort the queue, and then to attempt to process all jobs in order. When it attempts to run the job, sendmail first checks to see if the job is locked. If so, it ignores the job.

There is no attempt to insure that only one queue processor exists at any time, since there is no guarantee that a job cannot take forever to process (however, sendmail does include heuristics to try to abort jobs that are taking absurd amounts of time; technically, this violates RFC 821, but is blessed by RFC 1123). Due to the locking algorithm, it is impossible for one job to freeze the entire queue. However, an uncooperative recipient host or a program recipient that never returns can accumulate many processes in your system. Unfortunately, there is no completely general way to solve this.

In some cases, you may find that a major host going down for a couple of days may create a prohibitively large queue. This will result in sendmail spending an inordinate amount of time sorting the queue. This situation can be fixed by moving the queue to a temporary place and creating a new queue. The old queue can be run later when the offending host returns to service.

To do this, it is acceptable to move the entire queue directory:

cd /var/spool
mv mqueue omqueue; mkdir mqueue; chmod 700 mqueue
You should then kill the existing daemon (since it will still be processing in the old queue directory) and create a new daemon.

To run the old mail queue, issue the following command:

/usr/sbin/sendmail -C /etc/mail/queue.cf -q
The -C flag specifies an alternate configuration file queue.cf which should refer to the moved queue directory
O QueueDirectory=/var/spool/omqueue
and the -q flag says to just run every job in the queue. You can also specify the moved queue directory on the command line
/usr/sbin/sendmail -oQ/var/spool/omqueue -q
but this requires that you do not have queue groups in the configuration file, because those are not subdirectories of the moved directory. See the section about "Queue Group Declaration" for details; you most likely need a different configuration file to correctly deal with this problem. However, a proper configuration of queue groups should avoid filling up queue directories, so you shouldn't run into this problem. If you have a tendency toward voyeurism, you can use the -v flag to watch what is going on.

When the queue is finally emptied, you can remove the directory:

rmdir /var/spool/omqueue

Disk Based Connection Information

Sendmail stores a large amount of information about each remote system it has connected to in memory. It is possible to preserve some of this information on disk as well, by using the HostStatusDirectory option, so that it may be shared between several invocations of sendmail. This allows mail to be queued immediately or skipped during a queue run if there has been a recent failure in connecting to a remote machine.

Additionally enabling SingleThreadDelivery has the added effect of single-threading mail delivery to a destination. This can be quite helpful if the remote machine is running an SMTP server that is easily overloaded or cannot accept more than a single connection at a time, but can cause some messages to be punted to a future queue run. It also applies to all hosts, so setting this because you have one machine on site that runs some software that is easily overrun can cause mail to other hosts to be slowed down. If this option is set, you probably want to set the MinQueueAge option as well and run the queue fairly frequently; this way jobs that are skipped because another sendmail is talking to the same host will be tried again quickly rather than being delayed for a long time.

The disk based host information is stored in a subdirectory of the mqueue directory called .hoststat [9]. Removing this directory and its subdirectories has an effect similar to the purgestat command and is completely safe. However, purgestat only removes expired (Timeout.hoststatus) data. The information in these directories can be perused with the hoststat command, which will indicate the host name, the last access, and the status of that access. An asterisk in the left most column indicates that a sendmail process currently has the host locked for mail delivery.

The disk based connection information is treated the same way as memory based connection information for the purpose of timeouts. By default, information about host failures is valid for 30 minutes. This can be adjusted with the Timeout.hoststatus option.

The connection information stored on disk may be expired at any time with the purgestat command or by invoking sendmail with the -bH switch. The connection information may be viewed with the hoststat command or by invoking sendmail with the -bh switch.

The Service Switch

The implementation of certain system services such as host and user name lookup is controlled by the service switch. If the host operating system supports such a switch, and sendmail knows about it, sendmail will use the native version. Ultrix, Solaris, and DEC OSF/1 are examples of such systems[10].

If the underlying operating system does not support a service switch (e.g., SunOS 4.X, HP-UX, BSD) then sendmail will provide a stub implementation. The ServiceSwitchFile option points to the name of a file that has the service definitions. Each line has the name of a service and the possible implementations of that service. For example, the file:

hosts dns files nis
aliases files nis
will ask sendmail to look for hosts in the Domain Name System first. If the requested host name is not found, it tries local files, and if that fails it tries NIS. Similarly, when looking for aliases it will try the local files first followed by NIS.

Notice: since sendmail must access MX records for correct operation, it will use DNS if it is configured in the ServiceSwitchFile file. Hence an entry like

hosts files dns
will not avoid DNS lookups even if a host can be found in /etc/hosts.

Service switches are not completely integrated. For example, despite the fact that the host entry listed in the above example specifies to look in NIS, on SunOS this won't happen because the system implementation of gethostbyname(3) doesn't understand this.

The Alias Database

After recipient addresses are read from the SMTP connection or command line they are parsed by ruleset 0, which must resolve to a { mailer, host, address} triple. If the flags selected by the mailer include the A (aliasable) flag, the address part of the triple is looked up as the key (i.e., the left hand side) into the alias database. If there is a match, the address is deleted from the send queue and all addresses on the right hand side of the alias are added in place of the alias that was found. This is a recursive operation, so aliases found in the right hand side of the alias are similarly expanded.

The alias database exists in two forms. One is a text form, maintained in the file /etc/mail/aliases. The aliases are of the form

name: name1, name2, ...
Only local names may be aliased; e.g.,
eric@prep.ai.MIT.EDU: eric@CS.Berkeley.EDU
will not have the desired effect (except on prep.ai.MIT.EDU, and they probably don't want me)[11]. Aliases may be continued by starting any continuation lines with a space or a tab or by putting a backslash directly before the newline. Blank lines and lines beginning with a sharp sign ( #) are comments.

The second form is processed by the ndbm(3) [12] or the Berkeley DB library. This form is in the file /etc/mail/aliases.db (if using NEWDB) or /etc/mail/aliases.dir and /etc/mail/aliases.pag (if using NDBM). This is the form that sendmail actually uses to resolve aliases. This technique is used to improve performance.

The control of search order is actually set by the service switch. Essentially, the entry

O AliasFile=switch:aliases
is always added as the first alias entry; also, the first alias file name without a class (e.g., without nis: on the front) will be used as the name of the file for a ``files'' entry in the aliases switch. For example, if the configuration file contains
O AliasFile=/etc/mail/aliases
and the service switch contains
aliases nis files nisplus
then aliases will first be searched in the NIS database, then in /etc/mail/aliases, then in the NIS+ database.

You can also use NIS-based alias files. For example, the specification:

O AliasFile=/etc/mail/aliases
O AliasFile=nis:mail.aliases@my.nis.domain
will first search the /etc/mail/aliases file and then the map named mail.aliases in my.nis.domain. Warning: if you build your own NIS-based alias files, be sure to provide the -l flag to makedbm(8) to map upper case letters in the keys to lower case; otherwise, aliases with upper case letters in their names won't match incoming addresses.

Additional flags can be added after the colon exactly like a K line -- for example:

O AliasFile=nis:-N mail.aliases@my.nis.domain
will search the appropriate NIS map and always include null bytes in the key. Also:
O AliasFile=nis:-f mail.aliases@my.nis.domain
will prevent sendmail from downcasing the key before the alias lookup.

Rebuilding the alias database

The hash or dbm version of the database may be rebuilt explicitly by executing the command

newaliases
This is equivalent to giving sendmail the -bi flag:
/usr/sbin/sendmail -bi

If you have multiple aliases databases specified, the -bi flag rebuilds all the database types it understands (for example, it can rebuild NDBM databases but not NIS databases).

Potential problems

There are a number of problems that can occur with the alias database. They all result from a sendmail process accessing the DBM version while it is only partially built. This can happen under two circumstances: One process accesses the database while another process is rebuilding it, or the process rebuilding the database dies (due to being killed or a system crash) before completing the rebuild.

Sendmail has three techniques to try to relieve these problems. First, it ignores interrupts while rebuilding the database; this avoids the problem of someone aborting the process leaving a partially rebuilt database. Second, it locks the database source file during the rebuild -- but that may not work over NFS or if the file is unwritable. Third, at the end of the rebuild it adds an alias of the form

@: @
(which is not normally legal). Before sendmail will access the database, it checks to insure that this entry exists[13].

List owners

If an error occurs on sending to a certain address, say x, sendmail will look for an alias of the form owner-x to receive the errors. This is typically useful for a mailing list where the submitter of the list has no control over the maintenance of the list itself; in this case the list maintainer would be the owner of the list. For example:

unix-wizards: eric@ucbarpa, wnj@monet, nosuchuser,
sam@matisse
owner-unix-wizards: unix-wizards-request
unix-wizards-request: eric@ucbarpa
would cause eric@ucbarpa to get the error that will occur when someone sends to unix-wizards due to the inclusion of nosuchuser on the list.

List owners also cause the envelope sender address to be modified. The contents of the owner alias are used if they point to a single user, otherwise the name of the alias itself is used. For this reason, and to obey Internet conventions, the owner- address normally points at the -request address; this causes messages to go out with the typical Internet convention of using `` list-request'' as the return address.

User Information Database

This option is deprecated, use virtusertable and genericstable instead as explained in cf/README. If you have a version of sendmail with the user information database compiled in, and you have specified one or more databases using the U option, the databases will be searched for a user:maildrop entry. If found, the mail will be sent to the specified address.

Per-User Forwarding (.forward Files)

As an alternative to the alias database, any user may put a file with the name .forward in his or her home directory. If this file exists, sendmail redirects mail for that user to the list of addresses listed in the .forward file. Note that aliases are fully expanded before forward files are referenced. For example, if the home directory for user mckusick has a .forward file with contents:

mckusick@ernie
kirk@calder
then any mail arriving for mckusick will be redirected to the specified accounts.

Actually, the configuration file defines a sequence of filenames to check. By default, this is the user's .forward file, but can be defined to be more generally using the ForwardPath option. If you change this, you will have to inform your user base of the change; .forward is pretty well incorporated into the collective subconscious.

Special Header Lines

Several header lines have special interpretations defined by the configuration file. Others have interpretations built into sendmail that cannot be changed without changing the code. These built-ins are described here.

Errors-To:

If errors occur anywhere during processing, this header will cause error messages to go to the listed addresses. This is intended for mailing lists.

The Errors-To: header was created in the bad old days when UUCP didn't understand the distinction between an envelope and a header; this was a hack to provide what should now be passed as the envelope sender address. It should go away. It is only used if the UseErrorsTo option is set.

The Errors-To: header is officially deprecated and will go away in a future release.

Apparently-To:

RFC 822 requires at least one recipient field (To:, Cc:, or Bcc: line) in every message. If a message comes in with no recipients listed in the message then sendmail will adjust the header based on the NoRecipientAction option. One of the possible actions is to add an Apparently-To: header line for any recipients it is aware of.

The Apparently-To: header is non-standard and is both deprecated and strongly discouraged.

Precedence

The Precedence: header can be used as a crude control of message priority. It tweaks the sort order in the queue and can be configured to change the message timeout values. The precedence of a message also controls how delivery status notifications (DSNs) are processed for that message.

IDENT Protocol Support

Sendmail supports the IDENT protocol as defined in RFC 1413. Note that the RFC states a client should wait at least 30 seconds for a response. The default Timeout.ident is 5 seconds as many sites have adopted the practice of dropping IDENT queries. This has lead to delays processing mail. Although this enhances identification of the author of an email message by doing a ``call back'' to the originating system to include the owner of a particular TCP connection in the audit trail it is in no sense perfect; a determined forger can easily spoof the IDENT protocol. The following description is excerpted from RFC 1413:

6. Security Considerations

The information returned by this protocol is at most as trustworthy as the host providing it OR the organization operating the host. For example, a PC in an open lab has few if any controls on it to prevent a user from having this protocol return any identifier the user wants. Likewise, if the host has been compromised the information returned may be completely erroneous and misleading.

The Identification Protocol is not intended as an authorization or access control protocol. At best, it provides some additional auditing information with respect to TCP connections. At worst, it can provide misleading, incorrect, or maliciously incorrect information.

The use of the information returned by this protocol for other than auditing is strongly discouraged. Specifically, using Identification Protocol information to make access control decisions - either as the primary method (i.e., no other checks) or as an adjunct to other methods may result in a weakening of normal host security.

An Identification server may reveal information about users, entities, objects or processes which might normally be considered private. An Identification server provides service which is a rough analog of the CallerID services provided by some phone companies and many of the same privacy considerations and arguments that apply to the CallerID service apply to Identification. If you wouldn't run a "finger" server due to privacy considerations you may not want to run this protocol.

In some cases your system may not work properly with IDENT support due to a bug in the TCP/IP implementation. The symptoms will be that for some hosts the SMTP connection will be closed almost immediately. If this is true or if you do not want to use IDENT, you should set the IDENT timeout to zero; this will disable the IDENT protocol.


[Contents] [Previous] [Next]
This document was translated by troff2html v0.21 on October 10, 2001.


Claus Aßmann Please send comments to: <ca at sendmail.org>