smtpd-filters(7) — Linux manual page

NAME | DESCRIPTION | DESIGN | REPORT AND FILTER | PROTOCOL | CONFIGURATION | REPORT EVENTS | FILTER REQUESTS | SEE ALSO | HISTORY | COLOPHON

SMTPD-FILTERS(7)     Miscellaneous Information Manual   SMTPD-FILTERS(7)

NAME         top

       smtpd-filters — filtering API for the smtpd daemon

DESCRIPTION         top

       The smtpd(8) daemon provides a Simple Mail Transfer Protocol
       (SMTP) implementation, which allows ordinary machines to become
       Mail eXchangers (MX).  Some features that are commonly used by
       MX, such as delivery reporting or spam filtering, are outside the
       scope of SMTP and too complex to fit in smtpd(8).

       Because an MX may need to provide these features, smtpd(8)
       provides an API to extend its behavior through smtpd-filters.

       At runtime, smtpd(8) can report events to smtpd-filters, querying
       what it should answer to these events.  This allows the decision
       logic to rely on third-party programs.

DESIGN         top

       smtpd-filters are programs that run as unique standalone
       processes, they do not share smtpd(8) memory space.  They are
       executed by smtpd(8) at startup and expected to run in an
       infinite loop, reading events and filtering requests from
       stdin(4), writing responses to stdout(4) and logging to
       stderr(4).  They are not allowed to terminate.

       Because smtpd-filters are standalone programs that communicate
       with smtpd(8) through fd(4), they may run as different users than
       smtpd(8) and may be written in any language.  smtpd-filters must
       not use blocking I/O, they must support answering asynchronously
       to smtpd(8).

REPORT AND FILTER         top

       The API relies on two streams, report and filter.

       The report stream is a one-way stream which allows smtpd(8) to
       inform smtpd-filters in real-time about events.  Report events do
       not expect an answer from smtpd-filters; they are just meant to
       provide information.  A filter should be able to replicate the
       smtpd(8) state for a session by gathering information coming from
       report events.  No decision is ever taken by the report stream.

       The filter stream is a two-way stream which allows smtpd(8) to
       query smtpd-filters about what it should do with a session at a
       given phase.  Filter requests expect an answer from
       smtpd-filters; smtpd(8) will not let the session move forward
       until then.  A decision must always be taken by the filter
       stream.

       It is sometimes possible to rely on filter requests to gather
       information, but because a response is expected by smtpd(8), this
       is more costly than using report events.  The correct pattern for
       writing filters is to use report events to create a local state
       for a session, then use filter requests to take decisions based
       on this state.  The only case when using filter requests instead
       of report events is correct is when a decision is required for
       the filter request and there is no need for more information than
       that of the event itself.

PROTOCOL         top

       The protocol consists of human-readable lines exchanged between
       smtpd-filters and smtpd(8), through fd(4).

       The protocol begins with a handshake.  First, smtpd(8) provides
       smtpd-filters with general configuration information in the form
       of key-value lines:

             config|smtpd-version|7.5.0
             config|protocol|0.7
             config|smtp-session-timeout|300
             config|subsystem|smtp-in
             config|ready

       Then, smtpd-filters register the stream, subsystem and event they
       want to handle:

             register|report|smtp-in|link-connect
             register|ready

       Finally, smtpd(8) emits report events and filter requests,
       expecting smtpd-filters to respond or not depending on the
       stream:

             report|0.7|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25
             report|0.7|1576147242.200225|smtp-in|link-connect|7641dfb3798eb5bf|mail.openbsd.org|pass|199.185.178.25:31205|45.77.67.80:25
             report|0.7|1576148447.982572|smtp-in|link-connect|7641dfc063102cbd|mail.openbsd.org|pass|199.185.178.25:24786|45.77.67.80:25

       The character “|” may only appear in the last field of a payload,
       in which case it should be considered a regular character and not
       a separator.  No other field may contain a “|”.

       The list of subsystems and events, as well as the format of
       requests and responses, are documented in the sections below.

CONFIGURATION         top

       During the initial handshake, smtpd(8) emits a series of
       configuration keys and values.  The list is meant to be ignored
       by smtpd-filters that do not require it and consumed gracefully
       by filters that do.

       There are currently three keys:

             config|smtpd-version|7.5.0
             config|protocol|0.7
             config|smtp-session-timeout|300
             config|subsystem|smtp-in

       When smtpd(8) has sent all configuration keys, it emits the
       following line:

             config|ready

REPORT EVENTS         top

       There is currently only one subsystem supported in the API: smtp-
       in.

       Each report event is generated by smtpd(8) as a single line
       similar to the one below:

             report|0.7|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

       The format consists of a protocol prefix containing the stream,
       the protocol version, the timestamp, the subsystem, the event and
       the unique session identifier, separated by “|”:

             report|0.7|1576146008.006099|smtp-in|link-connect|7641df9771b4ed00

       It is followed by a suffix containing the event-specific
       parameters, also separated by “|”:

             mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

       The list of events and event-specific parameters for smtp-in are
       as follows:

       link-connect: rdns fcrdns src dest
               This event is generated upon connection.

               rdns contains the reverse DNS hostname for the remote end
               or an empty string if none.

               fcrdns contains the string “pass” or “fail” depending on
               if the remote end validates FCrDNS.

               src contains either the IP address and port of the source
               address, in the format “address:port”, or the path to a
               UNIX socket in the format “unix:/path”.

               dest holds either the IP address and port of the
               destination address, in the format “address:port”, or the
               path to a UNIX socket in the format “unix:/path”.

       link-greeting: hostname
               This event is generated upon display of the server
               banner.

               hostname contains the hostname displayed in the banner.

       link-identify: method identity
               This event is generated upon “HELO” or “EHLO” command
               from the client.

               method contains the string “HELO” or “EHLO” indicating
               the method used by the client.

               identity contains the identity provided by the client.

       link-tls: tls-string
               This event is generated upon successful negotiation of
               TLS.

               tls-string contains a colon-separated list of TLS
               properties including the TLS version, the cipher suite
               used by the session and the cipher strength in bits.

       link-disconnect
               This event is generated upon disconnection of the client.

       link-auth: result username
               This event is generated upon an authentication attempt by
               the client.

               result contains the string “pass”, “fail” or “error”
               depending on the result of the authentication attempt.

               username contains the username used for the
               authentication attempt.

       tx-reset: [message-id]
               This event is generated when a transaction is reset.

               If reset took place during a transaction, message-id
               contains the identifier of the transaction being reset.

       tx-begin: message-id
               This event is generated when a transaction is initiated.

               message-id contains the identifier for the transaction.

       tx-mail: message-id result address
               This event is generated when client emits “MAIL FROM”.

               message-id contains the identifier for the transaction.

               result contains “ok” if the sender was accepted,
               “permfail” if it was rejected or “tempfail” if it was
               rejected for a transient error.

               address contains the e-mail address of the sender.  The
               address is normalized and sanitized, the characters “<”
               and “>” are removed, along with any parameters to “MAIL
               FROM”.

       tx-rcpt: message-id result address
               This event is generated when client emits “RCPT TO”.

               message-id contains the identifier for the transaction.

               result contains “ok” if the recipient was accepted,
               “permfail” if it was rejected or “tempfail” if it was
               rejected for a transient error.

               address contains the e-mail address of the recipient.
               The address is normalized and sanitized, the characters
               “<” and “>” are removed, along with any parameters to
               “RCPT TO”.

       tx-envelope: message-id envelope-id
               This event is generated when an envelope is accepted.

               envelope-id contains the unique identifier for the
               envelope.

       tx-data: message-id result
               This event is generated when client has emitted “DATA”.

               message-id contains the unique identifier for the
               transaction.

               result contains “ok” if server accepted the message for
               processing, “permfail” if it has not been accepted and
               “tempfail” if a transient error prevented message
               processing.

       tx-commit: message-id message-size
               This event is generated when a transaction has been
               accepted by the server.

               message-id contains the unique identifier for the SMTP
               transaction.

               message-size contains the size of the message submitted
               in the “DATA” phase of the SMTP transaction.

       tx-rollback: message-id
               This event is generated when a transaction has been
               rejected by the server.

               message-id contains the unique identifier for the SMTP
               transaction.

       protocol-client: command
               This event is generated for every command submitted by
               the client.  It contains the raw command as received by
               the server.

               command contains the command emitted by the client to the
               server.

       protocol-server: response
               This event is generated for every response emitted by the
               server.  It contains the raw response as emitted by the
               server.

               response contains the response emitted by the server to
               the client.

       filter-report: filter-kind name message
               This event is generated when a filter emits a report.

               filter-kind may be either “builtin” or “proc” depending
               on if the filter is an smtpd(8) builtin filter or a proc
               filter implementing the API.

               name is the name of the filter that generated the report.

               message is a filter-specific message.

       filter-response: phase response [param]
               This event is generated when a filter responds to a
               filtering request.

               phase contains the phase name for the request.  The
               phases are documented in the next section.

               response contains the response of the filter to the
               request, it is either one of “proceed”, “report”,
               “reject”, “disconnect”, “junk or” “rewrite”.

               If specified, param is the parameter to the response.

       timeout
               This event is generated when a timeout happens for a
               session.

FILTER REQUESTS         top

       There is currently only one subsystem supported in the API: smtp-
       in.

       Filter requests allow smtpd(8) to query smtpd-filters about what
       to do with a session at a particular phase.  In addition, they
       allow smtpd-filters to alter the content of a message by adding,
       modifying, or suppressing lines of input in a way that is similar
       to what program like sed(1) or grep(1) would do.

       Each filter request is generated by smtpd(8) as a single line
       similar to the one below.  Fields are separated by the “|”
       character.

             filter|0.7|1576146008.006099|smtp-in|connect|7641df9771b4ed00|1ef1c203cc576e5d|mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

       The format consists of a protocol prefix containing the stream,
       the protocol version, the timestamp, the subsystem, the filtering
       phase, the unique session identifier and an opaque token that the
       filter should provide in its response:

             filter|0.7|1576146008.006099|smtp-in|connect|7641df9771b4ed00|1ef1c203cc576e5d

       It is followed by a suffix containing the phase-specific
       parameters of the filter request, also separated by “|”:

             mail.openbsd.org|pass|199.185.178.25:33174|45.77.67.80:25

       Unlike with report events, smtpd(8) expects answers from filter
       requests and will not allow a session to move forward until the
       filter has instructed smtpd(8) how to treat it.

       For all phases except “data-line”, responses must follow the same
       construct: a message of type “filter-result”, followed by the
       unique session id, the opaque token, a decision and optional
       decision-specific parameters:

             filter-result|7641df9771b4ed00|1ef1c203cc576e5d|proceed
             filter-result|7641df9771b4ed00|1ef1c203cc576e5d|reject|550 nope

       The possible decisions for a “filter-result” message are
       documented below.

       For the “data-line” phase, smtpd-filters are fed a stream of
       lines corresponding to the message to filter, terminated by a
       single dot:

             filter|0.7|1576146008.006099|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|line 1
             filter|0.7|1576146008.006103|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|line 2
             filter|0.7|1576146008.006105|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|.

       They are expected to return an output stream similarly terminated
       by a single dot.  A filter may add to, suppress, modify or echo
       back the lines it receives.  Ultimately, smtpd(8) assumes that
       the message consists of the output from smtpd-filters.

       Note that filters may be chained, and the lines that are input
       into a subsequent filter are the lines that are output from a
       previous filter.

       The response to “data-line” requests use their own construct.  A
       “filter-dataline” prefix, followed by the unique session
       identifier, the opaque token and the output line as follows:

             filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|line 1
             filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|line 2
             filter-dataline|7641df9771b4ed00|1ef1c203cc576e5d|.

       The list of events and event-specific parameters for smtp-in are
       as follows:

       connect: rdns fcrdns src dest
               This request is emitted after connection, before the
               banner is displayed.

       helo: identity
               This request is emitted after the client has emitted
               “HELO”.

       ehlo: identity
               This request is emitted after the client has emitted
               “EHLO”.

       starttls: tls-string
               This request is emitted after the client has requested
               “STARTTLS”.

       auth: auth
               This request is emitted after the client has requested
               “AUTH”.

       mail-from: address
               This request is emitted after the client has requested
               “MAIL FROM”.

       rcpt-to: address
               This request is emitted after the client has requested
               “RCPT TO”.

       data    This request is emitted after the client has requested
               “DATA”.

       data-line: line
               This request is emitted for each line of input in the
               “DATA” phase.  The lines are raw dot-escaped SMTP DATA
               input, terminated with a single dot.

       commit  This request is emitted after the final single dot is
               received.

       For every filtering phase, excepted “data-line”, the following
       decisions may be taken by a filter:

       proceed
               No action is taken, session or transaction may be passed
               to the next filter.

       junk    The session or transaction is marked as spam.  smtpd(8)
               will prepend an “X-Spam” header to the message.

       reject error
               The command is rejected with the message error.  The
               message must be a valid SMTP message including status
               code, 5xx or 4xx.

               Messages starting with a 5xx status result in a permanent
               failure, those starting with a 4xx status result in a
               temporary failure.

               Messages starting with a 421 status will result in a
               client disconnect.

       disconnect error
               The client is disconnected with the message error.  The
               message must be a valid SMTP message including status
               code, 5xx or 4xx.

               Messages starting with a 5xx status result in a permanent
               failure, those starting with a 4xx status result in a
               temporary failure.

       rewrite parameter
               The command parameter is rewritten.

               This decision allows a filter to perform a rewrite of
               client-submitted commands before they are processed by
               the SMTP engine.  parameter is expected to be a valid
               SMTP parameter for the command.

       report parameter
               Generates a report with parameter for this filter.

SEE ALSO         top

       smtpd(8)

HISTORY         top

       smtpd-filters first appeared in OpenBSD 6.6.

COLOPHON         top

       This page is part of the OpenSMTPD (a FREE implementation of the
       server-side SMTP protocol) project.  Information about the
       project can be found at https://www.opensmtpd.org/.  If you have
       a bug report for this manual page, see
       ⟨https://github.com/OpenSMTPD/OpenSMTPD/issues⟩.  This page was
       obtained from the project's upstream Git repository
       ⟨https://github.com/OpenSMTPD/OpenSMTPD.git⟩ on 2024-06-14.  (At
       that time, the date of the most recent commit that was found in
       the repository was 2024-06-09.)  If you discover any rendering
       problems in this HTML version of the page, or you believe there
       is a better or more up-to-date source for the page, or you have
       corrections or improvements to the information in this COLOPHON
       (which is not part of the original manual page), send a mail to
       [email protected]

GNU                           May 14, 2024              SMTPD-FILTERS(7)